2025-07-24T04:22:43.0723751Z Current runner version: '2.326.0' 2025-07-24T04:22:43.0730033Z Runner name: 'gpud4f2' 2025-07-24T04:22:43.0731079Z Runner group name: 'linux.rocm.gpu.group' 2025-07-24T04:22:43.0732175Z Machine name: 'gpud4f2' 2025-07-24T04:22:43.0735483Z ##[group]GITHUB_TOKEN Permissions 2025-07-24T04:22:43.0737663Z Contents: read 2025-07-24T04:22:43.0738703Z Metadata: read 2025-07-24T04:22:43.0739258Z ##[endgroup] 2025-07-24T04:22:43.0741458Z Secret source: Actions 2025-07-24T04:22:43.0742500Z Prepare workflow directory 2025-07-24T04:22:43.5359508Z Prepare all required actions 2025-07-24T04:22:43.5421309Z Getting action download info 2025-07-24T04:22:43.8771374Z Download action repository 'pytorch/pytorch@main' (SHA:e09d80c545d9f006f9fa28de6144531bb2af33ff) 2025-07-24T04:22:48.7766660Z Download action repository 'aws-actions/configure-aws-credentials@ececac1a45f3b08a01d2dd070d28d111c5fe6722' (SHA:ececac1a45f3b08a01d2dd070d28d111c5fe6722) 2025-07-24T04:22:49.3163658Z Download action repository 'aws-actions/amazon-ecr-login@062b18b96a7aff071d4dc91bc00c4c1a7945b076' (SHA:062b18b96a7aff071d4dc91bc00c4c1a7945b076) 2025-07-24T04:22:49.8344945Z Download action repository 'pytorch/test-infra@main' (SHA:e32c622fbd62d34f173b9f131f1eca78782b2be2) 2025-07-24T04:22:50.7396973Z Download action repository 'actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02' (SHA:ea165f8d65b6e75b540449e92b4886f43607fa02) 2025-07-24T04:22:51.4142210Z Getting action download info 2025-07-24T04:22:51.5433439Z Download action repository 'actions/checkout@v4' (SHA:11bd71901bbe5b1630ceea73d27597364c9af683) 2025-07-24T04:22:52.1747080Z Getting action download info 2025-07-24T04:22:52.2997905Z Download action repository 'nick-fields/retry@v3.0.0' (SHA:7152eba30c6575329ac0576536151aca5a72780e) 2025-07-24T04:22:52.7915571Z Getting action download info 2025-07-24T04:22:52.9292797Z Uses: pytorch/pytorch/.github/workflows/_rocm-test.yml@refs/tags/ciflow/trunk/149961 (27c8ef1839e0747b36769efb192befc9308ff93c) 2025-07-24T04:22:52.9296934Z ##[group] Inputs 2025-07-24T04:22:52.9297337Z build-environment: linux-jammy-rocm-py3.10 2025-07-24T04:22:52.9298306Z test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 2, "runner": "linux.rocm.gpu.2"}, {"config": "default", "shard": 2, "num_shards": 2, "runner": "linux.rocm.gpu.2"}, {"config": "distributed", "shard": 1, "num_shards": 1, "runner": "linux.rocm.gpu.4"}]} 2025-07-24T04:22:52.9299562Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f 2025-07-24T04:22:52.9300234Z sync-tag: 2025-07-24T04:22:52.9300971Z timeout-minutes: 300 2025-07-24T04:22:52.9301725Z tests-to-include: test_nn test_torch test_cuda test_ops test_unary_ufuncs test_binary_ufuncs test_autograd inductor/test_torchinductor distributed/test_c10d_common distributed/test_c10d_nccl 2025-07-24T04:22:52.9302465Z dashboard-tag: 2025-07-24T04:22:52.9303019Z disable-monitor: true 2025-07-24T04:22:52.9303293Z monitor-log-interval: 5 2025-07-24T04:22:52.9303605Z monitor-data-collect-interval: 1 2025-07-24T04:22:52.9303893Z ##[endgroup] 2025-07-24T04:22:52.9304264Z Complete job name: linux-jammy-rocm-py3.10 / test (default, 1, 2, linux.rocm.gpu.2) 2025-07-24T04:22:53.1273535Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2025-07-24T04:22:53.1274250Z with: 2025-07-24T04:22:53.1274447Z no-sudo: true 2025-07-24T04:22:53.1274681Z submodules: recursive 2025-07-24T04:22:53.1274936Z fetch-depth: 0 2025-07-24T04:22:53.1275288Z env: 2025-07-24T04:22:53.1275478Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:22:53.1275693Z ##[endgroup] 2025-07-24T04:22:53.1360727Z ##[group]Run echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-07-24T04:22:53.1361581Z echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-07-24T04:22:53.1396358Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:22:53.1397040Z env: 2025-07-24T04:22:53.1397226Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:22:53.1397465Z ##[endgroup] 2025-07-24T04:22:53.1618082Z ##[group]Run # Use all available CPUs for fetching 2025-07-24T04:22:53.1618504Z # Use all available CPUs for fetching 2025-07-24T04:22:53.1618792Z cd "${GITHUB_WORKSPACE}" 2025-07-24T04:22:53.1619093Z git config --global fetch.parallel 0 2025-07-24T04:22:53.1619419Z git config --global submodule.fetchJobs 0 2025-07-24T04:22:53.1619706Z  2025-07-24T04:22:53.1619998Z # Clean workspace. The default checkout action should also do this, but 2025-07-24T04:22:53.1620390Z # do it here as well just in case 2025-07-24T04:22:53.1620658Z if [[ -d .git ]]; then 2025-07-24T04:22:53.1620908Z  if [ -z "${NO_SUDO}" ]; then 2025-07-24T04:22:53.1621170Z  sudo git clean -ffdx 2025-07-24T04:22:53.1621396Z  else 2025-07-24T04:22:53.1621621Z  git clean -ffdx 2025-07-24T04:22:53.1621834Z  fi 2025-07-24T04:22:53.1622017Z fi 2025-07-24T04:22:53.1650736Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:22:53.1651072Z env: 2025-07-24T04:22:53.1651265Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:22:53.1651478Z NO_SUDO: true 2025-07-24T04:22:53.1651746Z ##[endgroup] 2025-07-24T04:22:53.6672785Z Removing .additional_ci_files/ 2025-07-24T04:22:53.6673201Z Removing build/ 2025-07-24T04:22:53.6673419Z Removing dist/ 2025-07-24T04:22:53.6673630Z Removing test/test-reports/ 2025-07-24T04:22:53.6734281Z ##[group]Run actions/checkout@v4 2025-07-24T04:22:53.6734566Z with: 2025-07-24T04:22:53.6734797Z ref: 27c8ef1839e0747b36769efb192befc9308ff93c 2025-07-24T04:22:53.6735097Z fetch-depth: 0 2025-07-24T04:22:53.6735302Z submodules: recursive 2025-07-24T04:22:53.6735525Z show-progress: false 2025-07-24T04:22:53.6735745Z repository: pytorch/pytorch 2025-07-24T04:22:53.6736155Z token: *** 2025-07-24T04:22:53.6736361Z ssh-strict: true 2025-07-24T04:22:53.6736556Z ssh-user: git 2025-07-24T04:22:53.6736782Z persist-credentials: true 2025-07-24T04:22:53.6737027Z clean: true 2025-07-24T04:22:53.6737251Z sparse-checkout-cone-mode: true 2025-07-24T04:22:53.6737500Z fetch-tags: false 2025-07-24T04:22:53.6737692Z lfs: false 2025-07-24T04:22:53.6737883Z set-safe-directory: true 2025-07-24T04:22:53.6738109Z env: 2025-07-24T04:22:53.6738295Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:22:53.6738515Z ##[endgroup] 2025-07-24T04:22:53.7843092Z Syncing repository: pytorch/pytorch 2025-07-24T04:22:53.7844677Z ##[group]Getting Git version info 2025-07-24T04:22:53.7845245Z Working directory is '/var/home/pytorchci/actions-runner/_work/pytorch/pytorch' 2025-07-24T04:22:53.7846115Z [command]/usr/bin/git version 2025-07-24T04:22:53.7898579Z git version 2.34.1 2025-07-24T04:22:53.7899602Z ##[endgroup] 2025-07-24T04:22:53.7903259Z Copying '/var/home/pytorchci/.gitconfig' to '/var/home/pytorchci/actions-runner/_work/_temp/57792783-f217-43d8-bbe4-253044ad7fa6/.gitconfig' 2025-07-24T04:22:53.7904682Z Temporarily overriding HOME='/var/home/pytorchci/actions-runner/_work/_temp/57792783-f217-43d8-bbe4-253044ad7fa6' before making global git config changes 2025-07-24T04:22:53.7905478Z Adding repository directory to the temporary git global config as a safe directory 2025-07-24T04:22:53.7912751Z [command]/usr/bin/git config --global --add safe.directory /var/home/pytorchci/actions-runner/_work/pytorch/pytorch 2025-07-24T04:22:53.7966154Z [command]/usr/bin/git config --local --get remote.origin.url 2025-07-24T04:22:53.7992896Z https://github.com/pytorch/pytorch 2025-07-24T04:22:53.8009113Z ##[group]Removing previously created refs, to avoid conflicts 2025-07-24T04:22:53.8011356Z [command]/usr/bin/git rev-parse --symbolic-full-name --verify --quiet HEAD 2025-07-24T04:22:53.8039936Z HEAD 2025-07-24T04:22:53.8092234Z ##[endgroup] 2025-07-24T04:22:53.8094654Z [command]/usr/bin/git submodule status 2025-07-24T04:22:53.8547381Z 7e1e1fe3858c63c251c637ae41a20de425dde96f android/libs/fbjni (v0.1.0-12-g7e1e1fe) 2025-07-24T04:22:53.8680010Z 4dfe081cf6bcd15db339cf2680b9281b8451eeb3 third_party/FP16 (4dfe081) 2025-07-24T04:22:53.8807027Z b408327ac2a15ec3e43352421954f5b1967701d1 third_party/FXdiv (b408327) 2025-07-24T04:22:53.8942575Z c07e3a0400713d546e0dea2d5466dd22ea389c73 third_party/NNPACK (c07e3a0) 2025-07-24T04:22:53.9007996Z 2942f167cc30c5e3a44a2aecd5b0d9c07ff61a07 third_party/NVTX (v3.1.0-263-g2942f16) 2025-07-24T04:22:53.9107304Z 1d8f600fd424278486eade7ed3e877c99f0846b1 third_party/VulkanMemoryAllocator (v2.1.0-982-g1d8f600) 2025-07-24T04:22:53.9674245Z 51a0103656eff6fc9bfd39a4597923c4b542c883 third_party/XNNPACK (remotes/origin/ds/ndk-1243-g51a0103656) 2025-07-24T04:22:53.9715277Z 01aae101b9e5e94d6c16a9514c9fb8df99c93150 third_party/aiter (v0.1.1-92-g01aae101) 2025-07-24T04:22:53.9759991Z 299e5928955cc62af9968370293b916f5130916f third_party/benchmark (v1.9.3) 2025-07-24T04:22:53.9850185Z df6023e305f389bbf7249b0c4414e649f3ad6598 third_party/composable_kernel (rocm-6.4.1-337-gdf6023e30) 2025-07-24T04:22:54.0017767Z 3af7f2c16147f3fbc6e4d717032daf505dc1652c third_party/cpp-httplib (v0.20.1) 2025-07-24T04:22:54.0205461Z 5e3d2445e6a84d9599bee2bf78edbb4d80865e1d third_party/cpuinfo (5e3d244) 2025-07-24T04:22:54.0258680Z f937055efc6d414d11f4c6577e3977fe74f35fb6 third_party/cudnn_frontend (v0.5-52-gf937055) 2025-07-24T04:22:54.0385183Z b995f933179c22d3fe0d871c3a53d11e4681950f third_party/cutlass (v4.0.0) 2025-07-24T04:22:54.0448801Z 157e88b750c452bef2ab4653fe9d1eeb151ce4c3 third_party/fbgemm (v1.2.0-186-g157e88b7) 2025-07-24T04:22:54.0563599Z 979702c87a8713a8e0a5e9fee122b90d2ef13be5 third_party/flash-attention (v2.7.4) 2025-07-24T04:22:54.0605043Z a2cd1ea3b6d3fee220106b5fed3f7ce8da9eb757 third_party/flatbuffers (v24.12.23) 2025-07-24T04:22:54.1040371Z 40626af88bd7df9a5fb80be7b25ac85b122d6c21 third_party/fmt (11.2.0) 2025-07-24T04:22:54.1196230Z 3fb5c176c17c765a3492cd2f0321b0dab712f350 third_party/gemmlowp/gemmlowp (remotes/origin/revert-87-master-135-g3fb5c17) 2025-07-24T04:22:54.1371201Z c7b7b022c124d9643957d9bd55f57ac59fce8fa2 third_party/gloo (remotes/origin/gh/c-p-i-o/1/base-33-gc7b7b02) 2025-07-24T04:22:54.1620040Z 52eb8108c5bdec04579160ae17225d66034bd723 third_party/googletest (release-1.8.0-3544-g52eb8108) 2025-07-24T04:22:54.1727624Z 719d8e6cd7f7a0e01b155657526d693acf97c2b3 third_party/ideep (pytorch-rls-v3.7.1) 2025-07-24T04:22:54.1815491Z dec1d23ca65ab069d225dfe40dea14f455170959 third_party/ittapi (v3.25.5) 2025-07-24T04:22:54.2141150Z 5e7501833f1021ce6f618572d3baf657b6319658 third_party/kineto (remotes/origin/sraikund/test-98-g5e75018) 2025-07-24T04:22:54.2179600Z cca02c2f69dd18e1f12647c1c0bdc8cf90e680c7 third_party/kleidiai (v1.8.0) 2025-07-24T04:22:54.2216898Z fbd8b99c2b828428947d70fdc046bb55609be93e third_party/mimalloc (v2.2.4) 2025-07-24T04:22:54.2257550Z 55f93686c01528224f448c19128836e7df245f72 third_party/nlohmann (v3.12.0) 2025-07-24T04:22:54.2625369Z e709452ef2bbc1d113faf678c24e6d3467696e83 third_party/onnx (v1.18.0) 2025-07-24T04:22:54.2666253Z a799f4aed9c94b765dcdaabaeab7d5e7e2310878 third_party/opentelemetry-cpp (v1.14.2) 2025-07-24T04:22:54.2709746Z 0fa0ef591e38c2758e3184c6c23e497b9f732ffa third_party/pocketfft (release_for_eigen-40-g0fa0ef5) 2025-07-24T04:22:54.3133639Z d1eca4e4b421cd2997495c4b4e65cea6be4e9b8a third_party/protobuf (v3.7.0-rc.2-1279-gd1eca4e4b) 2025-07-24T04:22:54.3259787Z 072586a71b55b7f8c584153d223e95687148a900 third_party/psimd (heads/master) 2025-07-24T04:22:54.3344681Z 4fe0e1e183925bf8cfa6aae24237e724a96479b8 third_party/pthreadpool (0.1-144-g4fe0e1e) 2025-07-24T04:22:54.3412391Z a2e59f0e7065404b44dfe92a28aca47ba1378dc4 third_party/pybind11 (v2.11.0-182-ga2e59f0e) 2025-07-24T04:22:54.3538281Z f45429b087dd7d5bc78bb40dc7cf06425c252d67 third_party/python-peachpy (remotes/origin/pre-generated) 2025-07-24T04:22:54.3644905Z 5a1d179df9cf652951b59010a2d2075372d67f68 third_party/sleef (3.8) 2025-07-24T04:22:54.3764033Z 52791a2fd214b2a9dc5759d36725909c1daa7f2e third_party/tensorpipe (remotes/origin/master) 2025-07-24T04:22:54.3788598Z ##[group]Cleaning the repository 2025-07-24T04:22:54.3796109Z [command]/usr/bin/git clean -ffdx 2025-07-24T04:22:54.4175562Z [command]/usr/bin/git reset --hard HEAD 2025-07-24T04:22:54.5285434Z HEAD is now at 41cd0d034bc update triton commit hash 2025-07-24T04:22:54.5336623Z ##[endgroup] 2025-07-24T04:22:54.5338942Z ##[group]Disabling automatic garbage collection 2025-07-24T04:22:54.5343192Z [command]/usr/bin/git config --local gc.auto 0 2025-07-24T04:22:54.5391143Z ##[endgroup] 2025-07-24T04:22:54.5391711Z ##[group]Setting up auth 2025-07-24T04:22:54.5399526Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2025-07-24T04:22:54.5440715Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2025-07-24T04:22:54.5841376Z Entering 'android/libs/fbjni' 2025-07-24T04:22:54.5915665Z Entering 'third_party/FP16' 2025-07-24T04:22:54.5989343Z Entering 'third_party/FXdiv' 2025-07-24T04:22:54.6064093Z Entering 'third_party/NNPACK' 2025-07-24T04:22:54.6141485Z Entering 'third_party/NVTX' 2025-07-24T04:22:54.6220708Z Entering 'third_party/VulkanMemoryAllocator' 2025-07-24T04:22:54.6289743Z Entering 'third_party/XNNPACK' 2025-07-24T04:22:54.6372917Z Entering 'third_party/aiter' 2025-07-24T04:22:54.6448780Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-07-24T04:22:54.6527597Z Entering 'third_party/benchmark' 2025-07-24T04:22:54.6598977Z Entering 'third_party/composable_kernel' 2025-07-24T04:22:54.6675891Z Entering 'third_party/cpp-httplib' 2025-07-24T04:22:54.6744705Z Entering 'third_party/cpuinfo' 2025-07-24T04:22:54.6816780Z Entering 'third_party/cudnn_frontend' 2025-07-24T04:22:54.6885953Z Entering 'third_party/cutlass' 2025-07-24T04:22:54.6970116Z Entering 'third_party/fbgemm' 2025-07-24T04:22:54.7040970Z Entering 'third_party/fbgemm/external/asmjit' 2025-07-24T04:22:54.7107521Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-07-24T04:22:54.7180550Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-07-24T04:22:54.7251698Z Entering 'third_party/fbgemm/external/cutlass' 2025-07-24T04:22:54.7321840Z Entering 'third_party/fbgemm/external/googletest' 2025-07-24T04:22:54.7388157Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-07-24T04:22:54.7452359Z Entering 'third_party/fbgemm/external/json' 2025-07-24T04:22:54.7526976Z Entering 'third_party/flash-attention' 2025-07-24T04:22:54.7595463Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-07-24T04:22:54.7670428Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-07-24T04:22:54.7752120Z Entering 'third_party/flatbuffers' 2025-07-24T04:22:54.7825788Z Entering 'third_party/fmt' 2025-07-24T04:22:54.7896715Z Entering 'third_party/gemmlowp/gemmlowp' 2025-07-24T04:22:54.7974236Z Entering 'third_party/gloo' 2025-07-24T04:22:54.8053469Z Entering 'third_party/googletest' 2025-07-24T04:22:54.8122781Z Entering 'third_party/ideep' 2025-07-24T04:22:54.8187942Z Entering 'third_party/ideep/mkl-dnn' 2025-07-24T04:22:54.8267652Z Entering 'third_party/ittapi' 2025-07-24T04:22:54.8344998Z Entering 'third_party/kineto' 2025-07-24T04:22:54.8417033Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-07-24T04:22:54.8491653Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-07-24T04:22:54.8557683Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-07-24T04:22:54.8625794Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-07-24T04:22:54.8696806Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-07-24T04:22:54.8766548Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-07-24T04:22:54.8836566Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-07-24T04:22:54.8904735Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-07-24T04:22:54.8967571Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-07-24T04:22:54.9034112Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-07-24T04:22:54.9101506Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-07-24T04:22:54.9169504Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-07-24T04:22:54.9237294Z Entering 'third_party/kleidiai' 2025-07-24T04:22:54.9306748Z Entering 'third_party/mimalloc' 2025-07-24T04:22:54.9379793Z Entering 'third_party/nlohmann' 2025-07-24T04:22:54.9459244Z Entering 'third_party/onnx' 2025-07-24T04:22:54.9554574Z Entering 'third_party/onnx/third_party/pybind11' 2025-07-24T04:22:54.9634102Z Entering 'third_party/opentelemetry-cpp' 2025-07-24T04:22:54.9706384Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-07-24T04:22:54.9775869Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-07-24T04:22:54.9836899Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-07-24T04:22:54.9905499Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-07-24T04:22:54.9977764Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-07-24T04:22:55.0038716Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-07-24T04:22:55.0100138Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-07-24T04:22:55.0162904Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-07-24T04:22:55.0233475Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-07-24T04:22:55.0312279Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-07-24T04:22:55.0405811Z Entering 'third_party/pocketfft' 2025-07-24T04:22:55.0476247Z Entering 'third_party/protobuf' 2025-07-24T04:22:55.0550272Z Entering 'third_party/protobuf/third_party/benchmark' 2025-07-24T04:22:55.0615263Z Entering 'third_party/protobuf/third_party/googletest' 2025-07-24T04:22:55.0685361Z Entering 'third_party/psimd' 2025-07-24T04:22:55.0754592Z Entering 'third_party/pthreadpool' 2025-07-24T04:22:55.0828067Z Entering 'third_party/pybind11' 2025-07-24T04:22:55.0899385Z Entering 'third_party/python-peachpy' 2025-07-24T04:22:55.0979037Z Entering 'third_party/sleef' 2025-07-24T04:22:55.1059161Z Entering 'third_party/tensorpipe' 2025-07-24T04:22:55.1126903Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-07-24T04:22:55.1193723Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-07-24T04:22:55.1261493Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-07-24T04:22:55.1333156Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-07-24T04:22:55.1393183Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-07-24T04:22:55.1496837Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2025-07-24T04:22:55.1538444Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2025-07-24T04:22:55.1914630Z Entering 'android/libs/fbjni' 2025-07-24T04:22:55.1987034Z Entering 'third_party/FP16' 2025-07-24T04:22:55.2061311Z Entering 'third_party/FXdiv' 2025-07-24T04:22:55.2133663Z Entering 'third_party/NNPACK' 2025-07-24T04:22:55.2212265Z Entering 'third_party/NVTX' 2025-07-24T04:22:55.2292196Z Entering 'third_party/VulkanMemoryAllocator' 2025-07-24T04:22:55.2371418Z Entering 'third_party/XNNPACK' 2025-07-24T04:22:55.2457325Z Entering 'third_party/aiter' 2025-07-24T04:22:55.2532948Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-07-24T04:22:55.2612348Z Entering 'third_party/benchmark' 2025-07-24T04:22:55.2692727Z Entering 'third_party/composable_kernel' 2025-07-24T04:22:55.2782417Z Entering 'third_party/cpp-httplib' 2025-07-24T04:22:55.2853561Z Entering 'third_party/cpuinfo' 2025-07-24T04:22:55.2934043Z Entering 'third_party/cudnn_frontend' 2025-07-24T04:22:55.3012408Z Entering 'third_party/cutlass' 2025-07-24T04:22:55.3092394Z Entering 'third_party/fbgemm' 2025-07-24T04:22:55.3155326Z Entering 'third_party/fbgemm/external/asmjit' 2025-07-24T04:22:55.3225086Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-07-24T04:22:55.3303164Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-07-24T04:22:55.3367821Z Entering 'third_party/fbgemm/external/cutlass' 2025-07-24T04:22:55.3444050Z Entering 'third_party/fbgemm/external/googletest' 2025-07-24T04:22:55.3506532Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-07-24T04:22:55.3575491Z Entering 'third_party/fbgemm/external/json' 2025-07-24T04:22:55.3651815Z Entering 'third_party/flash-attention' 2025-07-24T04:22:55.3721859Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-07-24T04:22:55.3811150Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-07-24T04:22:55.3891368Z Entering 'third_party/flatbuffers' 2025-07-24T04:22:55.3956978Z Entering 'third_party/fmt' 2025-07-24T04:22:55.4029916Z Entering 'third_party/gemmlowp/gemmlowp' 2025-07-24T04:22:55.4104342Z Entering 'third_party/gloo' 2025-07-24T04:22:55.4176801Z Entering 'third_party/googletest' 2025-07-24T04:22:55.4255385Z Entering 'third_party/ideep' 2025-07-24T04:22:55.4322139Z Entering 'third_party/ideep/mkl-dnn' 2025-07-24T04:22:55.4403116Z Entering 'third_party/ittapi' 2025-07-24T04:22:55.4474016Z Entering 'third_party/kineto' 2025-07-24T04:22:55.4544933Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-07-24T04:22:55.4613654Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-07-24T04:22:55.4677723Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-07-24T04:22:55.4746870Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-07-24T04:22:55.4815971Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-07-24T04:22:55.4876892Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-07-24T04:22:55.4952202Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-07-24T04:22:55.5019986Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-07-24T04:22:55.5090190Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-07-24T04:22:55.5153969Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-07-24T04:22:55.5224642Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-07-24T04:22:55.5296787Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-07-24T04:22:55.5364555Z Entering 'third_party/kleidiai' 2025-07-24T04:22:55.5433992Z Entering 'third_party/mimalloc' 2025-07-24T04:22:55.5507992Z Entering 'third_party/nlohmann' 2025-07-24T04:22:55.5582039Z Entering 'third_party/onnx' 2025-07-24T04:22:55.5665877Z Entering 'third_party/onnx/third_party/pybind11' 2025-07-24T04:22:55.5744721Z Entering 'third_party/opentelemetry-cpp' 2025-07-24T04:22:55.5819271Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-07-24T04:22:55.5886807Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-07-24T04:22:55.5953874Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-07-24T04:22:55.6020828Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-07-24T04:22:55.6094730Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-07-24T04:22:55.6158220Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-07-24T04:22:55.6222328Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-07-24T04:22:55.6288492Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-07-24T04:22:55.6359815Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-07-24T04:22:55.6429806Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-07-24T04:22:55.6524891Z Entering 'third_party/pocketfft' 2025-07-24T04:22:55.6597731Z Entering 'third_party/protobuf' 2025-07-24T04:22:55.6672933Z Entering 'third_party/protobuf/third_party/benchmark' 2025-07-24T04:22:55.6742256Z Entering 'third_party/protobuf/third_party/googletest' 2025-07-24T04:22:55.6807128Z Entering 'third_party/psimd' 2025-07-24T04:22:55.6875431Z Entering 'third_party/pthreadpool' 2025-07-24T04:22:55.6949495Z Entering 'third_party/pybind11' 2025-07-24T04:22:55.7021105Z Entering 'third_party/python-peachpy' 2025-07-24T04:22:55.7094787Z Entering 'third_party/sleef' 2025-07-24T04:22:55.7162590Z Entering 'third_party/tensorpipe' 2025-07-24T04:22:55.7230646Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-07-24T04:22:55.7293711Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-07-24T04:22:55.7369759Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-07-24T04:22:55.7430126Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-07-24T04:22:55.7490557Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-07-24T04:22:55.7589057Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2025-07-24T04:22:55.7646673Z ##[endgroup] 2025-07-24T04:22:55.7647268Z ##[group]Fetching the repository 2025-07-24T04:22:55.7654480Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --no-recurse-submodules origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2025-07-24T04:22:56.4644640Z From https://github.com/pytorch/pytorch 2025-07-24T04:22:56.4645431Z - [deleted] (none) -> ciflow/inductor/158997 2025-07-24T04:22:56.5579630Z - [deleted] (none) -> ciflow/trunk/158997 2025-07-24T04:22:57.6155783Z 67d3299aa9b..b06d46043b7 gh/Sidharth123-cpu/44/head -> origin/gh/Sidharth123-cpu/44/head 2025-07-24T04:22:57.6164340Z + 45fba11f1a4...2d8b2d70bdd gh/Sidharth123-cpu/44/orig -> origin/gh/Sidharth123-cpu/44/orig (forced update) 2025-07-24T04:22:57.6196849Z f4af658fa20..dad92517389 gh/benjaminglass1/79/base -> origin/gh/benjaminglass1/79/base 2025-07-24T04:22:57.6199986Z 52da9230dfb..27c8ef1839e gh/benjaminglass1/79/head -> origin/gh/benjaminglass1/79/head 2025-07-24T04:22:57.6204025Z + 46c75b7f6b7...c1380dc4c14 gh/benjaminglass1/79/orig -> origin/gh/benjaminglass1/79/orig (forced update) 2025-07-24T04:22:57.6206622Z d039700eb01..8d9fd35a226 gh/benjaminglass1/89/base -> origin/gh/benjaminglass1/89/base 2025-07-24T04:22:57.6209950Z 773e031adce..091eba09e5d gh/benjaminglass1/89/head -> origin/gh/benjaminglass1/89/head 2025-07-24T04:22:57.6214103Z + 236813478bb...954dbb9e140 gh/benjaminglass1/89/orig -> origin/gh/benjaminglass1/89/orig (forced update) 2025-07-24T04:22:57.6217829Z 56e4568bae8..3b7f39e1133 gh/benjaminglass1/93/base -> origin/gh/benjaminglass1/93/base 2025-07-24T04:22:57.6220892Z 019403bc448..a690e8642e3 gh/benjaminglass1/93/head -> origin/gh/benjaminglass1/93/head 2025-07-24T04:22:57.6223367Z + aad5ac72896...4926aacc053 gh/benjaminglass1/93/orig -> origin/gh/benjaminglass1/93/orig (forced update) 2025-07-24T04:22:57.6255477Z + b1bf4598b20...8c532d7320f gh/fffrog/120/orig -> origin/gh/fffrog/120/orig (forced update) 2025-07-24T04:22:57.6258239Z 0b6c4d78244..662f4b47362 gh/fffrog/121/head -> origin/gh/fffrog/121/head 2025-07-24T04:22:57.6261410Z + c3cd9cf9273...25f61c107b3 gh/fffrog/121/orig -> origin/gh/fffrog/121/orig (forced update) 2025-07-24T04:22:57.6264464Z 2cf131a53f9..e9b67f5334a gh/fffrog/122/base -> origin/gh/fffrog/122/base 2025-07-24T04:22:57.6267451Z 1de5a4948da..c86ecb1fa2f gh/fffrog/122/head -> origin/gh/fffrog/122/head 2025-07-24T04:22:57.6270686Z + fb9bc1325a6...6a3bc42980f gh/fffrog/122/orig -> origin/gh/fffrog/122/orig (forced update) 2025-07-24T04:22:57.6305310Z c447bac5ce3..565608a7081 gh/kwen2501/197/base -> origin/gh/kwen2501/197/base 2025-07-24T04:22:57.6308721Z 87a5be4ecaf..376c425c279 gh/kwen2501/197/head -> origin/gh/kwen2501/197/head 2025-07-24T04:22:57.6312305Z + bce8057c719...fa047de8a2d gh/kwen2501/197/orig -> origin/gh/kwen2501/197/orig (forced update) 2025-07-24T04:22:57.6354505Z 07df6ba7f55..e09d80c545d main -> origin/main 2025-07-24T04:22:57.6370819Z t [tag update] ciflow/inductor-rocm/158074 -> ciflow/inductor-rocm/158074 2025-07-24T04:22:57.6372978Z t [tag update] ciflow/inductor/149961 -> ciflow/inductor/149961 2025-07-24T04:22:57.6374962Z t [tag update] ciflow/inductor/155877 -> ciflow/inductor/155877 2025-07-24T04:22:57.6377485Z t [tag update] ciflow/inductor/158074 -> ciflow/inductor/158074 2025-07-24T04:22:57.6381312Z t [tag update] ciflow/inductor/159011 -> ciflow/inductor/159011 2025-07-24T04:22:57.6384223Z t [tag update] ciflow/rocm/155877 -> ciflow/rocm/155877 2025-07-24T04:22:57.6386070Z t [tag update] ciflow/rocm/158074 -> ciflow/rocm/158074 2025-07-24T04:22:57.6388924Z t [tag update] ciflow/trunk/149961 -> ciflow/trunk/149961 2025-07-24T04:22:57.6391991Z t [tag update] ciflow/trunk/158441 -> ciflow/trunk/158441 2025-07-24T04:22:57.6394559Z t [tag update] ciflow/trunk/158938 -> ciflow/trunk/158938 2025-07-24T04:22:57.6440055Z * [new tag] trunk/e09d80c545d9f006f9fa28de6144531bb2af33ff -> trunk/e09d80c545d9f006f9fa28de6144531bb2af33ff 2025-07-24T04:22:57.7792678Z [command]/usr/bin/git rev-parse --verify --quiet 27c8ef1839e0747b36769efb192befc9308ff93c^{object} 2025-07-24T04:22:57.7878733Z 27c8ef1839e0747b36769efb192befc9308ff93c 2025-07-24T04:22:57.7888474Z ##[endgroup] 2025-07-24T04:22:57.7889083Z ##[group]Determining the checkout info 2025-07-24T04:22:57.7889695Z ##[endgroup] 2025-07-24T04:22:57.7893413Z [command]/usr/bin/git sparse-checkout disable 2025-07-24T04:22:57.8118164Z [command]/usr/bin/git config --local --unset-all extensions.worktreeConfig 2025-07-24T04:22:57.8156156Z ##[group]Checking out the ref 2025-07-24T04:22:57.8159142Z [command]/usr/bin/git checkout --progress --force 27c8ef1839e0747b36769efb192befc9308ff93c 2025-07-24T04:22:58.1337921Z Previous HEAD position was 41cd0d034bc update triton commit hash 2025-07-24T04:22:58.1357865Z HEAD is now at 27c8ef1839e Update 2025-07-24T04:22:58.1473984Z ##[endgroup] 2025-07-24T04:22:58.1474871Z ##[group]Setting up auth for fetching submodules 2025-07-24T04:22:58.1479205Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2025-07-24T04:22:58.1531617Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2025-07-24T04:22:58.1572127Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2025-07-24T04:22:58.1615896Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2025-07-24T04:22:58.1655058Z ##[endgroup] 2025-07-24T04:22:58.1655651Z ##[group]Fetching submodules 2025-07-24T04:22:58.1658060Z [command]/usr/bin/git submodule sync --recursive 2025-07-24T04:22:58.2058770Z Synchronizing submodule url for 'android/libs/fbjni' 2025-07-24T04:22:58.2115550Z Synchronizing submodule url for 'third_party/FP16' 2025-07-24T04:22:58.2174252Z Synchronizing submodule url for 'third_party/FXdiv' 2025-07-24T04:22:58.2233555Z Synchronizing submodule url for 'third_party/NNPACK' 2025-07-24T04:22:58.2294267Z Synchronizing submodule url for 'third_party/NVTX' 2025-07-24T04:22:58.2356162Z Synchronizing submodule url for 'third_party/VulkanMemoryAllocator' 2025-07-24T04:22:58.2415581Z Synchronizing submodule url for 'third_party/XNNPACK' 2025-07-24T04:22:58.2493357Z Synchronizing submodule url for 'third_party/aiter' 2025-07-24T04:22:58.2551002Z Synchronizing submodule url for 'third_party/aiter/3rdparty/composable_kernel' 2025-07-24T04:22:58.2620758Z Synchronizing submodule url for 'third_party/benchmark' 2025-07-24T04:22:58.2682946Z Synchronizing submodule url for 'third_party/composable_kernel' 2025-07-24T04:22:58.2756960Z Synchronizing submodule url for 'third_party/cpp-httplib' 2025-07-24T04:22:58.2815808Z Synchronizing submodule url for 'third_party/cpuinfo' 2025-07-24T04:22:58.2876889Z Synchronizing submodule url for 'third_party/cudnn_frontend' 2025-07-24T04:22:58.2937089Z Synchronizing submodule url for 'third_party/cutlass' 2025-07-24T04:22:58.3007768Z Synchronizing submodule url for 'third_party/fbgemm' 2025-07-24T04:22:58.3063744Z Synchronizing submodule url for 'third_party/fbgemm/external/asmjit' 2025-07-24T04:22:58.3117525Z Synchronizing submodule url for 'third_party/fbgemm/external/composable_kernel' 2025-07-24T04:22:58.3179545Z Synchronizing submodule url for 'third_party/fbgemm/external/cpuinfo' 2025-07-24T04:22:58.3236016Z Synchronizing submodule url for 'third_party/fbgemm/external/cutlass' 2025-07-24T04:22:58.3300753Z Synchronizing submodule url for 'third_party/fbgemm/external/googletest' 2025-07-24T04:22:58.3352997Z Synchronizing submodule url for 'third_party/fbgemm/external/hipify_torch' 2025-07-24T04:22:58.3406405Z Synchronizing submodule url for 'third_party/fbgemm/external/json' 2025-07-24T04:22:58.3463350Z Synchronizing submodule url for 'third_party/flash-attention' 2025-07-24T04:22:58.3521118Z Synchronizing submodule url for 'third_party/flash-attention/csrc/composable_kernel' 2025-07-24T04:22:58.3581602Z Synchronizing submodule url for 'third_party/flash-attention/csrc/cutlass' 2025-07-24T04:22:58.3651133Z Synchronizing submodule url for 'third_party/flatbuffers' 2025-07-24T04:22:58.3716172Z Synchronizing submodule url for 'third_party/fmt' 2025-07-24T04:22:58.3777202Z Synchronizing submodule url for 'third_party/gemmlowp/gemmlowp' 2025-07-24T04:22:58.3836137Z Synchronizing submodule url for 'third_party/gloo' 2025-07-24T04:22:58.3896377Z Synchronizing submodule url for 'third_party/googletest' 2025-07-24T04:22:58.3956059Z Synchronizing submodule url for 'third_party/ideep' 2025-07-24T04:22:58.4013382Z Synchronizing submodule url for 'third_party/ideep/mkl-dnn' 2025-07-24T04:22:58.4083943Z Synchronizing submodule url for 'third_party/ittapi' 2025-07-24T04:22:58.4141441Z Synchronizing submodule url for 'third_party/kineto' 2025-07-24T04:22:58.4201723Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog' 2025-07-24T04:22:58.4253641Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-07-24T04:22:58.4308724Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-07-24T04:22:58.4365537Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-07-24T04:22:58.4419324Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-07-24T04:22:58.4467699Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-07-24T04:22:58.4531066Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-07-24T04:22:58.4587428Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-07-24T04:22:58.4646814Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-07-24T04:22:58.4699487Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-07-24T04:22:58.4758644Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/fmt' 2025-07-24T04:22:58.4813978Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/googletest' 2025-07-24T04:22:58.4875763Z Synchronizing submodule url for 'third_party/kleidiai' 2025-07-24T04:22:58.4938961Z Synchronizing submodule url for 'third_party/mimalloc' 2025-07-24T04:22:58.4998561Z Synchronizing submodule url for 'third_party/nlohmann' 2025-07-24T04:22:58.5060042Z Synchronizing submodule url for 'third_party/onnx' 2025-07-24T04:22:58.5131300Z Synchronizing submodule url for 'third_party/onnx/third_party/pybind11' 2025-07-24T04:22:58.5200767Z Synchronizing submodule url for 'third_party/opentelemetry-cpp' 2025-07-24T04:22:58.5257072Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-07-24T04:22:58.5310537Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/googletest' 2025-07-24T04:22:58.5368528Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-07-24T04:22:58.5419160Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-07-24T04:22:58.5474442Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-07-24T04:22:58.5529038Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-07-24T04:22:58.5578935Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-07-24T04:22:58.5628247Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-07-24T04:22:58.5681064Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-07-24T04:22:58.5740801Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-07-24T04:22:58.5818885Z Synchronizing submodule url for 'third_party/pocketfft' 2025-07-24T04:22:58.5878828Z Synchronizing submodule url for 'third_party/protobuf' 2025-07-24T04:22:58.5939581Z Synchronizing submodule url for 'third_party/protobuf/third_party/benchmark' 2025-07-24T04:22:58.5991696Z Synchronizing submodule url for 'third_party/protobuf/third_party/googletest' 2025-07-24T04:22:58.6055363Z Synchronizing submodule url for 'third_party/psimd' 2025-07-24T04:22:58.6114695Z Synchronizing submodule url for 'third_party/pthreadpool' 2025-07-24T04:22:58.6172397Z Synchronizing submodule url for 'third_party/pybind11' 2025-07-24T04:22:58.6229277Z Synchronizing submodule url for 'third_party/python-peachpy' 2025-07-24T04:22:58.6292208Z Synchronizing submodule url for 'third_party/sleef' 2025-07-24T04:22:58.6350669Z Synchronizing submodule url for 'third_party/tensorpipe' 2025-07-24T04:22:58.6408028Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/googletest' 2025-07-24T04:22:58.6459760Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/libnop' 2025-07-24T04:22:58.6512788Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/libuv' 2025-07-24T04:22:58.6569789Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/pybind11' 2025-07-24T04:22:58.6618764Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-07-24T04:22:58.6702798Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2025-07-24T04:22:58.7456254Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2025-07-24T04:22:58.7839142Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2025-07-24T04:22:58.8205750Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2025-07-24T04:22:58.8581538Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2025-07-24T04:22:58.8960372Z Submodule path 'third_party/NVTX': checked out '2942f167cc30c5e3a44a2aecd5b0d9c07ff61a07' 2025-07-24T04:22:58.9333121Z Submodule path 'third_party/VulkanMemoryAllocator': checked out '1d8f600fd424278486eade7ed3e877c99f0846b1' 2025-07-24T04:22:58.9896652Z Submodule path 'third_party/XNNPACK': checked out '51a0103656eff6fc9bfd39a4597923c4b542c883' 2025-07-24T04:22:59.0381733Z Submodule path 'third_party/aiter': checked out '01aae101b9e5e94d6c16a9514c9fb8df99c93150' 2025-07-24T04:22:59.0903146Z Submodule path 'third_party/aiter/3rdparty/composable_kernel': checked out 'cffe8fa2a442ac8e80dd236a1a5d24fe3d7e0cbf' 2025-07-24T04:22:59.1293366Z Submodule path 'third_party/benchmark': checked out '299e5928955cc62af9968370293b916f5130916f' 2025-07-24T04:22:59.3044729Z Submodule path 'third_party/composable_kernel': checked out '434d19f696da62c12b5372b32cbc9ba968588d7e' 2025-07-24T04:22:59.3451418Z Submodule path 'third_party/cpp-httplib': checked out '3af7f2c16147f3fbc6e4d717032daf505dc1652c' 2025-07-24T04:22:59.3827858Z Submodule path 'third_party/cpuinfo': checked out '5e3d2445e6a84d9599bee2bf78edbb4d80865e1d' 2025-07-24T04:22:59.4224667Z Submodule path 'third_party/cudnn_frontend': checked out 'f937055efc6d414d11f4c6577e3977fe74f35fb6' 2025-07-24T04:22:59.4698233Z Submodule path 'third_party/cutlass': checked out 'b995f933179c22d3fe0d871c3a53d11e4681950f' 2025-07-24T04:22:59.5185773Z Submodule path 'third_party/fbgemm': checked out '157e88b750c452bef2ab4653fe9d1eeb151ce4c3' 2025-07-24T04:22:59.5510458Z Submodule path 'third_party/fbgemm/external/asmjit': checked out 'e5d7c0bd5d9aec44d68830187138149e6a8c4e32' 2025-07-24T04:22:59.5914306Z Submodule path 'third_party/fbgemm/external/composable_kernel': checked out '4a61bdd4bd4ed730e078aebc7c0fcf046ff29406' 2025-07-24T04:22:59.6270309Z Submodule path 'third_party/fbgemm/external/cpuinfo': checked out '6543fec09b2f04ac4a666882998b534afc9c1349' 2025-07-24T04:22:59.6691197Z Submodule path 'third_party/fbgemm/external/cutlass': checked out '3ed8d2ec4ba35ef5d9d8353826209b6f868f63d3' 2025-07-24T04:22:59.7040702Z Submodule path 'third_party/fbgemm/external/googletest': checked out 'f8d7d77c06936315286eb55f8de22cd23c188571' 2025-07-24T04:22:59.7363757Z Submodule path 'third_party/fbgemm/external/hipify_torch': checked out 'a4337c69fe0e2552a7b7b0669178926beeed828c' 2025-07-24T04:22:59.7755554Z Submodule path 'third_party/fbgemm/external/json': checked out '9cca280a4d0ccf0c08f47a99aa71d1b0e52f8d03' 2025-07-24T04:22:59.8174584Z Submodule path 'third_party/flash-attention': checked out '979702c87a8713a8e0a5e9fee122b90d2ef13be5' 2025-07-24T04:22:59.8693436Z Submodule path 'third_party/flash-attention/csrc/composable_kernel': checked out '888317e698e9803c62bd38568abc9e05d7709f33' 2025-07-24T04:22:59.9135602Z Submodule path 'third_party/flash-attention/csrc/cutlass': checked out 'c506e16788cb08416a4a57e11a9067beeee29420' 2025-07-24T04:22:59.9577401Z Submodule path 'third_party/flatbuffers': checked out 'a2cd1ea3b6d3fee220106b5fed3f7ce8da9eb757' 2025-07-24T04:22:59.9969875Z Submodule path 'third_party/fmt': checked out '40626af88bd7df9a5fb80be7b25ac85b122d6c21' 2025-07-24T04:23:00.0335331Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2025-07-24T04:23:00.0705887Z Submodule path 'third_party/gloo': checked out 'c7b7b022c124d9643957d9bd55f57ac59fce8fa2' 2025-07-24T04:23:00.1085942Z Submodule path 'third_party/googletest': checked out '52eb8108c5bdec04579160ae17225d66034bd723' 2025-07-24T04:23:00.1455842Z Submodule path 'third_party/ideep': checked out '719d8e6cd7f7a0e01b155657526d693acf97c2b3' 2025-07-24T04:23:00.1983797Z Submodule path 'third_party/ideep/mkl-dnn': checked out '8d263e693366ef8db40acc569cc7d8edf644556d' 2025-07-24T04:23:00.2369651Z Submodule path 'third_party/ittapi': checked out 'dec1d23ca65ab069d225dfe40dea14f455170959' 2025-07-24T04:23:00.2756478Z Submodule path 'third_party/kineto': checked out '5e7501833f1021ce6f618572d3baf657b6319658' 2025-07-24T04:23:00.3120162Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2025-07-24T04:23:00.3480615Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2025-07-24T04:23:00.3836534Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2025-07-24T04:23:00.4202114Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2025-07-24T04:23:00.4545207Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2025-07-24T04:23:00.4875826Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2025-07-24T04:23:00.5219754Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2025-07-24T04:23:00.5565640Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2025-07-24T04:23:00.5961773Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2025-07-24T04:23:00.6292014Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2025-07-24T04:23:00.6646750Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2025-07-24T04:23:00.6972612Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2025-07-24T04:23:00.7355437Z Submodule path 'third_party/kleidiai': checked out 'cca02c2f69dd18e1f12647c1c0bdc8cf90e680c7' 2025-07-24T04:23:00.7745410Z Submodule path 'third_party/mimalloc': checked out 'fbd8b99c2b828428947d70fdc046bb55609be93e' 2025-07-24T04:23:00.8165359Z Submodule path 'third_party/nlohmann': checked out '55f93686c01528224f448c19128836e7df245f72' 2025-07-24T04:23:00.8727130Z Submodule path 'third_party/onnx': checked out 'e709452ef2bbc1d113faf678c24e6d3467696e83' 2025-07-24T04:23:00.9137555Z Submodule path 'third_party/onnx/third_party/pybind11': checked out 'a2e59f0e7065404b44dfe92a28aca47ba1378dc4' 2025-07-24T04:23:00.9586266Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2025-07-24T04:23:00.9915338Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2025-07-24T04:23:01.0249446Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2025-07-24T04:23:01.0563759Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2025-07-24T04:23:01.0942831Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2025-07-24T04:23:01.1275301Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2025-07-24T04:23:01.1601676Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2025-07-24T04:23:01.1935298Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2025-07-24T04:23:01.2301489Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2025-07-24T04:23:01.2650887Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2025-07-24T04:23:01.3196060Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2025-07-24T04:23:01.3599843Z Submodule path 'third_party/pocketfft': checked out '0fa0ef591e38c2758e3184c6c23e497b9f732ffa' 2025-07-24T04:23:01.4184401Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2025-07-24T04:23:01.4522519Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2025-07-24T04:23:01.4860868Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2025-07-24T04:23:01.5210668Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2025-07-24T04:23:01.5562845Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2025-07-24T04:23:01.5951085Z Submodule path 'third_party/pybind11': checked out 'a2e59f0e7065404b44dfe92a28aca47ba1378dc4' 2025-07-24T04:23:01.6313704Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2025-07-24T04:23:01.6685011Z Submodule path 'third_party/sleef': checked out '5a1d179df9cf652951b59010a2d2075372d67f68' 2025-07-24T04:23:01.7054830Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2025-07-24T04:23:01.7392073Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2025-07-24T04:23:01.7711626Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2025-07-24T04:23:01.8212809Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2025-07-24T04:23:01.8574984Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2025-07-24T04:23:01.8891008Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2025-07-24T04:23:01.9026678Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2025-07-24T04:23:01.9397167Z Entering 'android/libs/fbjni' 2025-07-24T04:23:01.9460398Z Entering 'third_party/FP16' 2025-07-24T04:23:01.9530226Z Entering 'third_party/FXdiv' 2025-07-24T04:23:01.9593718Z Entering 'third_party/NNPACK' 2025-07-24T04:23:01.9660661Z Entering 'third_party/NVTX' 2025-07-24T04:23:01.9728102Z Entering 'third_party/VulkanMemoryAllocator' 2025-07-24T04:23:01.9791209Z Entering 'third_party/XNNPACK' 2025-07-24T04:23:01.9880928Z Entering 'third_party/aiter' 2025-07-24T04:23:01.9944368Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-07-24T04:23:02.0016258Z Entering 'third_party/benchmark' 2025-07-24T04:23:02.0075419Z Entering 'third_party/composable_kernel' 2025-07-24T04:23:02.0150841Z Entering 'third_party/cpp-httplib' 2025-07-24T04:23:02.0220184Z Entering 'third_party/cpuinfo' 2025-07-24T04:23:02.0279285Z Entering 'third_party/cudnn_frontend' 2025-07-24T04:23:02.0342016Z Entering 'third_party/cutlass' 2025-07-24T04:23:02.0421030Z Entering 'third_party/fbgemm' 2025-07-24T04:23:02.0491570Z Entering 'third_party/fbgemm/external/asmjit' 2025-07-24T04:23:02.0546858Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-07-24T04:23:02.0618136Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-07-24T04:23:02.0671848Z Entering 'third_party/fbgemm/external/cutlass' 2025-07-24T04:23:02.0742245Z Entering 'third_party/fbgemm/external/googletest' 2025-07-24T04:23:02.0805262Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-07-24T04:23:02.0862593Z Entering 'third_party/fbgemm/external/json' 2025-07-24T04:23:02.0935520Z Entering 'third_party/flash-attention' 2025-07-24T04:23:02.0993000Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-07-24T04:23:02.1059775Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-07-24T04:23:02.1129289Z Entering 'third_party/flatbuffers' 2025-07-24T04:23:02.1192961Z Entering 'third_party/fmt' 2025-07-24T04:23:02.1256455Z Entering 'third_party/gemmlowp/gemmlowp' 2025-07-24T04:23:02.1315402Z Entering 'third_party/gloo' 2025-07-24T04:23:02.1375895Z Entering 'third_party/googletest' 2025-07-24T04:23:02.1441651Z Entering 'third_party/ideep' 2025-07-24T04:23:02.1503934Z Entering 'third_party/ideep/mkl-dnn' 2025-07-24T04:23:02.1577144Z Entering 'third_party/ittapi' 2025-07-24T04:23:02.1635873Z Entering 'third_party/kineto' 2025-07-24T04:23:02.1700750Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-07-24T04:23:02.1759537Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-07-24T04:23:02.1819335Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-07-24T04:23:02.1881657Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-07-24T04:23:02.1936376Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-07-24T04:23:02.1991937Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-07-24T04:23:02.2057447Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-07-24T04:23:02.2114764Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-07-24T04:23:02.2174571Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-07-24T04:23:02.2234117Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-07-24T04:23:02.2297692Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-07-24T04:23:02.2355818Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-07-24T04:23:02.2417921Z Entering 'third_party/kleidiai' 2025-07-24T04:23:02.2483837Z Entering 'third_party/mimalloc' 2025-07-24T04:23:02.2548870Z Entering 'third_party/nlohmann' 2025-07-24T04:23:02.2617697Z Entering 'third_party/onnx' 2025-07-24T04:23:02.2701518Z Entering 'third_party/onnx/third_party/pybind11' 2025-07-24T04:23:02.2763817Z Entering 'third_party/opentelemetry-cpp' 2025-07-24T04:23:02.2828674Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-07-24T04:23:02.2888233Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-07-24T04:23:02.2946815Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-07-24T04:23:02.3008746Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-07-24T04:23:02.3066820Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-07-24T04:23:02.3132968Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-07-24T04:23:02.3190078Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-07-24T04:23:02.3253027Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-07-24T04:23:02.3312381Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-07-24T04:23:02.3379186Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-07-24T04:23:02.3457849Z Entering 'third_party/pocketfft' 2025-07-24T04:23:02.3519249Z Entering 'third_party/protobuf' 2025-07-24T04:23:02.3582809Z Entering 'third_party/protobuf/third_party/benchmark' 2025-07-24T04:23:02.3644168Z Entering 'third_party/protobuf/third_party/googletest' 2025-07-24T04:23:02.3711739Z Entering 'third_party/psimd' 2025-07-24T04:23:02.3777267Z Entering 'third_party/pthreadpool' 2025-07-24T04:23:02.3839602Z Entering 'third_party/pybind11' 2025-07-24T04:23:02.3902444Z Entering 'third_party/python-peachpy' 2025-07-24T04:23:02.3966985Z Entering 'third_party/sleef' 2025-07-24T04:23:02.4030378Z Entering 'third_party/tensorpipe' 2025-07-24T04:23:02.4096596Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-07-24T04:23:02.4153385Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-07-24T04:23:02.4216298Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-07-24T04:23:02.4272626Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-07-24T04:23:02.4334760Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-07-24T04:23:02.4418197Z ##[endgroup] 2025-07-24T04:23:02.4418646Z ##[group]Persisting credentials for submodules 2025-07-24T04:23:02.4424713Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2025-07-24T04:23:02.4798173Z Entering 'android/libs/fbjni' 2025-07-24T04:23:02.4836813Z url.https://github.com/.insteadof 2025-07-24T04:23:02.4837704Z url.https://github.com/.insteadof 2025-07-24T04:23:02.4905268Z Entering 'third_party/FP16' 2025-07-24T04:23:02.4933168Z url.https://github.com/.insteadof 2025-07-24T04:23:02.4933617Z url.https://github.com/.insteadof 2025-07-24T04:23:02.4984346Z Entering 'third_party/FXdiv' 2025-07-24T04:23:02.5024390Z url.https://github.com/.insteadof 2025-07-24T04:23:02.5024659Z url.https://github.com/.insteadof 2025-07-24T04:23:02.5077620Z Entering 'third_party/NNPACK' 2025-07-24T04:23:02.5113561Z url.https://github.com/.insteadof 2025-07-24T04:23:02.5113997Z url.https://github.com/.insteadof 2025-07-24T04:23:02.5169170Z Entering 'third_party/NVTX' 2025-07-24T04:23:02.5211415Z url.https://github.com/.insteadof 2025-07-24T04:23:02.5211873Z url.https://github.com/.insteadof 2025-07-24T04:23:02.5264948Z Entering 'third_party/VulkanMemoryAllocator' 2025-07-24T04:23:02.5305063Z url.https://github.com/.insteadof 2025-07-24T04:23:02.5305581Z url.https://github.com/.insteadof 2025-07-24T04:23:02.5358398Z Entering 'third_party/XNNPACK' 2025-07-24T04:23:02.5393943Z url.https://github.com/.insteadof 2025-07-24T04:23:02.5394375Z url.https://github.com/.insteadof 2025-07-24T04:23:02.5465718Z Entering 'third_party/aiter' 2025-07-24T04:23:02.5506281Z url.https://github.com/.insteadof 2025-07-24T04:23:02.5506722Z url.https://github.com/.insteadof 2025-07-24T04:23:02.5557781Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-07-24T04:23:02.5591271Z url.https://github.com/.insteadof 2025-07-24T04:23:02.5591721Z url.https://github.com/.insteadof 2025-07-24T04:23:02.5654836Z Entering 'third_party/benchmark' 2025-07-24T04:23:02.5687933Z url.https://github.com/.insteadof 2025-07-24T04:23:02.5688376Z url.https://github.com/.insteadof 2025-07-24T04:23:02.5742144Z Entering 'third_party/composable_kernel' 2025-07-24T04:23:02.5782138Z url.https://github.com/.insteadof 2025-07-24T04:23:02.5782401Z url.https://github.com/.insteadof 2025-07-24T04:23:02.5843998Z Entering 'third_party/cpp-httplib' 2025-07-24T04:23:02.5885963Z url.https://github.com/.insteadof 2025-07-24T04:23:02.5886423Z url.https://github.com/.insteadof 2025-07-24T04:23:02.5935880Z Entering 'third_party/cpuinfo' 2025-07-24T04:23:02.5972059Z url.https://github.com/.insteadof 2025-07-24T04:23:02.5972507Z url.https://github.com/.insteadof 2025-07-24T04:23:02.6027569Z Entering 'third_party/cudnn_frontend' 2025-07-24T04:23:02.6065740Z url.https://github.com/.insteadof 2025-07-24T04:23:02.6066297Z url.https://github.com/.insteadof 2025-07-24T04:23:02.6121422Z Entering 'third_party/cutlass' 2025-07-24T04:23:02.6158498Z url.https://github.com/.insteadof 2025-07-24T04:23:02.6158956Z url.https://github.com/.insteadof 2025-07-24T04:23:02.6222636Z Entering 'third_party/fbgemm' 2025-07-24T04:23:02.6264125Z url.https://github.com/.insteadof 2025-07-24T04:23:02.6264558Z url.https://github.com/.insteadof 2025-07-24T04:23:02.6320443Z Entering 'third_party/fbgemm/external/asmjit' 2025-07-24T04:23:02.6357352Z url.https://github.com/.insteadof 2025-07-24T04:23:02.6357831Z url.https://github.com/.insteadof 2025-07-24T04:23:02.6411965Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-07-24T04:23:02.6448085Z url.https://github.com/.insteadof 2025-07-24T04:23:02.6448886Z url.https://github.com/.insteadof 2025-07-24T04:23:02.6506048Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-07-24T04:23:02.6544161Z url.https://github.com/.insteadof 2025-07-24T04:23:02.6544633Z url.https://github.com/.insteadof 2025-07-24T04:23:02.6593727Z Entering 'third_party/fbgemm/external/cutlass' 2025-07-24T04:23:02.6633529Z url.https://github.com/.insteadof 2025-07-24T04:23:02.6633964Z url.https://github.com/.insteadof 2025-07-24T04:23:02.6699891Z Entering 'third_party/fbgemm/external/googletest' 2025-07-24T04:23:02.6739013Z url.https://github.com/.insteadof 2025-07-24T04:23:02.6739447Z url.https://github.com/.insteadof 2025-07-24T04:23:02.6789979Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-07-24T04:23:02.6822124Z url.https://github.com/.insteadof 2025-07-24T04:23:02.6822930Z url.https://github.com/.insteadof 2025-07-24T04:23:02.6869528Z Entering 'third_party/fbgemm/external/json' 2025-07-24T04:23:02.6903186Z url.https://github.com/.insteadof 2025-07-24T04:23:02.6903649Z url.https://github.com/.insteadof 2025-07-24T04:23:02.6958460Z Entering 'third_party/flash-attention' 2025-07-24T04:23:02.6994154Z url.https://github.com/.insteadof 2025-07-24T04:23:02.7052654Z url.https://github.com/.insteadof 2025-07-24T04:23:02.7053192Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-07-24T04:23:02.7089013Z url.https://github.com/.insteadof 2025-07-24T04:23:02.7089522Z url.https://github.com/.insteadof 2025-07-24T04:23:02.7144851Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-07-24T04:23:02.7184535Z url.https://github.com/.insteadof 2025-07-24T04:23:02.7184972Z url.https://github.com/.insteadof 2025-07-24T04:23:02.7250661Z Entering 'third_party/flatbuffers' 2025-07-24T04:23:02.7289701Z url.https://github.com/.insteadof 2025-07-24T04:23:02.7290244Z url.https://github.com/.insteadof 2025-07-24T04:23:02.7346364Z Entering 'third_party/fmt' 2025-07-24T04:23:02.7386736Z url.https://github.com/.insteadof 2025-07-24T04:23:02.7387174Z url.https://github.com/.insteadof 2025-07-24T04:23:02.7443115Z Entering 'third_party/gemmlowp/gemmlowp' 2025-07-24T04:23:02.7481249Z url.https://github.com/.insteadof 2025-07-24T04:23:02.7481683Z url.https://github.com/.insteadof 2025-07-24T04:23:02.7537034Z Entering 'third_party/gloo' 2025-07-24T04:23:02.7580093Z url.https://github.com/.insteadof 2025-07-24T04:23:02.7580688Z url.https://github.com/.insteadof 2025-07-24T04:23:02.7632384Z Entering 'third_party/googletest' 2025-07-24T04:23:02.7669849Z url.https://github.com/.insteadof 2025-07-24T04:23:02.7670352Z url.https://github.com/.insteadof 2025-07-24T04:23:02.7724481Z Entering 'third_party/ideep' 2025-07-24T04:23:02.7767305Z url.https://github.com/.insteadof 2025-07-24T04:23:02.7767777Z url.https://github.com/.insteadof 2025-07-24T04:23:02.7816750Z Entering 'third_party/ideep/mkl-dnn' 2025-07-24T04:23:02.7852577Z url.https://github.com/.insteadof 2025-07-24T04:23:02.7853257Z url.https://github.com/.insteadof 2025-07-24T04:23:02.7918411Z Entering 'third_party/ittapi' 2025-07-24T04:23:02.7959381Z url.https://github.com/.insteadof 2025-07-24T04:23:02.7959831Z url.https://github.com/.insteadof 2025-07-24T04:23:02.8016357Z Entering 'third_party/kineto' 2025-07-24T04:23:02.8060052Z url.https://github.com/.insteadof 2025-07-24T04:23:02.8060564Z url.https://github.com/.insteadof 2025-07-24T04:23:02.8112031Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-07-24T04:23:02.8149723Z url.https://github.com/.insteadof 2025-07-24T04:23:02.8150163Z url.https://github.com/.insteadof 2025-07-24T04:23:02.8201428Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-07-24T04:23:02.8239266Z url.https://github.com/.insteadof 2025-07-24T04:23:02.8239719Z url.https://github.com/.insteadof 2025-07-24T04:23:02.8297139Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-07-24T04:23:02.8339996Z url.https://github.com/.insteadof 2025-07-24T04:23:02.8340300Z url.https://github.com/.insteadof 2025-07-24T04:23:02.8394683Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-07-24T04:23:02.8430019Z url.https://github.com/.insteadof 2025-07-24T04:23:02.8430479Z url.https://github.com/.insteadof 2025-07-24T04:23:02.8484224Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-07-24T04:23:02.8521672Z url.https://github.com/.insteadof 2025-07-24T04:23:02.8522116Z url.https://github.com/.insteadof 2025-07-24T04:23:02.8574474Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-07-24T04:23:02.8611923Z url.https://github.com/.insteadof 2025-07-24T04:23:02.8612463Z url.https://github.com/.insteadof 2025-07-24T04:23:02.8671584Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-07-24T04:23:02.8708944Z url.https://github.com/.insteadof 2025-07-24T04:23:02.8709796Z url.https://github.com/.insteadof 2025-07-24T04:23:02.8764315Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-07-24T04:23:02.8801734Z url.https://github.com/.insteadof 2025-07-24T04:23:02.8856825Z url.https://github.com/.insteadof 2025-07-24T04:23:02.8857203Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-07-24T04:23:02.8887788Z url.https://github.com/.insteadof 2025-07-24T04:23:02.8888217Z url.https://github.com/.insteadof 2025-07-24T04:23:02.8940369Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-07-24T04:23:02.8979748Z url.https://github.com/.insteadof 2025-07-24T04:23:02.8980229Z url.https://github.com/.insteadof 2025-07-24T04:23:02.9039183Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-07-24T04:23:02.9071949Z url.https://github.com/.insteadof 2025-07-24T04:23:02.9072390Z url.https://github.com/.insteadof 2025-07-24T04:23:02.9125540Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-07-24T04:23:02.9162116Z url.https://github.com/.insteadof 2025-07-24T04:23:02.9162665Z url.https://github.com/.insteadof 2025-07-24T04:23:02.9250608Z Entering 'third_party/kleidiai' 2025-07-24T04:23:02.9262547Z url.https://github.com/.insteadof 2025-07-24T04:23:02.9262826Z url.https://github.com/.insteadof 2025-07-24T04:23:02.9316509Z Entering 'third_party/mimalloc' 2025-07-24T04:23:02.9350638Z url.https://github.com/.insteadof 2025-07-24T04:23:02.9351081Z url.https://github.com/.insteadof 2025-07-24T04:23:02.9405632Z Entering 'third_party/nlohmann' 2025-07-24T04:23:02.9445231Z url.https://github.com/.insteadof 2025-07-24T04:23:02.9445692Z url.https://github.com/.insteadof 2025-07-24T04:23:02.9501704Z Entering 'third_party/onnx' 2025-07-24T04:23:02.9541956Z url.https://github.com/.insteadof 2025-07-24T04:23:02.9542474Z url.https://github.com/.insteadof 2025-07-24T04:23:02.9613302Z Entering 'third_party/onnx/third_party/pybind11' 2025-07-24T04:23:02.9652374Z url.https://github.com/.insteadof 2025-07-24T04:23:02.9652886Z url.https://github.com/.insteadof 2025-07-24T04:23:02.9710220Z Entering 'third_party/opentelemetry-cpp' 2025-07-24T04:23:02.9750242Z url.https://github.com/.insteadof 2025-07-24T04:23:02.9750674Z url.https://github.com/.insteadof 2025-07-24T04:23:02.9801710Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-07-24T04:23:02.9840099Z url.https://github.com/.insteadof 2025-07-24T04:23:02.9840536Z url.https://github.com/.insteadof 2025-07-24T04:23:02.9896369Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-07-24T04:23:02.9933507Z url.https://github.com/.insteadof 2025-07-24T04:23:02.9933943Z url.https://github.com/.insteadof 2025-07-24T04:23:02.9981924Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-07-24T04:23:03.0019611Z url.https://github.com/.insteadof 2025-07-24T04:23:03.0020038Z url.https://github.com/.insteadof 2025-07-24T04:23:03.0072919Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-07-24T04:23:03.0110348Z url.https://github.com/.insteadof 2025-07-24T04:23:03.0110788Z url.https://github.com/.insteadof 2025-07-24T04:23:03.0161075Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-07-24T04:23:03.0195660Z url.https://github.com/.insteadof 2025-07-24T04:23:03.0196140Z url.https://github.com/.insteadof 2025-07-24T04:23:03.0247554Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-07-24T04:23:03.0280171Z url.https://github.com/.insteadof 2025-07-24T04:23:03.0280608Z url.https://github.com/.insteadof 2025-07-24T04:23:03.0337302Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-07-24T04:23:03.0374611Z url.https://github.com/.insteadof 2025-07-24T04:23:03.0375128Z url.https://github.com/.insteadof 2025-07-24T04:23:03.0420334Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-07-24T04:23:03.0456008Z url.https://github.com/.insteadof 2025-07-24T04:23:03.0456991Z url.https://github.com/.insteadof 2025-07-24T04:23:03.0511238Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-07-24T04:23:03.0551120Z url.https://github.com/.insteadof 2025-07-24T04:23:03.0551593Z url.https://github.com/.insteadof 2025-07-24T04:23:03.0603503Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-07-24T04:23:03.0641321Z url.https://github.com/.insteadof 2025-07-24T04:23:03.0641757Z url.https://github.com/.insteadof 2025-07-24T04:23:03.0724973Z Entering 'third_party/pocketfft' 2025-07-24T04:23:03.0765318Z url.https://github.com/.insteadof 2025-07-24T04:23:03.0765746Z url.https://github.com/.insteadof 2025-07-24T04:23:03.0821629Z Entering 'third_party/protobuf' 2025-07-24T04:23:03.0864867Z url.https://github.com/.insteadof 2025-07-24T04:23:03.0865161Z url.https://github.com/.insteadof 2025-07-24T04:23:03.0923722Z Entering 'third_party/protobuf/third_party/benchmark' 2025-07-24T04:23:03.0962786Z url.https://github.com/.insteadof 2025-07-24T04:23:03.0963289Z url.https://github.com/.insteadof 2025-07-24T04:23:03.1012402Z Entering 'third_party/protobuf/third_party/googletest' 2025-07-24T04:23:03.1052945Z url.https://github.com/.insteadof 2025-07-24T04:23:03.1053419Z url.https://github.com/.insteadof 2025-07-24T04:23:03.1111982Z Entering 'third_party/psimd' 2025-07-24T04:23:03.1146779Z url.https://github.com/.insteadof 2025-07-24T04:23:03.1147232Z url.https://github.com/.insteadof 2025-07-24T04:23:03.1200617Z Entering 'third_party/pthreadpool' 2025-07-24T04:23:03.1236826Z url.https://github.com/.insteadof 2025-07-24T04:23:03.1237287Z url.https://github.com/.insteadof 2025-07-24T04:23:03.1292845Z Entering 'third_party/pybind11' 2025-07-24T04:23:03.1335512Z url.https://github.com/.insteadof 2025-07-24T04:23:03.1336038Z url.https://github.com/.insteadof 2025-07-24T04:23:03.1389478Z Entering 'third_party/python-peachpy' 2025-07-24T04:23:03.1425763Z url.https://github.com/.insteadof 2025-07-24T04:23:03.1426278Z url.https://github.com/.insteadof 2025-07-24T04:23:03.1481096Z Entering 'third_party/sleef' 2025-07-24T04:23:03.1524512Z url.https://github.com/.insteadof 2025-07-24T04:23:03.1524972Z url.https://github.com/.insteadof 2025-07-24T04:23:03.1575769Z Entering 'third_party/tensorpipe' 2025-07-24T04:23:03.1616223Z url.https://github.com/.insteadof 2025-07-24T04:23:03.1616758Z url.https://github.com/.insteadof 2025-07-24T04:23:03.1668729Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-07-24T04:23:03.1704454Z url.https://github.com/.insteadof 2025-07-24T04:23:03.1704874Z url.https://github.com/.insteadof 2025-07-24T04:23:03.1753202Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-07-24T04:23:03.1784075Z url.https://github.com/.insteadof 2025-07-24T04:23:03.1784357Z url.https://github.com/.insteadof 2025-07-24T04:23:03.1832735Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-07-24T04:23:03.1866137Z url.https://github.com/.insteadof 2025-07-24T04:23:03.1866643Z url.https://github.com/.insteadof 2025-07-24T04:23:03.1919276Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-07-24T04:23:03.1953103Z url.https://github.com/.insteadof 2025-07-24T04:23:03.1953540Z url.https://github.com/.insteadof 2025-07-24T04:23:03.2009000Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-07-24T04:23:03.2046764Z url.https://github.com/.insteadof 2025-07-24T04:23:03.2047186Z url.https://github.com/.insteadof 2025-07-24T04:23:03.2133377Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2025-07-24T04:23:03.2513617Z Entering 'android/libs/fbjni' 2025-07-24T04:23:03.2569611Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2025-07-24T04:23:03.2603148Z Entering 'third_party/FP16' 2025-07-24T04:23:03.2666911Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2025-07-24T04:23:03.2698985Z Entering 'third_party/FXdiv' 2025-07-24T04:23:03.2760700Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2025-07-24T04:23:03.2792694Z Entering 'third_party/NNPACK' 2025-07-24T04:23:03.2850080Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2025-07-24T04:23:03.2883717Z Entering 'third_party/NVTX' 2025-07-24T04:23:03.2944284Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NVTX/config remote.origin.url 2025-07-24T04:23:03.2978570Z Entering 'third_party/VulkanMemoryAllocator' 2025-07-24T04:23:03.3040743Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2025-07-24T04:23:03.3072980Z Entering 'third_party/XNNPACK' 2025-07-24T04:23:03.3130583Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2025-07-24T04:23:03.3180162Z Entering 'third_party/aiter' 2025-07-24T04:23:03.3241101Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/aiter/config remote.origin.url 2025-07-24T04:23:03.3272810Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-07-24T04:23:03.3330725Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/aiter/modules/3rdparty/composable_kernel/config remote.origin.url 2025-07-24T04:23:03.3371415Z Entering 'third_party/benchmark' 2025-07-24T04:23:03.3424127Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2025-07-24T04:23:03.3453828Z Entering 'third_party/composable_kernel' 2025-07-24T04:23:03.3509312Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/composable_kernel/config remote.origin.url 2025-07-24T04:23:03.3550255Z Entering 'third_party/cpp-httplib' 2025-07-24T04:23:03.3610991Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2025-07-24T04:23:03.3644008Z Entering 'third_party/cpuinfo' 2025-07-24T04:23:03.3703256Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2025-07-24T04:23:03.3734881Z Entering 'third_party/cudnn_frontend' 2025-07-24T04:23:03.3788193Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2025-07-24T04:23:03.3820794Z Entering 'third_party/cutlass' 2025-07-24T04:23:03.3881571Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2025-07-24T04:23:03.3924207Z Entering 'third_party/fbgemm' 2025-07-24T04:23:03.3983694Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2025-07-24T04:23:03.4015614Z Entering 'third_party/fbgemm/external/asmjit' 2025-07-24T04:23:03.4071715Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/asmjit/config remote.origin.url 2025-07-24T04:23:03.4105819Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-07-24T04:23:03.4163530Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/composable_kernel/config remote.origin.url 2025-07-24T04:23:03.4200626Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-07-24T04:23:03.4259438Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/cpuinfo/config remote.origin.url 2025-07-24T04:23:03.4289746Z Entering 'third_party/fbgemm/external/cutlass' 2025-07-24T04:23:03.4344738Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/cutlass/config remote.origin.url 2025-07-24T04:23:03.4386504Z Entering 'third_party/fbgemm/external/googletest' 2025-07-24T04:23:03.4444916Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/googletest/config remote.origin.url 2025-07-24T04:23:03.4472966Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-07-24T04:23:03.4532752Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/hipify_torch/config remote.origin.url 2025-07-24T04:23:03.4557984Z Entering 'third_party/fbgemm/external/json' 2025-07-24T04:23:03.4612052Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/json/config remote.origin.url 2025-07-24T04:23:03.4643831Z Entering 'third_party/flash-attention' 2025-07-24T04:23:03.4704354Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/config remote.origin.url 2025-07-24T04:23:03.4734691Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-07-24T04:23:03.4791749Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/modules/csrc/composable_kernel/config remote.origin.url 2025-07-24T04:23:03.4833538Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-07-24T04:23:03.4893363Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/modules/csrc/cutlass/config remote.origin.url 2025-07-24T04:23:03.4935482Z Entering 'third_party/flatbuffers' 2025-07-24T04:23:03.4991337Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2025-07-24T04:23:03.5030872Z Entering 'third_party/fmt' 2025-07-24T04:23:03.5092975Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2025-07-24T04:23:03.5121369Z Entering 'third_party/gemmlowp/gemmlowp' 2025-07-24T04:23:03.5183903Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2025-07-24T04:23:03.5219182Z Entering 'third_party/gloo' 2025-07-24T04:23:03.5284375Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2025-07-24T04:23:03.5317380Z Entering 'third_party/googletest' 2025-07-24T04:23:03.5375082Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2025-07-24T04:23:03.5407964Z Entering 'third_party/ideep' 2025-07-24T04:23:03.5469299Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2025-07-24T04:23:03.5500847Z Entering 'third_party/ideep/mkl-dnn' 2025-07-24T04:23:03.5565347Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2025-07-24T04:23:03.5607868Z Entering 'third_party/ittapi' 2025-07-24T04:23:03.5667264Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2025-07-24T04:23:03.5699304Z Entering 'third_party/kineto' 2025-07-24T04:23:03.5754246Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2025-07-24T04:23:03.5788589Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-07-24T04:23:03.5847816Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2025-07-24T04:23:03.5875282Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-07-24T04:23:03.5935377Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2025-07-24T04:23:03.5962922Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-07-24T04:23:03.6026117Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2025-07-24T04:23:03.6059494Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-07-24T04:23:03.6109724Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2025-07-24T04:23:03.6141193Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-07-24T04:23:03.6194356Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2025-07-24T04:23:03.6222124Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-07-24T04:23:03.6283971Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2025-07-24T04:23:03.6316136Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-07-24T04:23:03.6377610Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2025-07-24T04:23:03.6410794Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-07-24T04:23:03.6463326Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2025-07-24T04:23:03.6494018Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-07-24T04:23:03.6557888Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2025-07-24T04:23:03.6589254Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-07-24T04:23:03.6646276Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2025-07-24T04:23:03.6676890Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-07-24T04:23:03.6736822Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2025-07-24T04:23:03.6770543Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-07-24T04:23:03.6824223Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2025-07-24T04:23:03.6858716Z Entering 'third_party/kleidiai' 2025-07-24T04:23:03.6919096Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kleidiai/config remote.origin.url 2025-07-24T04:23:03.6951036Z Entering 'third_party/mimalloc' 2025-07-24T04:23:03.7008101Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2025-07-24T04:23:03.7041059Z Entering 'third_party/nlohmann' 2025-07-24T04:23:03.7104303Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2025-07-24T04:23:03.7139202Z Entering 'third_party/onnx' 2025-07-24T04:23:03.7200111Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2025-07-24T04:23:03.7247028Z Entering 'third_party/onnx/third_party/pybind11' 2025-07-24T04:23:03.7304845Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2025-07-24T04:23:03.7340799Z Entering 'third_party/opentelemetry-cpp' 2025-07-24T04:23:03.7401220Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2025-07-24T04:23:03.7431480Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-07-24T04:23:03.7489356Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2025-07-24T04:23:03.7519267Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-07-24T04:23:03.7579289Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2025-07-24T04:23:03.7612506Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-07-24T04:23:03.7667539Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2025-07-24T04:23:03.7701182Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-07-24T04:23:03.7759954Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2025-07-24T04:23:03.7790924Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-07-24T04:23:03.7848961Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2025-07-24T04:23:03.7874915Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-07-24T04:23:03.7930018Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2025-07-24T04:23:03.7959602Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-07-24T04:23:03.8018840Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2025-07-24T04:23:03.8051108Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-07-24T04:23:03.8107280Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2025-07-24T04:23:03.8138262Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-07-24T04:23:03.8200879Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2025-07-24T04:23:03.8235993Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-07-24T04:23:03.8291041Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2025-07-24T04:23:03.8347410Z Entering 'third_party/pocketfft' 2025-07-24T04:23:03.8409728Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2025-07-24T04:23:03.8442740Z Entering 'third_party/protobuf' 2025-07-24T04:23:03.8502275Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2025-07-24T04:23:03.8538868Z Entering 'third_party/protobuf/third_party/benchmark' 2025-07-24T04:23:03.8601424Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2025-07-24T04:23:03.8629823Z Entering 'third_party/protobuf/third_party/googletest' 2025-07-24T04:23:03.8690386Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2025-07-24T04:23:03.8721333Z Entering 'third_party/psimd' 2025-07-24T04:23:03.8780739Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2025-07-24T04:23:03.8815937Z Entering 'third_party/pthreadpool' 2025-07-24T04:23:03.8881404Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2025-07-24T04:23:03.8913539Z Entering 'third_party/pybind11' 2025-07-24T04:23:03.8970885Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2025-07-24T04:23:03.9004512Z Entering 'third_party/python-peachpy' 2025-07-24T04:23:03.9062733Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2025-07-24T04:23:03.9096925Z Entering 'third_party/sleef' 2025-07-24T04:23:03.9162125Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2025-07-24T04:23:03.9193909Z Entering 'third_party/tensorpipe' 2025-07-24T04:23:03.9251282Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2025-07-24T04:23:03.9282109Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-07-24T04:23:03.9341776Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2025-07-24T04:23:03.9375297Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-07-24T04:23:03.9428544Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2025-07-24T04:23:03.9458292Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-07-24T04:23:03.9522207Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2025-07-24T04:23:03.9552195Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-07-24T04:23:03.9609807Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2025-07-24T04:23:03.9634114Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-07-24T04:23:03.9692582Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2025-07-24T04:23:04.0028083Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2025-07-24T04:23:04.0398428Z Entering 'android/libs/fbjni' 2025-07-24T04:23:04.0461173Z Entering 'third_party/FP16' 2025-07-24T04:23:04.0520877Z Entering 'third_party/FXdiv' 2025-07-24T04:23:04.0583605Z Entering 'third_party/NNPACK' 2025-07-24T04:23:04.0649708Z Entering 'third_party/NVTX' 2025-07-24T04:23:04.0714110Z Entering 'third_party/VulkanMemoryAllocator' 2025-07-24T04:23:04.0779518Z Entering 'third_party/XNNPACK' 2025-07-24T04:23:04.0855620Z Entering 'third_party/aiter' 2025-07-24T04:23:04.0915760Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-07-24T04:23:04.0993318Z Entering 'third_party/benchmark' 2025-07-24T04:23:04.1060528Z Entering 'third_party/composable_kernel' 2025-07-24T04:23:04.1130682Z Entering 'third_party/cpp-httplib' 2025-07-24T04:23:04.1195138Z Entering 'third_party/cpuinfo' 2025-07-24T04:23:04.1261216Z Entering 'third_party/cudnn_frontend' 2025-07-24T04:23:04.1320919Z Entering 'third_party/cutlass' 2025-07-24T04:23:04.1394809Z Entering 'third_party/fbgemm' 2025-07-24T04:23:04.1461121Z Entering 'third_party/fbgemm/external/asmjit' 2025-07-24T04:23:04.1515399Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-07-24T04:23:04.1580682Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-07-24T04:23:04.1631546Z Entering 'third_party/fbgemm/external/cutlass' 2025-07-24T04:23:04.1699758Z Entering 'third_party/fbgemm/external/googletest' 2025-07-24T04:23:04.1759095Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-07-24T04:23:04.1813346Z Entering 'third_party/fbgemm/external/json' 2025-07-24T04:23:04.1877344Z Entering 'third_party/flash-attention' 2025-07-24T04:23:04.1940814Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-07-24T04:23:04.2007191Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-07-24T04:23:04.2079770Z Entering 'third_party/flatbuffers' 2025-07-24T04:23:04.2149249Z Entering 'third_party/fmt' 2025-07-24T04:23:04.2213307Z Entering 'third_party/gemmlowp/gemmlowp' 2025-07-24T04:23:04.2275577Z Entering 'third_party/gloo' 2025-07-24T04:23:04.2341394Z Entering 'third_party/googletest' 2025-07-24T04:23:04.2410262Z Entering 'third_party/ideep' 2025-07-24T04:23:04.2467522Z Entering 'third_party/ideep/mkl-dnn' 2025-07-24T04:23:04.2544127Z Entering 'third_party/ittapi' 2025-07-24T04:23:04.2613496Z Entering 'third_party/kineto' 2025-07-24T04:23:04.2679431Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-07-24T04:23:04.2734861Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-07-24T04:23:04.2795127Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-07-24T04:23:04.2854677Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-07-24T04:23:04.2912462Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-07-24T04:23:04.2970824Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-07-24T04:23:04.3032443Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-07-24T04:23:04.3092773Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-07-24T04:23:04.3149456Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-07-24T04:23:04.3214219Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-07-24T04:23:04.3273954Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-07-24T04:23:04.3332804Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-07-24T04:23:04.3391251Z Entering 'third_party/kleidiai' 2025-07-24T04:23:04.3454974Z Entering 'third_party/mimalloc' 2025-07-24T04:23:04.3518359Z Entering 'third_party/nlohmann' 2025-07-24T04:23:04.3586242Z Entering 'third_party/onnx' 2025-07-24T04:23:04.3671586Z Entering 'third_party/onnx/third_party/pybind11' 2025-07-24T04:23:04.3740748Z Entering 'third_party/opentelemetry-cpp' 2025-07-24T04:23:04.3805276Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-07-24T04:23:04.3864016Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-07-24T04:23:04.3924007Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-07-24T04:23:04.3984896Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-07-24T04:23:04.4038791Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-07-24T04:23:04.4095532Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-07-24T04:23:04.4144745Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-07-24T04:23:04.4193541Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-07-24T04:23:04.4256479Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-07-24T04:23:04.4316662Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-07-24T04:23:04.4401070Z Entering 'third_party/pocketfft' 2025-07-24T04:23:04.4465741Z Entering 'third_party/protobuf' 2025-07-24T04:23:04.4532300Z Entering 'third_party/protobuf/third_party/benchmark' 2025-07-24T04:23:04.4590288Z Entering 'third_party/protobuf/third_party/googletest' 2025-07-24T04:23:04.4659512Z Entering 'third_party/psimd' 2025-07-24T04:23:04.4717493Z Entering 'third_party/pthreadpool' 2025-07-24T04:23:04.4781014Z Entering 'third_party/pybind11' 2025-07-24T04:23:04.4844919Z Entering 'third_party/python-peachpy' 2025-07-24T04:23:04.4910072Z Entering 'third_party/sleef' 2025-07-24T04:23:04.4977623Z Entering 'third_party/tensorpipe' 2025-07-24T04:23:04.5036752Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-07-24T04:23:04.5097120Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-07-24T04:23:04.5152604Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-07-24T04:23:04.5216548Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-07-24T04:23:04.5270406Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-07-24T04:23:04.5361078Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2025-07-24T04:23:04.5739169Z Entering 'android/libs/fbjni' 2025-07-24T04:23:04.5801712Z Entering 'third_party/FP16' 2025-07-24T04:23:04.5868500Z Entering 'third_party/FXdiv' 2025-07-24T04:23:04.5938473Z Entering 'third_party/NNPACK' 2025-07-24T04:23:04.5996080Z Entering 'third_party/NVTX' 2025-07-24T04:23:04.6062258Z Entering 'third_party/VulkanMemoryAllocator' 2025-07-24T04:23:04.6128067Z Entering 'third_party/XNNPACK' 2025-07-24T04:23:04.6211657Z Entering 'third_party/aiter' 2025-07-24T04:23:04.6275332Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-07-24T04:23:04.6352894Z Entering 'third_party/benchmark' 2025-07-24T04:23:04.6420581Z Entering 'third_party/composable_kernel' 2025-07-24T04:23:04.6491686Z Entering 'third_party/cpp-httplib' 2025-07-24T04:23:04.6555462Z Entering 'third_party/cpuinfo' 2025-07-24T04:23:04.6621669Z Entering 'third_party/cudnn_frontend' 2025-07-24T04:23:04.6683783Z Entering 'third_party/cutlass' 2025-07-24T04:23:04.6762590Z Entering 'third_party/fbgemm' 2025-07-24T04:23:04.6825707Z Entering 'third_party/fbgemm/external/asmjit' 2025-07-24T04:23:04.6889989Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-07-24T04:23:04.6951045Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-07-24T04:23:04.7012411Z Entering 'third_party/fbgemm/external/cutlass' 2025-07-24T04:23:04.7084676Z Entering 'third_party/fbgemm/external/googletest' 2025-07-24T04:23:04.7143831Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-07-24T04:23:04.7208640Z Entering 'third_party/fbgemm/external/json' 2025-07-24T04:23:04.7276521Z Entering 'third_party/flash-attention' 2025-07-24T04:23:04.7340301Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-07-24T04:23:04.7407225Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-07-24T04:23:04.7480566Z Entering 'third_party/flatbuffers' 2025-07-24T04:23:04.7548074Z Entering 'third_party/fmt' 2025-07-24T04:23:04.7613041Z Entering 'third_party/gemmlowp/gemmlowp' 2025-07-24T04:23:04.7676378Z Entering 'third_party/gloo' 2025-07-24T04:23:04.7742385Z Entering 'third_party/googletest' 2025-07-24T04:23:04.7807569Z Entering 'third_party/ideep' 2025-07-24T04:23:04.7866825Z Entering 'third_party/ideep/mkl-dnn' 2025-07-24T04:23:04.7943821Z Entering 'third_party/ittapi' 2025-07-24T04:23:04.8012871Z Entering 'third_party/kineto' 2025-07-24T04:23:04.8074617Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-07-24T04:23:04.8131403Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-07-24T04:23:04.8189721Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-07-24T04:23:04.8252614Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-07-24T04:23:04.8308449Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-07-24T04:23:04.8371629Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-07-24T04:23:04.8433961Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-07-24T04:23:04.8496331Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-07-24T04:23:04.8549002Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-07-24T04:23:04.8614234Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-07-24T04:23:04.8674918Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-07-24T04:23:04.8738091Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-07-24T04:23:04.8799494Z Entering 'third_party/kleidiai' 2025-07-24T04:23:04.8864414Z Entering 'third_party/mimalloc' 2025-07-24T04:23:04.8928685Z Entering 'third_party/nlohmann' 2025-07-24T04:23:04.8992823Z Entering 'third_party/onnx' 2025-07-24T04:23:04.9073888Z Entering 'third_party/onnx/third_party/pybind11' 2025-07-24T04:23:04.9141139Z Entering 'third_party/opentelemetry-cpp' 2025-07-24T04:23:04.9205560Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-07-24T04:23:04.9263791Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-07-24T04:23:04.9323228Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-07-24T04:23:04.9385512Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-07-24T04:23:04.9445938Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-07-24T04:23:04.9503976Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-07-24T04:23:04.9562508Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-07-24T04:23:04.9621277Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-07-24T04:23:04.9680214Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-07-24T04:23:04.9741709Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-07-24T04:23:04.9827056Z Entering 'third_party/pocketfft' 2025-07-24T04:23:04.9896458Z Entering 'third_party/protobuf' 2025-07-24T04:23:04.9960909Z Entering 'third_party/protobuf/third_party/benchmark' 2025-07-24T04:23:05.0019051Z Entering 'third_party/protobuf/third_party/googletest' 2025-07-24T04:23:05.0079449Z Entering 'third_party/psimd' 2025-07-24T04:23:05.0139156Z Entering 'third_party/pthreadpool' 2025-07-24T04:23:05.0198283Z Entering 'third_party/pybind11' 2025-07-24T04:23:05.0262105Z Entering 'third_party/python-peachpy' 2025-07-24T04:23:05.0326772Z Entering 'third_party/sleef' 2025-07-24T04:23:05.0391788Z Entering 'third_party/tensorpipe' 2025-07-24T04:23:05.0457045Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-07-24T04:23:05.0513384Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-07-24T04:23:05.0575376Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-07-24T04:23:05.0632187Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-07-24T04:23:05.0694701Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-07-24T04:23:05.0780828Z ##[endgroup] 2025-07-24T04:23:05.0886833Z [command]/usr/bin/git log -1 --format=%H 2025-07-24T04:23:05.0970304Z 27c8ef1839e0747b36769efb192befc9308ff93c 2025-07-24T04:23:05.1198946Z Prepare all required actions 2025-07-24T04:23:05.1199442Z Getting action download info 2025-07-24T04:23:05.3050390Z ##[group]Run ./.github/actions/setup-rocm 2025-07-24T04:23:05.3050662Z env: 2025-07-24T04:23:05.3050834Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:05.3051053Z ##[endgroup] 2025-07-24T04:23:05.3072574Z ##[group]Run dpkg -l | grep -E " rocm" 2025-07-24T04:23:05.3072881Z dpkg -l | grep -E " rocm" 2025-07-24T04:23:05.3101530Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:05.3101847Z env: 2025-07-24T04:23:05.3102044Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:05.3102265Z ##[endgroup] 2025-07-24T04:23:05.3340765Z ii rocm 6.3.3.60303-74~22.04 amd64 Radeon Open Compute (ROCm) software stack meta package 2025-07-24T04:23:05.3341754Z ii rocm-cmake 0.14.0.60303-74~22.04 amd64 rocm-cmake built using CMake 2025-07-24T04:23:05.3342769Z ii rocm-core 6.3.3.60303-74~22.04 amd64 Radeon Open Compute (ROCm) Runtime software stack 2025-07-24T04:23:05.3344224Z ii rocm-dbgapi 0.77.0.60303-74~22.04 amd64 Library to provide AMD GPU debugger API 2025-07-24T04:23:05.3345246Z ii rocm-debug-agent 2.0.3.60303-74~22.04 amd64 Radeon Open Compute Debug Agent (ROCdebug-agent) 2025-07-24T04:23:05.3346340Z ii rocm-developer-tools 6.3.3.60303-74~22.04 amd64 Radeon Open Compute (ROCm) Runtime software stack 2025-07-24T04:23:05.3347343Z ii rocm-device-libs 1.0.0.60303-74~22.04 amd64 Radeon Open Compute - device libraries 2025-07-24T04:23:05.3347896Z ii rocm-gdb 15.2.60303-74~22.04 amd64 ROCgdb 2025-07-24T04:23:05.3348475Z ii rocm-hip-libraries 6.3.3.60303-74~22.04 amd64 Radeon Open Compute (ROCm) Runtime software stack 2025-07-24T04:23:05.3349159Z ii rocm-hip-runtime 6.3.3.60303-74~22.04 amd64 Radeon Open Compute (ROCm) Runtime software stack 2025-07-24T04:23:05.3349840Z ii rocm-hip-runtime-dev 6.3.3.60303-74~22.04 amd64 Radeon Open Compute (ROCm) Runtime software stack 2025-07-24T04:23:05.3350488Z ii rocm-hip-sdk 6.3.3.60303-74~22.04 amd64 Radeon Open Compute (ROCm) Runtime software stack 2025-07-24T04:23:05.3351160Z ii rocm-language-runtime 6.3.3.60303-74~22.04 amd64 Radeon Open Compute (ROCm) Runtime software stack 2025-07-24T04:23:05.3351790Z ii rocm-llvm 18.0.0.25012.60303-74~22.04 amd64 ROCm core compiler 2025-07-24T04:23:05.3402182Z ii rocm-ml-libraries 6.3.3.60303-74~22.04 amd64 Radeon Open Compute (ROCm) Runtime software stack 2025-07-24T04:23:05.3403000Z ii rocm-ml-sdk 6.3.3.60303-74~22.04 amd64 Radeon Open Compute (ROCm) Runtime software stack 2025-07-24T04:23:05.3403549Z ii rocm-opencl 2.0.0.60303-74~22.04 amd64 clr built using CMake 2025-07-24T04:23:05.3404033Z ii rocm-opencl-dev 2.0.0.60303-74~22.04 amd64 clr built using CMake 2025-07-24T04:23:05.3404571Z ii rocm-opencl-runtime 6.3.3.60303-74~22.04 amd64 Radeon Open Compute (ROCm) Runtime software stack 2025-07-24T04:23:05.3405159Z ii rocm-opencl-sdk 6.3.3.60303-74~22.04 amd64 Radeon Open Compute (ROCm) Runtime software stack 2025-07-24T04:23:05.3406156Z ii rocm-openmp-sdk 6.3.3.60303-74~22.04 amd64 Radeon Open Compute (ROCm) OpenMP Software development Kit. 2025-07-24T04:23:05.3406845Z ii rocm-smi-lib 7.4.0.60303-74~22.04 amd64 AMD System Management libraries 2025-07-24T04:23:05.3407473Z ii rocm-utils 6.3.3.60303-74~22.04 amd64 Radeon Open Compute (ROCm) Runtime software stack 2025-07-24T04:23:05.3408016Z ii rocminfo 1.0.0.60303-74~22.04 amd64 Radeon Open Compute (ROCm) Runtime rocminfo tool 2025-07-24T04:23:05.3429176Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2025-07-24T04:23:05.3429692Z # ignore expansion of "docker ps -q" since it could be empty 2025-07-24T04:23:05.3430042Z # shellcheck disable=SC2046 2025-07-24T04:23:05.3430319Z docker stop $(docker ps -q) || true 2025-07-24T04:23:05.3430601Z # Prune all stopped containers. 2025-07-24T04:23:05.3430873Z docker container prune -f 2025-07-24T04:23:05.3461679Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:05.3462014Z env: 2025-07-24T04:23:05.3462198Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:05.3462420Z ##[endgroup] 2025-07-24T04:23:05.3897426Z "docker stop" requires at least 1 argument. 2025-07-24T04:23:05.3897868Z See 'docker stop --help'. 2025-07-24T04:23:05.3898091Z 2025-07-24T04:23:05.3898286Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2025-07-24T04:23:05.3898620Z 2025-07-24T04:23:05.3898758Z Stop one or more running containers 2025-07-24T04:23:05.4099995Z Total reclaimed space: 0B 2025-07-24T04:23:05.4155879Z ##[group]Run cat /etc/os-release || true 2025-07-24T04:23:05.4156207Z cat /etc/os-release || true 2025-07-24T04:23:05.4156503Z cat /etc/apt/sources.list.d/rocm.list || true 2025-07-24T04:23:05.4156817Z cat /opt/rocm/.info/version || true 2025-07-24T04:23:05.4157073Z whoami 2025-07-24T04:23:05.4186278Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:05.4186645Z env: 2025-07-24T04:23:05.4186831Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:05.4187050Z ##[endgroup] 2025-07-24T04:23:05.4249617Z PRETTY_NAME="Ubuntu 22.04.4 LTS" 2025-07-24T04:23:05.4249870Z NAME="Ubuntu" 2025-07-24T04:23:05.4250063Z VERSION_ID="22.04" 2025-07-24T04:23:05.4250283Z VERSION="22.04.4 LTS (Jammy Jellyfish)" 2025-07-24T04:23:05.4250548Z VERSION_CODENAME=jammy 2025-07-24T04:23:05.4250760Z ID=ubuntu 2025-07-24T04:23:05.4250924Z ID_LIKE=debian 2025-07-24T04:23:05.4251159Z HOME_URL="https://www.ubuntu.com/" 2025-07-24T04:23:05.4251453Z SUPPORT_URL="https://help.ubuntu.com/" 2025-07-24T04:23:05.4251790Z BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2025-07-24T04:23:05.4252251Z PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2025-07-24T04:23:05.4252662Z UBUNTU_CODENAME=jammy 2025-07-24T04:23:05.4265014Z deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/6.3.3 jammy main 2025-07-24T04:23:05.4283302Z 6.3.3-74 2025-07-24T04:23:05.4307284Z pytorchci 2025-07-24T04:23:05.4336666Z ##[group]Run dpkg -l | grep -E " amdgpu" 2025-07-24T04:23:05.4337011Z dpkg -l | grep -E " amdgpu" 2025-07-24T04:23:05.4366477Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:05.4366805Z env: 2025-07-24T04:23:05.4366997Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:05.4367218Z ##[endgroup] 2025-07-24T04:23:05.4598557Z ii amdgpu-core 1:6.3.60303-2119913.22.04 all Core meta package for unified amdgpu driver. 2025-07-24T04:23:05.4599344Z ii amdgpu-dkms 1:6.10.5.60303-2119913.22.04 all amdgpu driver in DKMS format. 2025-07-24T04:23:05.4600382Z ii amdgpu-dkms-firmware 1:6.10.5.60303-2119913.22.04 all firmware blobs used by amdgpu driver in DKMS format 2025-07-24T04:23:05.4601898Z ii amdgpu-install 6.3.60303-2119913.22.04 all AMDGPU driver repository and installer 2025-07-24T04:23:05.4628695Z ##[group]Run rocm-smi 2025-07-24T04:23:05.4628962Z rocm-smi 2025-07-24T04:23:05.4658243Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:05.4658572Z env: 2025-07-24T04:23:05.4658760Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:05.4658985Z ##[endgroup] 2025-07-24T04:23:05.6201691Z 2025-07-24T04:23:05.6201701Z 2025-07-24T04:23:05.6202001Z ========================================= ROCm System Management Interface ========================================= 2025-07-24T04:23:05.6203163Z =================================================== Concise Info =================================================== 2025-07-24T04:23:05.6204046Z Device Node IDs Temp Power Partitions SCLK MCLK Fan Perf PwrCap VRAM% GPU% 2025-07-24T04:23:05.6205393Z  (DID, GUID) (Edge) (Avg) (Mem, Compute, ID)  2025-07-24T04:23:05.6206494Z ==================================================================================================================== 2025-07-24T04:23:05.6207503Z 0 4 0x740c, 57586 37.0°C 89.0W N/A, N/A, 0 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 2025-07-24T04:23:05.6208457Z 1 5 0x740c, 45873 36.0°C N/A N/A, N/A, 0 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 2025-07-24T04:23:05.6209390Z 2 2 0x740c, 51627 31.0°C 92.0W N/A, N/A, 0 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 2025-07-24T04:23:05.6210292Z 3 3 0x740c, 64489 37.0°C N/A N/A, N/A, 0 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 2025-07-24T04:23:05.6211184Z 4 8 0x740c, 30939 31.0°C 95.0W N/A, N/A, 0 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 2025-07-24T04:23:05.6212122Z 5 9 0x740c, 8466 31.0°C N/A N/A, N/A, 0 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 2025-07-24T04:23:05.6212887Z 6 6 0x740c, 41154 34.0°C 95.0W N/A, N/A, 0 800Mhz 1600Mhz 0% auto 560.0W 0% 0% 2025-07-24T04:23:05.6213456Z 7 7 0x740c, 63755 32.0°C N/A N/A, N/A, 0 800Mhz 1600Mhz 0% auto 0.0W 0% 0% 2025-07-24T04:23:05.6218253Z ==================================================================================================================== 2025-07-24T04:23:05.6218625Z =============================================== End of ROCm SMI Log ================================================ 2025-07-24T04:23:05.6378523Z ##[group]Run rocminfo 2025-07-24T04:23:05.6378759Z rocminfo 2025-07-24T04:23:05.6404690Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:05.6405012Z env: 2025-07-24T04:23:05.6405209Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:05.6405415Z ##[endgroup] 2025-07-24T04:23:05.7735423Z ROCk module version 6.10.5 is loaded 2025-07-24T04:23:05.7735949Z ===================== 2025-07-24T04:23:05.7736297Z HSA System Attributes 2025-07-24T04:23:05.7736584Z ===================== 2025-07-24T04:23:05.7736874Z Runtime Version: 1.14 2025-07-24T04:23:05.7737206Z Runtime Ext Version: 1.6 2025-07-24T04:23:05.7737527Z System Timestamp Freq.: 1000.000000MHz 2025-07-24T04:23:05.7738065Z Sig. Max Wait Duration: 18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count) 2025-07-24T04:23:05.7738654Z Machine Model: LARGE 2025-07-24T04:23:05.7739143Z System Endianness: LITTLE 2025-07-24T04:23:05.7739558Z Mwaitx: DISABLED 2025-07-24T04:23:05.7739889Z DMAbuf Support: YES 2025-07-24T04:23:05.7740304Z 2025-07-24T04:23:05.7740403Z ========== 2025-07-24T04:23:05.7740655Z HSA Agents 2025-07-24T04:23:05.7740910Z ========== 2025-07-24T04:23:05.7741471Z ******* 2025-07-24T04:23:05.7741762Z Agent 1 2025-07-24T04:23:05.7742046Z ******* 2025-07-24T04:23:05.7742352Z Name: AMD EPYC 7713 64-Core Processor 2025-07-24T04:23:05.7742798Z Uuid: CPU-XX 2025-07-24T04:23:05.7743221Z Marketing Name: AMD EPYC 7713 64-Core Processor 2025-07-24T04:23:05.7743667Z Vendor Name: CPU 2025-07-24T04:23:05.7744083Z Feature: None specified 2025-07-24T04:23:05.7744533Z Profile: FULL_PROFILE 2025-07-24T04:23:05.7745071Z Float Round Mode: NEAR 2025-07-24T04:23:05.7745540Z Max Queue Number: 0(0x0) 2025-07-24T04:23:05.7745980Z Queue Min Size: 0(0x0) 2025-07-24T04:23:05.7746401Z Queue Max Size: 0(0x0) 2025-07-24T04:23:05.7747015Z Queue Type: MULTI 2025-07-24T04:23:05.7747403Z Node: 0 2025-07-24T04:23:05.7747816Z Device Type: CPU 2025-07-24T04:23:05.7748187Z Cache Info: 2025-07-24T04:23:05.7748510Z L1: 32768(0x8000) KB 2025-07-24T04:23:05.7748889Z Chip ID: 0(0x0) 2025-07-24T04:23:05.7749294Z ASIC Revision: 0(0x0) 2025-07-24T04:23:05.7749724Z Cacheline Size: 64(0x40) 2025-07-24T04:23:05.7750168Z Max Clock Freq. (MHz): 2000 2025-07-24T04:23:05.7750562Z BDFID: 0 2025-07-24T04:23:05.7750972Z Internal Node ID: 0 2025-07-24T04:23:05.7751398Z Compute Unit: 64 2025-07-24T04:23:05.7751819Z SIMDs per CU: 0 2025-07-24T04:23:05.7752232Z Shader Engines: 0 2025-07-24T04:23:05.7752668Z Shader Arrs. per Eng.: 0 2025-07-24T04:23:05.7753120Z WatchPts on Addr. Ranges:1 2025-07-24T04:23:05.7753512Z Memory Properties: 2025-07-24T04:23:05.7753797Z Features: None 2025-07-24T04:23:05.7754074Z Pool Info: 2025-07-24T04:23:05.7754350Z Pool 1 2025-07-24T04:23:05.7754689Z Segment: GLOBAL; FLAGS: FINE GRAINED 2025-07-24T04:23:05.7755115Z Size: 528249788(0x1f7c73bc) KB 2025-07-24T04:23:05.7755498Z Allocatable: TRUE 2025-07-24T04:23:05.7755822Z Alloc Granule: 4KB 2025-07-24T04:23:05.7756174Z Alloc Recommended Granule:4KB 2025-07-24T04:23:05.7756521Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7756857Z Accessible by all: TRUE 2025-07-24T04:23:05.7757139Z Pool 2 2025-07-24T04:23:05.7757413Z Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED 2025-07-24T04:23:05.7757731Z Size: 528249788(0x1f7c73bc) KB 2025-07-24T04:23:05.7758039Z Allocatable: TRUE 2025-07-24T04:23:05.7758377Z Alloc Granule: 4KB 2025-07-24T04:23:05.7758712Z Alloc Recommended Granule:4KB 2025-07-24T04:23:05.7759057Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7759536Z Accessible by all: TRUE 2025-07-24T04:23:05.7759838Z Pool 3 2025-07-24T04:23:05.7760097Z Segment: GLOBAL; FLAGS: KERNARG, FINE GRAINED 2025-07-24T04:23:05.7760413Z Size: 528249788(0x1f7c73bc) KB 2025-07-24T04:23:05.7760722Z Allocatable: TRUE 2025-07-24T04:23:05.7761043Z Alloc Granule: 4KB 2025-07-24T04:23:05.7761385Z Alloc Recommended Granule:4KB 2025-07-24T04:23:05.7761725Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7762062Z Accessible by all: TRUE 2025-07-24T04:23:05.7762428Z Pool 4 2025-07-24T04:23:05.7762699Z Segment: GLOBAL; FLAGS: COARSE GRAINED 2025-07-24T04:23:05.7763009Z Size: 528249788(0x1f7c73bc) KB 2025-07-24T04:23:05.7763322Z Allocatable: TRUE 2025-07-24T04:23:05.7763829Z Alloc Granule: 4KB 2025-07-24T04:23:05.7764166Z Alloc Recommended Granule:4KB 2025-07-24T04:23:05.7764513Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7764841Z Accessible by all: TRUE 2025-07-24T04:23:05.7765140Z ISA Info: 2025-07-24T04:23:05.7765351Z ******* 2025-07-24T04:23:05.7765544Z Agent 2 2025-07-24T04:23:05.7765747Z ******* 2025-07-24T04:23:05.7765979Z Name: AMD EPYC 7713 64-Core Processor 2025-07-24T04:23:05.7766284Z Uuid: CPU-XX 2025-07-24T04:23:05.7766603Z Marketing Name: AMD EPYC 7713 64-Core Processor 2025-07-24T04:23:05.7766948Z Vendor Name: CPU 2025-07-24T04:23:05.7767266Z Feature: None specified 2025-07-24T04:23:05.7767582Z Profile: FULL_PROFILE 2025-07-24T04:23:05.7767908Z Float Round Mode: NEAR 2025-07-24T04:23:05.7768232Z Max Queue Number: 0(0x0) 2025-07-24T04:23:05.7768551Z Queue Min Size: 0(0x0) 2025-07-24T04:23:05.7768860Z Queue Max Size: 0(0x0) 2025-07-24T04:23:05.7769169Z Queue Type: MULTI 2025-07-24T04:23:05.7769458Z Node: 1 2025-07-24T04:23:05.7769758Z Device Type: CPU 2025-07-24T04:23:05.7770037Z Cache Info: 2025-07-24T04:23:05.7770259Z L1: 32768(0x8000) KB 2025-07-24T04:23:05.7770549Z Chip ID: 0(0x0) 2025-07-24T04:23:05.7770851Z ASIC Revision: 0(0x0) 2025-07-24T04:23:05.7771173Z Cacheline Size: 64(0x40) 2025-07-24T04:23:05.7771490Z Max Clock Freq. (MHz): 2000 2025-07-24T04:23:05.7771796Z BDFID: 0 2025-07-24T04:23:05.7772105Z Internal Node ID: 1 2025-07-24T04:23:05.7772417Z Compute Unit: 64 2025-07-24T04:23:05.7772722Z SIMDs per CU: 0 2025-07-24T04:23:05.7773030Z Shader Engines: 0 2025-07-24T04:23:05.7773360Z Shader Arrs. per Eng.: 0 2025-07-24T04:23:05.7773732Z WatchPts on Addr. Ranges:1 2025-07-24T04:23:05.7774214Z Memory Properties: 2025-07-24T04:23:05.7774449Z Features: None 2025-07-24T04:23:05.7774666Z Pool Info: 2025-07-24T04:23:05.7774873Z Pool 1 2025-07-24T04:23:05.7775138Z Segment: GLOBAL; FLAGS: FINE GRAINED 2025-07-24T04:23:05.7775450Z Size: 528402436(0x1f7ec804) KB 2025-07-24T04:23:05.7775765Z Allocatable: TRUE 2025-07-24T04:23:05.7776084Z Alloc Granule: 4KB 2025-07-24T04:23:05.7776431Z Alloc Recommended Granule:4KB 2025-07-24T04:23:05.7776779Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7777109Z Accessible by all: TRUE 2025-07-24T04:23:05.7777399Z Pool 2 2025-07-24T04:23:05.7777658Z Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED 2025-07-24T04:23:05.7778102Z Size: 528402436(0x1f7ec804) KB 2025-07-24T04:23:05.7778402Z Allocatable: TRUE 2025-07-24T04:23:05.7778732Z Alloc Granule: 4KB 2025-07-24T04:23:05.7779066Z Alloc Recommended Granule:4KB 2025-07-24T04:23:05.7779413Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7779748Z Accessible by all: TRUE 2025-07-24T04:23:05.7780026Z Pool 3 2025-07-24T04:23:05.7780289Z Segment: GLOBAL; FLAGS: KERNARG, FINE GRAINED 2025-07-24T04:23:05.7780590Z Size: 528402436(0x1f7ec804) KB 2025-07-24T04:23:05.7780917Z Allocatable: TRUE 2025-07-24T04:23:05.7781242Z Alloc Granule: 4KB 2025-07-24T04:23:05.7781592Z Alloc Recommended Granule:4KB 2025-07-24T04:23:05.7781944Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7782285Z Accessible by all: TRUE 2025-07-24T04:23:05.7782582Z Pool 4 2025-07-24T04:23:05.7782839Z Segment: GLOBAL; FLAGS: COARSE GRAINED 2025-07-24T04:23:05.7783153Z Size: 528402436(0x1f7ec804) KB 2025-07-24T04:23:05.7783456Z Allocatable: TRUE 2025-07-24T04:23:05.7783786Z Alloc Granule: 4KB 2025-07-24T04:23:05.7784134Z Alloc Recommended Granule:4KB 2025-07-24T04:23:05.7784470Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7784821Z Accessible by all: TRUE 2025-07-24T04:23:05.7785116Z ISA Info: 2025-07-24T04:23:05.7785325Z ******* 2025-07-24T04:23:05.7785514Z Agent 3 2025-07-24T04:23:05.7785713Z ******* 2025-07-24T04:23:05.7785934Z Name: gfx90a 2025-07-24T04:23:05.7786233Z Uuid: GPU-0ca137a0ba8e063d 2025-07-24T04:23:05.7786558Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:23:05.7786882Z Vendor Name: AMD 2025-07-24T04:23:05.7787199Z Feature: KERNEL_DISPATCH 2025-07-24T04:23:05.7787510Z Profile: BASE_PROFILE 2025-07-24T04:23:05.7787841Z Float Round Mode: NEAR 2025-07-24T04:23:05.7788167Z Max Queue Number: 128(0x80) 2025-07-24T04:23:05.7788616Z Queue Min Size: 64(0x40) 2025-07-24T04:23:05.7788942Z Queue Max Size: 131072(0x20000) 2025-07-24T04:23:05.7789255Z Queue Type: MULTI 2025-07-24T04:23:05.7789552Z Node: 2 2025-07-24T04:23:05.7789844Z Device Type: GPU 2025-07-24T04:23:05.7790135Z Cache Info: 2025-07-24T04:23:05.7790372Z L1: 16(0x10) KB 2025-07-24T04:23:05.7790647Z L2: 8192(0x2000) KB 2025-07-24T04:23:05.7790935Z Chip ID: 29708(0x740c) 2025-07-24T04:23:05.7791236Z ASIC Revision: 1(0x1) 2025-07-24T04:23:05.7791563Z Cacheline Size: 128(0x80) 2025-07-24T04:23:05.7791899Z Max Clock Freq. (MHz): 1700 2025-07-24T04:23:05.7792210Z BDFID: 12800 2025-07-24T04:23:05.7792682Z Internal Node ID: 2 2025-07-24T04:23:05.7793015Z Compute Unit: 104 2025-07-24T04:23:05.7793331Z SIMDs per CU: 4 2025-07-24T04:23:05.7793655Z Shader Engines: 8 2025-07-24T04:23:05.7794000Z Shader Arrs. per Eng.: 1 2025-07-24T04:23:05.7794344Z WatchPts on Addr. Ranges:4 2025-07-24T04:23:05.7794693Z Coherent Host Access: FALSE 2025-07-24T04:23:05.7794998Z Memory Properties: 2025-07-24T04:23:05.7795252Z Features: KERNEL_DISPATCH 2025-07-24T04:23:05.7795563Z Fast F16 Operation: TRUE 2025-07-24T04:23:05.7795914Z Wavefront Size: 64(0x40) 2025-07-24T04:23:05.7796262Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:23:05.7796571Z Workgroup Max Size per Dimension: 2025-07-24T04:23:05.7796843Z x 1024(0x400) 2025-07-24T04:23:05.7797116Z y 1024(0x400) 2025-07-24T04:23:05.7797392Z z 1024(0x400) 2025-07-24T04:23:05.7797687Z Max Waves Per CU: 32(0x20) 2025-07-24T04:23:05.7798033Z Max Work-item Per CU: 2048(0x800) 2025-07-24T04:23:05.7798384Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:23:05.7798678Z Grid Max Size per Dimension: 2025-07-24T04:23:05.7798940Z x 4294967295(0xffffffff) 2025-07-24T04:23:05.7799225Z y 4294967295(0xffffffff) 2025-07-24T04:23:05.7799512Z z 4294967295(0xffffffff) 2025-07-24T04:23:05.7799826Z Max fbarriers/Workgrp: 32 2025-07-24T04:23:05.7803529Z Packet Processor uCode:: 92 2025-07-24T04:23:05.7803947Z SDMA engine uCode:: 9 2025-07-24T04:23:05.7804291Z IOMMU Support:: None 2025-07-24T04:23:05.7804587Z Pool Info: 2025-07-24T04:23:05.7804806Z Pool 1 2025-07-24T04:23:05.7805079Z Segment: GLOBAL; FLAGS: COARSE GRAINED 2025-07-24T04:23:05.7805404Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.7805720Z Allocatable: TRUE 2025-07-24T04:23:05.7806052Z Alloc Granule: 4KB 2025-07-24T04:23:05.7806627Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.7806989Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7807324Z Accessible by all: FALSE 2025-07-24T04:23:05.7807613Z Pool 2 2025-07-24T04:23:05.7807878Z Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED 2025-07-24T04:23:05.7808211Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.7808528Z Allocatable: TRUE 2025-07-24T04:23:05.7808856Z Alloc Granule: 4KB 2025-07-24T04:23:05.7809207Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.7809551Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7809896Z Accessible by all: FALSE 2025-07-24T04:23:05.7810182Z Pool 3 2025-07-24T04:23:05.7810454Z Segment: GLOBAL; FLAGS: FINE GRAINED 2025-07-24T04:23:05.7810919Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.7811227Z Allocatable: TRUE 2025-07-24T04:23:05.7811556Z Alloc Granule: 4KB 2025-07-24T04:23:05.7811894Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.7812247Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7812575Z Accessible by all: FALSE 2025-07-24T04:23:05.7812863Z Pool 4 2025-07-24T04:23:05.7813109Z Segment: GROUP 2025-07-24T04:23:05.7813423Z Size: 64(0x40) KB 2025-07-24T04:23:05.7813748Z Allocatable: FALSE 2025-07-24T04:23:05.7814078Z Alloc Granule: 0KB 2025-07-24T04:23:05.7814432Z Alloc Recommended Granule:0KB 2025-07-24T04:23:05.7814789Z Alloc Alignment: 0KB 2025-07-24T04:23:05.7815148Z Accessible by all: FALSE 2025-07-24T04:23:05.7815441Z ISA Info: 2025-07-24T04:23:05.7815658Z ISA 1 2025-07-24T04:23:05.7815946Z Name: amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack- 2025-07-24T04:23:05.7816301Z Machine Models: HSA_MACHINE_MODEL_LARGE 2025-07-24T04:23:05.7816646Z Profiles: HSA_PROFILE_BASE 2025-07-24T04:23:05.7816985Z Default Rounding Mode: NEAR 2025-07-24T04:23:05.7817339Z Default Rounding Mode: NEAR 2025-07-24T04:23:05.7817662Z Fast f16: TRUE 2025-07-24T04:23:05.7817993Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:23:05.7818311Z Workgroup Max Size per Dimension: 2025-07-24T04:23:05.7818589Z x 1024(0x400) 2025-07-24T04:23:05.7818869Z y 1024(0x400) 2025-07-24T04:23:05.7819132Z z 1024(0x400) 2025-07-24T04:23:05.7819434Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:23:05.7819724Z Grid Max Size per Dimension: 2025-07-24T04:23:05.7819970Z x 4294967295(0xffffffff) 2025-07-24T04:23:05.7820232Z y 4294967295(0xffffffff) 2025-07-24T04:23:05.7820505Z z 4294967295(0xffffffff) 2025-07-24T04:23:05.7820945Z FBarrier Max Size: 32 2025-07-24T04:23:05.7821231Z ******* 2025-07-24T04:23:05.7821443Z Agent 4 2025-07-24T04:23:05.7821635Z ******* 2025-07-24T04:23:05.7821862Z Name: gfx90a 2025-07-24T04:23:05.7822168Z Uuid: GPU-5ac94fbd079c5a52 2025-07-24T04:23:05.7822497Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:23:05.7822830Z Vendor Name: AMD 2025-07-24T04:23:05.7823147Z Feature: KERNEL_DISPATCH 2025-07-24T04:23:05.7823472Z Profile: BASE_PROFILE 2025-07-24T04:23:05.7823799Z Float Round Mode: NEAR 2025-07-24T04:23:05.7824131Z Max Queue Number: 128(0x80) 2025-07-24T04:23:05.7824446Z Queue Min Size: 64(0x40) 2025-07-24T04:23:05.7824766Z Queue Max Size: 131072(0x20000) 2025-07-24T04:23:05.7825230Z Queue Type: MULTI 2025-07-24T04:23:05.7825520Z Node: 3 2025-07-24T04:23:05.7825827Z Device Type: GPU 2025-07-24T04:23:05.7826105Z Cache Info: 2025-07-24T04:23:05.7826342Z L1: 16(0x10) KB 2025-07-24T04:23:05.7826618Z L2: 8192(0x2000) KB 2025-07-24T04:23:05.7826905Z Chip ID: 29708(0x740c) 2025-07-24T04:23:05.7827210Z ASIC Revision: 1(0x1) 2025-07-24T04:23:05.7827537Z Cacheline Size: 128(0x80) 2025-07-24T04:23:05.7827863Z Max Clock Freq. (MHz): 1700 2025-07-24T04:23:05.7828163Z BDFID: 13568 2025-07-24T04:23:05.7828479Z Internal Node ID: 3 2025-07-24T04:23:05.7828799Z Compute Unit: 104 2025-07-24T04:23:05.7829113Z SIMDs per CU: 4 2025-07-24T04:23:05.7829429Z Shader Engines: 8 2025-07-24T04:23:05.7829760Z Shader Arrs. per Eng.: 1 2025-07-24T04:23:05.7830108Z WatchPts on Addr. Ranges:4 2025-07-24T04:23:05.7830449Z Coherent Host Access: FALSE 2025-07-24T04:23:05.7830754Z Memory Properties: 2025-07-24T04:23:05.7830987Z Features: KERNEL_DISPATCH 2025-07-24T04:23:05.7831291Z Fast F16 Operation: TRUE 2025-07-24T04:23:05.7831620Z Wavefront Size: 64(0x40) 2025-07-24T04:23:05.7831961Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:23:05.7832273Z Workgroup Max Size per Dimension: 2025-07-24T04:23:05.7832529Z x 1024(0x400) 2025-07-24T04:23:05.7832800Z y 1024(0x400) 2025-07-24T04:23:05.7833063Z z 1024(0x400) 2025-07-24T04:23:05.7833369Z Max Waves Per CU: 32(0x20) 2025-07-24T04:23:05.7833687Z Max Work-item Per CU: 2048(0x800) 2025-07-24T04:23:05.7834020Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:23:05.7834311Z Grid Max Size per Dimension: 2025-07-24T04:23:05.7834547Z x 4294967295(0xffffffff) 2025-07-24T04:23:05.7834820Z y 4294967295(0xffffffff) 2025-07-24T04:23:05.7835084Z z 4294967295(0xffffffff) 2025-07-24T04:23:05.7835531Z Max fbarriers/Workgrp: 32 2025-07-24T04:23:05.7835896Z Packet Processor uCode:: 92 2025-07-24T04:23:05.7836236Z SDMA engine uCode:: 9 2025-07-24T04:23:05.7836560Z IOMMU Support:: None 2025-07-24T04:23:05.7836846Z Pool Info: 2025-07-24T04:23:05.7837058Z Pool 1 2025-07-24T04:23:05.7837324Z Segment: GLOBAL; FLAGS: COARSE GRAINED 2025-07-24T04:23:05.7837646Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.7837953Z Allocatable: TRUE 2025-07-24T04:23:05.7838282Z Alloc Granule: 4KB 2025-07-24T04:23:05.7838620Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.7838980Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7839454Z Accessible by all: FALSE 2025-07-24T04:23:05.7839743Z Pool 2 2025-07-24T04:23:05.7840007Z Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED 2025-07-24T04:23:05.7840317Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.7840632Z Allocatable: TRUE 2025-07-24T04:23:05.7840957Z Alloc Granule: 4KB 2025-07-24T04:23:05.7841305Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.7841652Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7841982Z Accessible by all: FALSE 2025-07-24T04:23:05.7842273Z Pool 3 2025-07-24T04:23:05.7842594Z Segment: GLOBAL; FLAGS: FINE GRAINED 2025-07-24T04:23:05.7842911Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.7843218Z Allocatable: TRUE 2025-07-24T04:23:05.7843554Z Alloc Granule: 4KB 2025-07-24T04:23:05.7843927Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.7844271Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7844613Z Accessible by all: FALSE 2025-07-24T04:23:05.7844896Z Pool 4 2025-07-24T04:23:05.7845148Z Segment: GROUP 2025-07-24T04:23:05.7845443Z Size: 64(0x40) KB 2025-07-24T04:23:05.7845752Z Allocatable: FALSE 2025-07-24T04:23:05.7846081Z Alloc Granule: 0KB 2025-07-24T04:23:05.7846420Z Alloc Recommended Granule:0KB 2025-07-24T04:23:05.7846776Z Alloc Alignment: 0KB 2025-07-24T04:23:05.7847107Z Accessible by all: FALSE 2025-07-24T04:23:05.7847401Z ISA Info: 2025-07-24T04:23:05.7847608Z ISA 1 2025-07-24T04:23:05.7847889Z Name: amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack- 2025-07-24T04:23:05.7848242Z Machine Models: HSA_MACHINE_MODEL_LARGE 2025-07-24T04:23:05.7848592Z Profiles: HSA_PROFILE_BASE 2025-07-24T04:23:05.7848935Z Default Rounding Mode: NEAR 2025-07-24T04:23:05.7849285Z Default Rounding Mode: NEAR 2025-07-24T04:23:05.7849609Z Fast f16: TRUE 2025-07-24T04:23:05.7850129Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:23:05.7850456Z Workgroup Max Size per Dimension: 2025-07-24T04:23:05.7850729Z x 1024(0x400) 2025-07-24T04:23:05.7851006Z y 1024(0x400) 2025-07-24T04:23:05.7851273Z z 1024(0x400) 2025-07-24T04:23:05.7851565Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:23:05.7851857Z Grid Max Size per Dimension: 2025-07-24T04:23:05.7852099Z x 4294967295(0xffffffff) 2025-07-24T04:23:05.7852377Z y 4294967295(0xffffffff) 2025-07-24T04:23:05.7852649Z z 4294967295(0xffffffff) 2025-07-24T04:23:05.7852959Z FBarrier Max Size: 32 2025-07-24T04:23:05.7853242Z ******* 2025-07-24T04:23:05.7853437Z Agent 5 2025-07-24T04:23:05.7853809Z ******* 2025-07-24T04:23:05.7854032Z Name: gfx90a 2025-07-24T04:23:05.7854340Z Uuid: GPU-5a57d8d7f376d86a 2025-07-24T04:23:05.7854670Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:23:05.7855006Z Vendor Name: AMD 2025-07-24T04:23:05.7855333Z Feature: KERNEL_DISPATCH 2025-07-24T04:23:05.7855657Z Profile: BASE_PROFILE 2025-07-24T04:23:05.7855982Z Float Round Mode: NEAR 2025-07-24T04:23:05.7856302Z Max Queue Number: 128(0x80) 2025-07-24T04:23:05.7856626Z Queue Min Size: 64(0x40) 2025-07-24T04:23:05.7856932Z Queue Max Size: 131072(0x20000) 2025-07-24T04:23:05.7857297Z Queue Type: MULTI 2025-07-24T04:23:05.7857596Z Node: 4 2025-07-24T04:23:05.7857901Z Device Type: GPU 2025-07-24T04:23:05.7858186Z Cache Info: 2025-07-24T04:23:05.7858416Z L1: 16(0x10) KB 2025-07-24T04:23:05.7858691Z L2: 8192(0x2000) KB 2025-07-24T04:23:05.7858971Z Chip ID: 29708(0x740c) 2025-07-24T04:23:05.7859287Z ASIC Revision: 1(0x1) 2025-07-24T04:23:05.7859611Z Cacheline Size: 128(0x80) 2025-07-24T04:23:05.7859941Z Max Clock Freq. (MHz): 1700 2025-07-24T04:23:05.7860252Z BDFID: 4352 2025-07-24T04:23:05.7860559Z Internal Node ID: 4 2025-07-24T04:23:05.7860890Z Compute Unit: 104 2025-07-24T04:23:05.7861198Z SIMDs per CU: 4 2025-07-24T04:23:05.7861520Z Shader Engines: 8 2025-07-24T04:23:05.7861842Z Shader Arrs. per Eng.: 1 2025-07-24T04:23:05.7862186Z WatchPts on Addr. Ranges:4 2025-07-24T04:23:05.7862536Z Coherent Host Access: FALSE 2025-07-24T04:23:05.7862834Z Memory Properties: 2025-07-24T04:23:05.7863078Z Features: KERNEL_DISPATCH 2025-07-24T04:23:05.7863379Z Fast F16 Operation: TRUE 2025-07-24T04:23:05.7863715Z Wavefront Size: 64(0x40) 2025-07-24T04:23:05.7864047Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:23:05.7864514Z Workgroup Max Size per Dimension: 2025-07-24T04:23:05.7864778Z x 1024(0x400) 2025-07-24T04:23:05.7865053Z y 1024(0x400) 2025-07-24T04:23:05.7865318Z z 1024(0x400) 2025-07-24T04:23:05.7865605Z Max Waves Per CU: 32(0x20) 2025-07-24T04:23:05.7865940Z Max Work-item Per CU: 2048(0x800) 2025-07-24T04:23:05.7866257Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:23:05.7866548Z Grid Max Size per Dimension: 2025-07-24T04:23:05.7866783Z x 4294967295(0xffffffff) 2025-07-24T04:23:05.7867056Z y 4294967295(0xffffffff) 2025-07-24T04:23:05.7867329Z z 4294967295(0xffffffff) 2025-07-24T04:23:05.7867640Z Max fbarriers/Workgrp: 32 2025-07-24T04:23:05.7868001Z Packet Processor uCode:: 92 2025-07-24T04:23:05.7868474Z SDMA engine uCode:: 9 2025-07-24T04:23:05.7868802Z IOMMU Support:: None 2025-07-24T04:23:05.7869082Z Pool Info: 2025-07-24T04:23:05.7869294Z Pool 1 2025-07-24T04:23:05.7869561Z Segment: GLOBAL; FLAGS: COARSE GRAINED 2025-07-24T04:23:05.7869880Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.7870207Z Allocatable: TRUE 2025-07-24T04:23:05.7870539Z Alloc Granule: 4KB 2025-07-24T04:23:05.7870888Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.7871231Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7871574Z Accessible by all: FALSE 2025-07-24T04:23:05.7871871Z Pool 2 2025-07-24T04:23:05.7872129Z Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED 2025-07-24T04:23:05.7872449Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.7872754Z Allocatable: TRUE 2025-07-24T04:23:05.7873083Z Alloc Granule: 4KB 2025-07-24T04:23:05.7873423Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.7873772Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7874109Z Accessible by all: FALSE 2025-07-24T04:23:05.7874391Z Pool 3 2025-07-24T04:23:05.7874651Z Segment: GLOBAL; FLAGS: FINE GRAINED 2025-07-24T04:23:05.7874958Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.7875281Z Allocatable: TRUE 2025-07-24T04:23:05.7875607Z Alloc Granule: 4KB 2025-07-24T04:23:05.7875956Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.7876292Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7876631Z Accessible by all: FALSE 2025-07-24T04:23:05.7876923Z Pool 4 2025-07-24T04:23:05.7877164Z Segment: GROUP 2025-07-24T04:23:05.7899456Z Size: 64(0x40) KB 2025-07-24T04:23:05.7899859Z Allocatable: FALSE 2025-07-24T04:23:05.7900214Z Alloc Granule: 0KB 2025-07-24T04:23:05.7900804Z Alloc Recommended Granule:0KB 2025-07-24T04:23:05.7901182Z Alloc Alignment: 0KB 2025-07-24T04:23:05.7901542Z Accessible by all: FALSE 2025-07-24T04:23:05.7901835Z ISA Info: 2025-07-24T04:23:05.7902061Z ISA 1 2025-07-24T04:23:05.7902344Z Name: amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack- 2025-07-24T04:23:05.7902750Z Machine Models: HSA_MACHINE_MODEL_LARGE 2025-07-24T04:23:05.7903119Z Profiles: HSA_PROFILE_BASE 2025-07-24T04:23:05.7903484Z Default Rounding Mode: NEAR 2025-07-24T04:23:05.7903845Z Default Rounding Mode: NEAR 2025-07-24T04:23:05.7904187Z Fast f16: TRUE 2025-07-24T04:23:05.7904529Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:23:05.7904850Z Workgroup Max Size per Dimension: 2025-07-24T04:23:05.7905309Z x 1024(0x400) 2025-07-24T04:23:05.7905587Z y 1024(0x400) 2025-07-24T04:23:05.7905860Z z 1024(0x400) 2025-07-24T04:23:05.7906154Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:23:05.7906462Z Grid Max Size per Dimension: 2025-07-24T04:23:05.7906728Z x 4294967295(0xffffffff) 2025-07-24T04:23:05.7906996Z y 4294967295(0xffffffff) 2025-07-24T04:23:05.7907272Z z 4294967295(0xffffffff) 2025-07-24T04:23:05.7907582Z FBarrier Max Size: 32 2025-07-24T04:23:05.7907876Z ******* 2025-07-24T04:23:05.7908087Z Agent 6 2025-07-24T04:23:05.7908306Z ******* 2025-07-24T04:23:05.7908544Z Name: gfx90a 2025-07-24T04:23:05.7908872Z Uuid: GPU-869e5dfc4dcadcc3 2025-07-24T04:23:05.7909215Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:23:05.7909558Z Vendor Name: AMD 2025-07-24T04:23:05.7909888Z Feature: KERNEL_DISPATCH 2025-07-24T04:23:05.7910207Z Profile: BASE_PROFILE 2025-07-24T04:23:05.7910547Z Float Round Mode: NEAR 2025-07-24T04:23:05.7910887Z Max Queue Number: 128(0x80) 2025-07-24T04:23:05.7911229Z Queue Min Size: 64(0x40) 2025-07-24T04:23:05.7911552Z Queue Max Size: 131072(0x20000) 2025-07-24T04:23:05.7911874Z Queue Type: MULTI 2025-07-24T04:23:05.7912182Z Node: 5 2025-07-24T04:23:05.7912485Z Device Type: GPU 2025-07-24T04:23:05.7912779Z Cache Info: 2025-07-24T04:23:05.7913018Z L1: 16(0x10) KB 2025-07-24T04:23:05.7913314Z L2: 8192(0x2000) KB 2025-07-24T04:23:05.7913622Z Chip ID: 29708(0x740c) 2025-07-24T04:23:05.7913944Z ASIC Revision: 1(0x1) 2025-07-24T04:23:05.7914288Z Cacheline Size: 128(0x80) 2025-07-24T04:23:05.7914624Z Max Clock Freq. (MHz): 1700 2025-07-24T04:23:05.7914938Z BDFID: 5120 2025-07-24T04:23:05.7915241Z Internal Node ID: 5 2025-07-24T04:23:05.7915737Z Compute Unit: 104 2025-07-24T04:23:05.7916080Z SIMDs per CU: 4 2025-07-24T04:23:05.7916398Z Shader Engines: 8 2025-07-24T04:23:05.7916747Z Shader Arrs. per Eng.: 1 2025-07-24T04:23:05.7917098Z WatchPts on Addr. Ranges:4 2025-07-24T04:23:05.7917468Z Coherent Host Access: FALSE 2025-07-24T04:23:05.7917789Z Memory Properties: 2025-07-24T04:23:05.7918057Z Features: KERNEL_DISPATCH 2025-07-24T04:23:05.7918372Z Fast F16 Operation: TRUE 2025-07-24T04:23:05.7918731Z Wavefront Size: 64(0x40) 2025-07-24T04:23:05.7919090Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:23:05.7919408Z Workgroup Max Size per Dimension: 2025-07-24T04:23:05.7919693Z x 1024(0x400) 2025-07-24T04:23:05.7920098Z y 1024(0x400) 2025-07-24T04:23:05.7920372Z z 1024(0x400) 2025-07-24T04:23:05.7920685Z Max Waves Per CU: 32(0x20) 2025-07-24T04:23:05.7921033Z Max Work-item Per CU: 2048(0x800) 2025-07-24T04:23:05.7921371Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:23:05.7921670Z Grid Max Size per Dimension: 2025-07-24T04:23:05.7921927Z x 4294967295(0xffffffff) 2025-07-24T04:23:05.7922226Z y 4294967295(0xffffffff) 2025-07-24T04:23:05.7922602Z z 4294967295(0xffffffff) 2025-07-24T04:23:05.7922923Z Max fbarriers/Workgrp: 32 2025-07-24T04:23:05.7923319Z Packet Processor uCode:: 92 2025-07-24T04:23:05.7923684Z SDMA engine uCode:: 9 2025-07-24T04:23:05.7924009Z IOMMU Support:: None 2025-07-24T04:23:05.7924299Z Pool Info: 2025-07-24T04:23:05.7924507Z Pool 1 2025-07-24T04:23:05.7924794Z Segment: GLOBAL; FLAGS: COARSE GRAINED 2025-07-24T04:23:05.7925119Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.7925446Z Allocatable: TRUE 2025-07-24T04:23:05.7925786Z Alloc Granule: 4KB 2025-07-24T04:23:05.7926124Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.7926475Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7926815Z Accessible by all: FALSE 2025-07-24T04:23:05.7927125Z Pool 2 2025-07-24T04:23:05.7927398Z Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED 2025-07-24T04:23:05.7927734Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.7928069Z Allocatable: TRUE 2025-07-24T04:23:05.7928397Z Alloc Granule: 4KB 2025-07-24T04:23:05.7928752Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.7929096Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7929442Z Accessible by all: FALSE 2025-07-24T04:23:05.7929731Z Pool 3 2025-07-24T04:23:05.7930001Z Segment: GLOBAL; FLAGS: FINE GRAINED 2025-07-24T04:23:05.7930310Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.7930802Z Allocatable: TRUE 2025-07-24T04:23:05.7931157Z Alloc Granule: 4KB 2025-07-24T04:23:05.7931508Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.7931870Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7932210Z Accessible by all: FALSE 2025-07-24T04:23:05.7932517Z Pool 4 2025-07-24T04:23:05.7932767Z Segment: GROUP 2025-07-24T04:23:05.7933089Z Size: 64(0x40) KB 2025-07-24T04:23:05.7933411Z Allocatable: FALSE 2025-07-24T04:23:05.7933743Z Alloc Granule: 0KB 2025-07-24T04:23:05.7934091Z Alloc Recommended Granule:0KB 2025-07-24T04:23:05.7934440Z Alloc Alignment: 0KB 2025-07-24T04:23:05.7934948Z Accessible by all: FALSE 2025-07-24T04:23:05.7935239Z ISA Info: 2025-07-24T04:23:05.7935471Z ISA 1 2025-07-24T04:23:05.7935765Z Name: amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack- 2025-07-24T04:23:05.7936134Z Machine Models: HSA_MACHINE_MODEL_LARGE 2025-07-24T04:23:05.7936501Z Profiles: HSA_PROFILE_BASE 2025-07-24T04:23:05.7936838Z Default Rounding Mode: NEAR 2025-07-24T04:23:05.7937218Z Default Rounding Mode: NEAR 2025-07-24T04:23:05.7937541Z Fast f16: TRUE 2025-07-24T04:23:05.7937871Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:23:05.7938195Z Workgroup Max Size per Dimension: 2025-07-24T04:23:05.7938477Z x 1024(0x400) 2025-07-24T04:23:05.7938769Z y 1024(0x400) 2025-07-24T04:23:05.7939029Z z 1024(0x400) 2025-07-24T04:23:05.7939340Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:23:05.7939635Z Grid Max Size per Dimension: 2025-07-24T04:23:05.7939896Z x 4294967295(0xffffffff) 2025-07-24T04:23:05.7940171Z y 4294967295(0xffffffff) 2025-07-24T04:23:05.7940452Z z 4294967295(0xffffffff) 2025-07-24T04:23:05.7941034Z FBarrier Max Size: 32 2025-07-24T04:23:05.7941327Z ******* 2025-07-24T04:23:05.7941547Z Agent 7 2025-07-24T04:23:05.7941741Z ******* 2025-07-24T04:23:05.7941987Z Name: gfx90a 2025-07-24T04:23:05.7942313Z Uuid: GPU-2d13401035fea6d5 2025-07-24T04:23:05.7942636Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:23:05.7943029Z Vendor Name: AMD 2025-07-24T04:23:05.7943343Z Feature: KERNEL_DISPATCH 2025-07-24T04:23:05.7943669Z Profile: BASE_PROFILE 2025-07-24T04:23:05.7943992Z Float Round Mode: NEAR 2025-07-24T04:23:05.7944328Z Max Queue Number: 128(0x80) 2025-07-24T04:23:05.7944663Z Queue Min Size: 64(0x40) 2025-07-24T04:23:05.7944979Z Queue Max Size: 131072(0x20000) 2025-07-24T04:23:05.7945308Z Queue Type: MULTI 2025-07-24T04:23:05.7945751Z Node: 6 2025-07-24T04:23:05.7946066Z Device Type: GPU 2025-07-24T04:23:05.7946344Z Cache Info: 2025-07-24T04:23:05.7946581Z L1: 16(0x10) KB 2025-07-24T04:23:05.7946867Z L2: 8192(0x2000) KB 2025-07-24T04:23:05.7947148Z Chip ID: 29708(0x740c) 2025-07-24T04:23:05.7947470Z ASIC Revision: 1(0x1) 2025-07-24T04:23:05.7947808Z Cacheline Size: 128(0x80) 2025-07-24T04:23:05.7948141Z Max Clock Freq. (MHz): 1700 2025-07-24T04:23:05.7948442Z BDFID: 44544 2025-07-24T04:23:05.7948749Z Internal Node ID: 6 2025-07-24T04:23:05.7949092Z Compute Unit: 104 2025-07-24T04:23:05.7949417Z SIMDs per CU: 4 2025-07-24T04:23:05.7949888Z Shader Engines: 8 2025-07-24T04:23:05.7950216Z Shader Arrs. per Eng.: 1 2025-07-24T04:23:05.7950562Z WatchPts on Addr. Ranges:4 2025-07-24T04:23:05.7950907Z Coherent Host Access: FALSE 2025-07-24T04:23:05.7951208Z Memory Properties: 2025-07-24T04:23:05.7951468Z Features: KERNEL_DISPATCH 2025-07-24T04:23:05.7951771Z Fast F16 Operation: TRUE 2025-07-24T04:23:05.7952118Z Wavefront Size: 64(0x40) 2025-07-24T04:23:05.7952445Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:23:05.7952756Z Workgroup Max Size per Dimension: 2025-07-24T04:23:05.7953016Z x 1024(0x400) 2025-07-24T04:23:05.7953290Z y 1024(0x400) 2025-07-24T04:23:05.7953569Z z 1024(0x400) 2025-07-24T04:23:05.7953864Z Max Waves Per CU: 32(0x20) 2025-07-24T04:23:05.7954204Z Max Work-item Per CU: 2048(0x800) 2025-07-24T04:23:05.7954531Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:23:05.7954833Z Grid Max Size per Dimension: 2025-07-24T04:23:05.7955063Z x 4294967295(0xffffffff) 2025-07-24T04:23:05.7955353Z y 4294967295(0xffffffff) 2025-07-24T04:23:05.7955623Z z 4294967295(0xffffffff) 2025-07-24T04:23:05.7955926Z Max fbarriers/Workgrp: 32 2025-07-24T04:23:05.7956281Z Packet Processor uCode:: 92 2025-07-24T04:23:05.7956622Z SDMA engine uCode:: 9 2025-07-24T04:23:05.7956968Z IOMMU Support:: None 2025-07-24T04:23:05.7957259Z Pool Info: 2025-07-24T04:23:05.7957468Z Pool 1 2025-07-24T04:23:05.7957743Z Segment: GLOBAL; FLAGS: COARSE GRAINED 2025-07-24T04:23:05.7958058Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.7958370Z Allocatable: TRUE 2025-07-24T04:23:05.7958694Z Alloc Granule: 4KB 2025-07-24T04:23:05.7959035Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.7959378Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7959716Z Accessible by all: FALSE 2025-07-24T04:23:05.7960006Z Pool 2 2025-07-24T04:23:05.7960396Z Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED 2025-07-24T04:23:05.7960724Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.7961026Z Allocatable: TRUE 2025-07-24T04:23:05.7961351Z Alloc Granule: 4KB 2025-07-24T04:23:05.7961688Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.7962030Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7962432Z Accessible by all: FALSE 2025-07-24T04:23:05.7962725Z Pool 3 2025-07-24T04:23:05.7962977Z Segment: GLOBAL; FLAGS: FINE GRAINED 2025-07-24T04:23:05.7963287Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.7963594Z Allocatable: TRUE 2025-07-24T04:23:05.7963921Z Alloc Granule: 4KB 2025-07-24T04:23:05.7964430Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.7964765Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7965102Z Accessible by all: FALSE 2025-07-24T04:23:05.7965381Z Pool 4 2025-07-24T04:23:05.7965636Z Segment: GROUP 2025-07-24T04:23:05.7965935Z Size: 64(0x40) KB 2025-07-24T04:23:05.7966236Z Allocatable: FALSE 2025-07-24T04:23:05.7966569Z Alloc Granule: 0KB 2025-07-24T04:23:05.7966902Z Alloc Recommended Granule:0KB 2025-07-24T04:23:05.7967251Z Alloc Alignment: 0KB 2025-07-24T04:23:05.7967583Z Accessible by all: FALSE 2025-07-24T04:23:05.7967884Z ISA Info: 2025-07-24T04:23:05.7968098Z ISA 1 2025-07-24T04:23:05.7968364Z Name: amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack- 2025-07-24T04:23:05.7968728Z Machine Models: HSA_MACHINE_MODEL_LARGE 2025-07-24T04:23:05.7969067Z Profiles: HSA_PROFILE_BASE 2025-07-24T04:23:05.7969411Z Default Rounding Mode: NEAR 2025-07-24T04:23:05.7969754Z Default Rounding Mode: NEAR 2025-07-24T04:23:05.7970079Z Fast f16: TRUE 2025-07-24T04:23:05.7970405Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:23:05.7970710Z Workgroup Max Size per Dimension: 2025-07-24T04:23:05.7970984Z x 1024(0x400) 2025-07-24T04:23:05.7971259Z y 1024(0x400) 2025-07-24T04:23:05.7971538Z z 1024(0x400) 2025-07-24T04:23:05.7971831Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:23:05.7972128Z Grid Max Size per Dimension: 2025-07-24T04:23:05.7972380Z x 4294967295(0xffffffff) 2025-07-24T04:23:05.7972652Z y 4294967295(0xffffffff) 2025-07-24T04:23:05.7972930Z z 4294967295(0xffffffff) 2025-07-24T04:23:05.7973234Z FBarrier Max Size: 32 2025-07-24T04:23:05.7973522Z ******* 2025-07-24T04:23:05.7973725Z Agent 8 2025-07-24T04:23:05.7973926Z ******* 2025-07-24T04:23:05.7974150Z Name: gfx90a 2025-07-24T04:23:05.7974636Z Uuid: GPU-d15980d687168477 2025-07-24T04:23:05.7974967Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:23:05.7975288Z Vendor Name: AMD 2025-07-24T04:23:05.7975611Z Feature: KERNEL_DISPATCH 2025-07-24T04:23:05.7975923Z Profile: BASE_PROFILE 2025-07-24T04:23:05.7976247Z Float Round Mode: NEAR 2025-07-24T04:23:05.7976571Z Max Queue Number: 128(0x80) 2025-07-24T04:23:05.7976887Z Queue Min Size: 64(0x40) 2025-07-24T04:23:05.7977199Z Queue Max Size: 131072(0x20000) 2025-07-24T04:23:05.7977504Z Queue Type: MULTI 2025-07-24T04:23:05.7977798Z Node: 7 2025-07-24T04:23:05.7978095Z Device Type: GPU 2025-07-24T04:23:05.7978503Z Cache Info: 2025-07-24T04:23:05.7978724Z L1: 16(0x10) KB 2025-07-24T04:23:05.7979007Z L2: 8192(0x2000) KB 2025-07-24T04:23:05.7979280Z Chip ID: 29708(0x740c) 2025-07-24T04:23:05.7979587Z ASIC Revision: 1(0x1) 2025-07-24T04:23:05.7979916Z Cacheline Size: 128(0x80) 2025-07-24T04:23:05.7980234Z Max Clock Freq. (MHz): 1700 2025-07-24T04:23:05.7980539Z BDFID: 45824 2025-07-24T04:23:05.7980842Z Internal Node ID: 7 2025-07-24T04:23:05.7981164Z Compute Unit: 104 2025-07-24T04:23:05.7981493Z SIMDs per CU: 4 2025-07-24T04:23:05.7981823Z Shader Engines: 8 2025-07-24T04:23:05.7982161Z Shader Arrs. per Eng.: 1 2025-07-24T04:23:05.7982489Z WatchPts on Addr. Ranges:4 2025-07-24T04:23:05.7982829Z Coherent Host Access: FALSE 2025-07-24T04:23:05.7983119Z Memory Properties: 2025-07-24T04:23:05.7983361Z Features: KERNEL_DISPATCH 2025-07-24T04:23:05.7983656Z Fast F16 Operation: TRUE 2025-07-24T04:23:05.7983985Z Wavefront Size: 64(0x40) 2025-07-24T04:23:05.7984313Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:23:05.7984608Z Workgroup Max Size per Dimension: 2025-07-24T04:23:05.7984862Z x 1024(0x400) 2025-07-24T04:23:05.7985127Z y 1024(0x400) 2025-07-24T04:23:05.7985390Z z 1024(0x400) 2025-07-24T04:23:05.7985680Z Max Waves Per CU: 32(0x20) 2025-07-24T04:23:05.7986006Z Max Work-item Per CU: 2048(0x800) 2025-07-24T04:23:05.7986330Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:23:05.7986612Z Grid Max Size per Dimension: 2025-07-24T04:23:05.7986846Z x 4294967295(0xffffffff) 2025-07-24T04:23:05.7987109Z y 4294967295(0xffffffff) 2025-07-24T04:23:05.7987376Z z 4294967295(0xffffffff) 2025-07-24T04:23:05.7987679Z Max fbarriers/Workgrp: 32 2025-07-24T04:23:05.7988032Z Packet Processor uCode:: 92 2025-07-24T04:23:05.7988376Z SDMA engine uCode:: 9 2025-07-24T04:23:05.7988836Z IOMMU Support:: None 2025-07-24T04:23:05.7989138Z Pool Info: 2025-07-24T04:23:05.7989344Z Pool 1 2025-07-24T04:23:05.7989612Z Segment: GLOBAL; FLAGS: COARSE GRAINED 2025-07-24T04:23:05.7989928Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.7990255Z Allocatable: TRUE 2025-07-24T04:23:05.7990582Z Alloc Granule: 4KB 2025-07-24T04:23:05.7990930Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.7991283Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7991610Z Accessible by all: FALSE 2025-07-24T04:23:05.7991899Z Pool 2 2025-07-24T04:23:05.7992158Z Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED 2025-07-24T04:23:05.7992481Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.7992915Z Allocatable: TRUE 2025-07-24T04:23:05.7993242Z Alloc Granule: 4KB 2025-07-24T04:23:05.7993585Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.7993922Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7994258Z Accessible by all: FALSE 2025-07-24T04:23:05.7994542Z Pool 3 2025-07-24T04:23:05.7994800Z Segment: GLOBAL; FLAGS: FINE GRAINED 2025-07-24T04:23:05.7995101Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.7995413Z Allocatable: TRUE 2025-07-24T04:23:05.7995737Z Alloc Granule: 4KB 2025-07-24T04:23:05.7996081Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.7996434Z Alloc Alignment: 4KB 2025-07-24T04:23:05.7996763Z Accessible by all: FALSE 2025-07-24T04:23:05.7997049Z Pool 4 2025-07-24T04:23:05.7997291Z Segment: GROUP 2025-07-24T04:23:05.7997588Z Size: 64(0x40) KB 2025-07-24T04:23:05.7997894Z Allocatable: FALSE 2025-07-24T04:23:05.7998214Z Alloc Granule: 0KB 2025-07-24T04:23:05.7998550Z Alloc Recommended Granule:0KB 2025-07-24T04:23:05.7998889Z Alloc Alignment: 0KB 2025-07-24T04:23:05.7999224Z Accessible by all: FALSE 2025-07-24T04:23:05.7999514Z ISA Info: 2025-07-24T04:23:05.7999727Z ISA 1 2025-07-24T04:23:05.7999990Z Name: amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack- 2025-07-24T04:23:05.8000346Z Machine Models: HSA_MACHINE_MODEL_LARGE 2025-07-24T04:23:05.8000688Z Profiles: HSA_PROFILE_BASE 2025-07-24T04:23:05.8001022Z Default Rounding Mode: NEAR 2025-07-24T04:23:05.8001368Z Default Rounding Mode: NEAR 2025-07-24T04:23:05.8001688Z Fast f16: TRUE 2025-07-24T04:23:05.8002013Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:23:05.8002318Z Workgroup Max Size per Dimension: 2025-07-24T04:23:05.8002654Z x 1024(0x400) 2025-07-24T04:23:05.8003087Z y 1024(0x400) 2025-07-24T04:23:05.8003355Z z 1024(0x400) 2025-07-24T04:23:05.8003659Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:23:05.8003945Z Grid Max Size per Dimension: 2025-07-24T04:23:05.8004188Z x 4294967295(0xffffffff) 2025-07-24T04:23:05.8004459Z y 4294967295(0xffffffff) 2025-07-24T04:23:05.8004731Z z 4294967295(0xffffffff) 2025-07-24T04:23:05.8005036Z FBarrier Max Size: 32 2025-07-24T04:23:05.8005312Z ******* 2025-07-24T04:23:05.8005517Z Agent 9 2025-07-24T04:23:05.8005708Z ******* 2025-07-24T04:23:05.8005938Z Name: gfx90a 2025-07-24T04:23:05.8006228Z Uuid: GPU-61b229e107271af2 2025-07-24T04:23:05.8006560Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:23:05.8007047Z Vendor Name: AMD 2025-07-24T04:23:05.8007365Z Feature: KERNEL_DISPATCH 2025-07-24T04:23:05.8007686Z Profile: BASE_PROFILE 2025-07-24T04:23:05.8008005Z Float Round Mode: NEAR 2025-07-24T04:23:05.8008333Z Max Queue Number: 128(0x80) 2025-07-24T04:23:05.8008646Z Queue Min Size: 64(0x40) 2025-07-24T04:23:05.8008958Z Queue Max Size: 131072(0x20000) 2025-07-24T04:23:05.8009272Z Queue Type: MULTI 2025-07-24T04:23:05.8009628Z Node: 8 2025-07-24T04:23:05.8009922Z Device Type: GPU 2025-07-24T04:23:05.8010214Z Cache Info: 2025-07-24T04:23:05.8010445Z L1: 16(0x10) KB 2025-07-24T04:23:05.8010722Z L2: 8192(0x2000) KB 2025-07-24T04:23:05.8010995Z Chip ID: 29708(0x740c) 2025-07-24T04:23:05.8011302Z ASIC Revision: 1(0x1) 2025-07-24T04:23:05.8011622Z Cacheline Size: 128(0x80) 2025-07-24T04:23:05.8011947Z Max Clock Freq. (MHz): 1700 2025-07-24T04:23:05.8012247Z BDFID: 36352 2025-07-24T04:23:05.8012549Z Internal Node ID: 8 2025-07-24T04:23:05.8012867Z Compute Unit: 104 2025-07-24T04:23:05.8013167Z SIMDs per CU: 4 2025-07-24T04:23:05.8013486Z Shader Engines: 8 2025-07-24T04:23:05.8013819Z Shader Arrs. per Eng.: 1 2025-07-24T04:23:05.8014155Z WatchPts on Addr. Ranges:4 2025-07-24T04:23:05.8014499Z Coherent Host Access: FALSE 2025-07-24T04:23:05.8014791Z Memory Properties: 2025-07-24T04:23:05.8015029Z Features: KERNEL_DISPATCH 2025-07-24T04:23:05.8015323Z Fast F16 Operation: TRUE 2025-07-24T04:23:05.8015665Z Wavefront Size: 64(0x40) 2025-07-24T04:23:05.8015989Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:23:05.8016291Z Workgroup Max Size per Dimension: 2025-07-24T04:23:05.8016544Z x 1024(0x400) 2025-07-24T04:23:05.8016805Z y 1024(0x400) 2025-07-24T04:23:05.8017202Z z 1024(0x400) 2025-07-24T04:23:05.8017498Z Max Waves Per CU: 32(0x20) 2025-07-24T04:23:05.8017823Z Max Work-item Per CU: 2048(0x800) 2025-07-24T04:23:05.8018142Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:23:05.8018428Z Grid Max Size per Dimension: 2025-07-24T04:23:05.8018664Z x 4294967295(0xffffffff) 2025-07-24T04:23:05.8018929Z y 4294967295(0xffffffff) 2025-07-24T04:23:05.8019195Z z 4294967295(0xffffffff) 2025-07-24T04:23:05.8019495Z Max fbarriers/Workgrp: 32 2025-07-24T04:23:05.8019852Z Packet Processor uCode:: 92 2025-07-24T04:23:05.8020188Z SDMA engine uCode:: 9 2025-07-24T04:23:05.8020517Z IOMMU Support:: None 2025-07-24T04:23:05.8020810Z Pool Info: 2025-07-24T04:23:05.8021177Z Pool 1 2025-07-24T04:23:05.8021445Z Segment: GLOBAL; FLAGS: COARSE GRAINED 2025-07-24T04:23:05.8021764Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.8022079Z Allocatable: TRUE 2025-07-24T04:23:05.8022404Z Alloc Granule: 4KB 2025-07-24T04:23:05.8022747Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.8023094Z Alloc Alignment: 4KB 2025-07-24T04:23:05.8023424Z Accessible by all: FALSE 2025-07-24T04:23:05.8023713Z Pool 2 2025-07-24T04:23:05.8023977Z Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED 2025-07-24T04:23:05.8024300Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.8024601Z Allocatable: TRUE 2025-07-24T04:23:05.8024940Z Alloc Granule: 4KB 2025-07-24T04:23:05.8025279Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.8025625Z Alloc Alignment: 4KB 2025-07-24T04:23:05.8025961Z Accessible by all: FALSE 2025-07-24T04:23:05.8026243Z Pool 3 2025-07-24T04:23:05.8026500Z Segment: GLOBAL; FLAGS: FINE GRAINED 2025-07-24T04:23:05.8026807Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.8027116Z Allocatable: TRUE 2025-07-24T04:23:05.8027433Z Alloc Granule: 4KB 2025-07-24T04:23:05.8027776Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.8028121Z Alloc Alignment: 4KB 2025-07-24T04:23:05.8028454Z Accessible by all: FALSE 2025-07-24T04:23:05.8028741Z Pool 4 2025-07-24T04:23:05.8028979Z Segment: GROUP 2025-07-24T04:23:05.8029281Z Size: 64(0x40) KB 2025-07-24T04:23:05.8029578Z Allocatable: FALSE 2025-07-24T04:23:05.8029902Z Alloc Granule: 0KB 2025-07-24T04:23:05.8030239Z Alloc Recommended Granule:0KB 2025-07-24T04:23:05.8030574Z Alloc Alignment: 0KB 2025-07-24T04:23:05.8030905Z Accessible by all: FALSE 2025-07-24T04:23:05.8031188Z ISA Info: 2025-07-24T04:23:05.8031529Z ISA 1 2025-07-24T04:23:05.8031796Z Name: amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack- 2025-07-24T04:23:05.8032157Z Machine Models: HSA_MACHINE_MODEL_LARGE 2025-07-24T04:23:05.8032498Z Profiles: HSA_PROFILE_BASE 2025-07-24T04:23:05.8032831Z Default Rounding Mode: NEAR 2025-07-24T04:23:05.8033182Z Default Rounding Mode: NEAR 2025-07-24T04:23:05.8033496Z Fast f16: TRUE 2025-07-24T04:23:05.8033820Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:23:05.8034122Z Workgroup Max Size per Dimension: 2025-07-24T04:23:05.8034388Z x 1024(0x400) 2025-07-24T04:23:05.8034661Z y 1024(0x400) 2025-07-24T04:23:05.8034925Z z 1024(0x400) 2025-07-24T04:23:05.8035354Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:23:05.8035639Z Grid Max Size per Dimension: 2025-07-24T04:23:05.8035889Z x 4294967295(0xffffffff) 2025-07-24T04:23:05.8036157Z y 4294967295(0xffffffff) 2025-07-24T04:23:05.8036427Z z 4294967295(0xffffffff) 2025-07-24T04:23:05.8036725Z FBarrier Max Size: 32 2025-07-24T04:23:05.8037009Z ******* 2025-07-24T04:23:05.8037215Z Agent 10 2025-07-24T04:23:05.8037406Z ******* 2025-07-24T04:23:05.8037632Z Name: gfx90a 2025-07-24T04:23:05.8037919Z Uuid: GPU-f5ee81c23f707083 2025-07-24T04:23:05.8038243Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:23:05.8038575Z Vendor Name: AMD 2025-07-24T04:23:05.8038897Z Feature: KERNEL_DISPATCH 2025-07-24T04:23:05.8039218Z Profile: BASE_PROFILE 2025-07-24T04:23:05.8039534Z Float Round Mode: NEAR 2025-07-24T04:23:05.8039913Z Max Queue Number: 128(0x80) 2025-07-24T04:23:05.8040227Z Queue Min Size: 64(0x40) 2025-07-24T04:23:05.8040543Z Queue Max Size: 131072(0x20000) 2025-07-24T04:23:05.8040847Z Queue Type: MULTI 2025-07-24T04:23:05.8041141Z Node: 9 2025-07-24T04:23:05.8041436Z Device Type: GPU 2025-07-24T04:23:05.8041712Z Cache Info: 2025-07-24T04:23:05.8041947Z L1: 16(0x10) KB 2025-07-24T04:23:05.8042219Z L2: 8192(0x2000) KB 2025-07-24T04:23:05.8042558Z Chip ID: 29708(0x740c) 2025-07-24T04:23:05.8042860Z ASIC Revision: 1(0x1) 2025-07-24T04:23:05.8043180Z Cacheline Size: 128(0x80) 2025-07-24T04:23:05.8043495Z Max Clock Freq. (MHz): 1700 2025-07-24T04:23:05.8044673Z BDFID: 37632 2025-07-24T04:23:05.8044981Z Internal Node ID: 9 2025-07-24T04:23:05.8045291Z Compute Unit: 104 2025-07-24T04:23:05.8045599Z SIMDs per CU: 4 2025-07-24T04:23:05.8045910Z Shader Engines: 8 2025-07-24T04:23:05.8046418Z Shader Arrs. per Eng.: 1 2025-07-24T04:23:05.8046760Z WatchPts on Addr. Ranges:4 2025-07-24T04:23:05.8047100Z Coherent Host Access: FALSE 2025-07-24T04:23:05.8047398Z Memory Properties: 2025-07-24T04:23:05.8047626Z Features: KERNEL_DISPATCH 2025-07-24T04:23:05.8047921Z Fast F16 Operation: TRUE 2025-07-24T04:23:05.8048245Z Wavefront Size: 64(0x40) 2025-07-24T04:23:05.8048572Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:23:05.8048869Z Workgroup Max Size per Dimension: 2025-07-24T04:23:05.8049119Z x 1024(0x400) 2025-07-24T04:23:05.8049384Z y 1024(0x400) 2025-07-24T04:23:05.8049639Z z 1024(0x400) 2025-07-24T04:23:05.8049940Z Max Waves Per CU: 32(0x20) 2025-07-24T04:23:05.8050425Z Max Work-item Per CU: 2048(0x800) 2025-07-24T04:23:05.8050748Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:23:05.8051030Z Grid Max Size per Dimension: 2025-07-24T04:23:05.8051266Z x 4294967295(0xffffffff) 2025-07-24T04:23:05.8051530Z y 4294967295(0xffffffff) 2025-07-24T04:23:05.8051805Z z 4294967295(0xffffffff) 2025-07-24T04:23:05.8052122Z Max fbarriers/Workgrp: 32 2025-07-24T04:23:05.8052471Z Packet Processor uCode:: 92 2025-07-24T04:23:05.8052816Z SDMA engine uCode:: 9 2025-07-24T04:23:05.8053137Z IOMMU Support:: None 2025-07-24T04:23:05.8053422Z Pool Info: 2025-07-24T04:23:05.8053633Z Pool 1 2025-07-24T04:23:05.8053908Z Segment: GLOBAL; FLAGS: COARSE GRAINED 2025-07-24T04:23:05.8054231Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.8054536Z Allocatable: TRUE 2025-07-24T04:23:05.8054869Z Alloc Granule: 4KB 2025-07-24T04:23:05.8055213Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.8055566Z Alloc Alignment: 4KB 2025-07-24T04:23:05.8055903Z Accessible by all: FALSE 2025-07-24T04:23:05.8056199Z Pool 2 2025-07-24T04:23:05.8056464Z Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED 2025-07-24T04:23:05.8056774Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.8057100Z Allocatable: TRUE 2025-07-24T04:23:05.8057426Z Alloc Granule: 4KB 2025-07-24T04:23:05.8057769Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.8058108Z Alloc Alignment: 4KB 2025-07-24T04:23:05.8058443Z Accessible by all: FALSE 2025-07-24T04:23:05.8058738Z Pool 3 2025-07-24T04:23:05.8058990Z Segment: GLOBAL; FLAGS: FINE GRAINED 2025-07-24T04:23:05.8059294Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:23:05.8059591Z Allocatable: TRUE 2025-07-24T04:23:05.8059917Z Alloc Granule: 4KB 2025-07-24T04:23:05.8060250Z Alloc Recommended Granule:2048KB 2025-07-24T04:23:05.8060723Z Alloc Alignment: 4KB 2025-07-24T04:23:05.8061071Z Accessible by all: FALSE 2025-07-24T04:23:05.8061362Z Pool 4 2025-07-24T04:23:05.8061614Z Segment: GROUP 2025-07-24T04:23:05.8061904Z Size: 64(0x40) KB 2025-07-24T04:23:05.8062210Z Allocatable: FALSE 2025-07-24T04:23:05.8062532Z Alloc Granule: 0KB 2025-07-24T04:23:05.8062870Z Alloc Recommended Granule:0KB 2025-07-24T04:23:05.8063214Z Alloc Alignment: 0KB 2025-07-24T04:23:05.8063540Z Accessible by all: FALSE 2025-07-24T04:23:05.8063832Z ISA Info: 2025-07-24T04:23:05.8064032Z ISA 1 2025-07-24T04:23:05.8064304Z Name: amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack- 2025-07-24T04:23:05.8064796Z Machine Models: HSA_MACHINE_MODEL_LARGE 2025-07-24T04:23:05.8065135Z Profiles: HSA_PROFILE_BASE 2025-07-24T04:23:05.8065479Z Default Rounding Mode: NEAR 2025-07-24T04:23:05.8065824Z Default Rounding Mode: NEAR 2025-07-24T04:23:05.8066141Z Fast f16: TRUE 2025-07-24T04:23:05.8066449Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:23:05.8066753Z Workgroup Max Size per Dimension: 2025-07-24T04:23:05.8067014Z x 1024(0x400) 2025-07-24T04:23:05.8067288Z y 1024(0x400) 2025-07-24T04:23:05.8067546Z z 1024(0x400) 2025-07-24T04:23:05.8067847Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:23:05.8068149Z Grid Max Size per Dimension: 2025-07-24T04:23:05.8068389Z x 4294967295(0xffffffff) 2025-07-24T04:23:05.8068661Z y 4294967295(0xffffffff) 2025-07-24T04:23:05.8068922Z z 4294967295(0xffffffff) 2025-07-24T04:23:05.8069228Z FBarrier Max Size: 32 2025-07-24T04:23:05.8069505Z *** Done *** 2025-07-24T04:23:05.8088490Z ##[group]Run ngpu=$(rocminfo | grep -c -E 'Name:.*\sgfx') 2025-07-24T04:23:05.8088881Z ngpu=$(rocminfo | grep -c -E 'Name:.*\sgfx') 2025-07-24T04:23:05.8089507Z msg="Please file an issue on pytorch/pytorch reporting the faulty runner. Include a link to the runner logs so the runner can be identified" 2025-07-24T04:23:05.8090091Z if [[ $ngpu -eq 0 ]]; then 2025-07-24T04:23:05.8090401Z  echo "Error: Failed to detect any GPUs on the runner" 2025-07-24T04:23:05.8090723Z  echo "$msg" 2025-07-24T04:23:05.8090931Z  exit 1 2025-07-24T04:23:05.8091122Z fi 2025-07-24T04:23:05.8091303Z if [[ $ngpu -eq 1 ]]; then 2025-07-24T04:23:05.8091681Z  echo "Error: only 1 GPU detected, at least 2 GPUs are needed for distributed jobs" 2025-07-24T04:23:05.8092058Z  echo "$msg" 2025-07-24T04:23:05.8092258Z  exit 1 2025-07-24T04:23:05.8092440Z fi 2025-07-24T04:23:05.8118366Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:05.8118702Z env: 2025-07-24T04:23:05.8118882Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:05.8119100Z ##[endgroup] 2025-07-24T04:23:05.9705431Z ##[group]Run pytorch/pytorch/.github/actions/diskspace-cleanup@main 2025-07-24T04:23:05.9705784Z with: 2025-07-24T04:23:05.9705975Z diskspace-cutoff: 70 2025-07-24T04:23:05.9706170Z env: 2025-07-24T04:23:05.9706597Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:05.9706812Z ##[endgroup] 2025-07-24T04:23:05.9741551Z ##[group]Run set -ex 2025-07-24T04:23:05.9741808Z set -ex 2025-07-24T04:23:05.9742005Z diskspace_cutoff=70 2025-07-24T04:23:05.9742320Z docker_root_dir=$(docker info -f '{{.DockerRootDir}}') 2025-07-24T04:23:05.9742662Z if [ ! -d "$docker_root_dir" ]; then 2025-07-24T04:23:05.9743114Z  echo "Docker root directory ($docker_root_dir) does not exist. Skipping disk space check." 2025-07-24T04:23:05.9743523Z  exit 0 2025-07-24T04:23:05.9743711Z fi 2025-07-24T04:23:05.9744069Z diskspace=$(df -H --output=pcent ${docker_root_dir} | sed -n 2p | sed 's/%//' | sed 's/ //') 2025-07-24T04:23:05.9744835Z msg="Please file an issue on pytorch/pytorch reporting the faulty runner. Include a link to the runner logs so the runner can be identified" 2025-07-24T04:23:05.9745489Z if [[ "$diskspace" -ge "$diskspace_cutoff" ]] ; then 2025-07-24T04:23:05.9745802Z  docker system prune -af 2025-07-24T04:23:05.9746460Z  diskspace_new=$(df -H --output=pcent ${docker_root_dir} | sed -n 2p | sed 's/%//' | sed 's/ //') 2025-07-24T04:23:05.9746943Z  if [[ "$diskspace_new" -gt "$diskspace_cutoff" ]] ; then 2025-07-24T04:23:05.9747444Z  echo "Error: Available diskspace is less than $diskspace_cutoff percent. Not enough diskspace." 2025-07-24T04:23:05.9747886Z  echo "$msg" 2025-07-24T04:23:05.9748098Z  exit 1 2025-07-24T04:23:05.9748300Z  else 2025-07-24T04:23:05.9748532Z  difference=$((diskspace - diskspace_new)) 2025-07-24T04:23:05.9748874Z  echo "Diskspace saved: $difference percent" 2025-07-24T04:23:05.9749153Z  fi 2025-07-24T04:23:05.9749330Z fi 2025-07-24T04:23:05.9778236Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:05.9778563Z env: 2025-07-24T04:23:05.9778769Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:05.9778999Z ##[endgroup] 2025-07-24T04:23:05.9825992Z + diskspace_cutoff=70 2025-07-24T04:23:05.9831725Z ++ docker info -f '{{.DockerRootDir}}' 2025-07-24T04:23:06.0376847Z + docker_root_dir=/media/4TB/docker-rootless 2025-07-24T04:23:06.0377243Z + '[' '!' -d /media/4TB/docker-rootless ']' 2025-07-24T04:23:06.0387684Z ++ df -H --output=pcent /media/4TB/docker-rootless 2025-07-24T04:23:06.0389318Z ++ sed -n 2p 2025-07-24T04:23:06.0389973Z ++ sed s/%// 2025-07-24T04:23:06.0394595Z ++ sed 's/ //' 2025-07-24T04:23:06.0419648Z + diskspace=65 2025-07-24T04:23:06.0420406Z + msg='Please file an issue on pytorch/pytorch reporting the faulty runner. Include a link to the runner logs so the runner can be identified' 2025-07-24T04:23:06.0421179Z + [[ 65 -ge 70 ]] 2025-07-24T04:23:06.0471355Z ##[group]Run RUNNER_ARTIFACT_DIR="${RUNNER_TEMP}/artifacts" 2025-07-24T04:23:06.0471784Z RUNNER_ARTIFACT_DIR="${RUNNER_TEMP}/artifacts" 2025-07-24T04:23:06.0472119Z rm -rf "${RUNNER_ARTIFACT_DIR}" 2025-07-24T04:23:06.0472413Z mkdir -p "${RUNNER_ARTIFACT_DIR}" 2025-07-24T04:23:06.0472794Z echo "RUNNER_ARTIFACT_DIR=${RUNNER_ARTIFACT_DIR}" >> "${GITHUB_ENV}" 2025-07-24T04:23:06.0473155Z  2025-07-24T04:23:06.0473403Z RUNNER_TEST_RESULTS_DIR="${RUNNER_TEMP}/test-results" 2025-07-24T04:23:06.0473760Z rm -rf "${RUNNER_TEST_RESULTS_DIR}" 2025-07-24T04:23:06.0474044Z mkdir -p "${RUNNER_TEST_RESULTS_DIR}" 2025-07-24T04:23:06.0474443Z echo "RUNNER_TEST_RESULTS_DIR=${RUNNER_TEST_RESULTS_DIR}" >> "${GITHUB_ENV}" 2025-07-24T04:23:06.0474808Z  2025-07-24T04:23:06.0475015Z RUNNER_DOCS_DIR="${RUNNER_TEMP}/docs" 2025-07-24T04:23:06.0475294Z rm -rf "${RUNNER_DOCS_DIR}" 2025-07-24T04:23:06.0475546Z mkdir -p "${RUNNER_DOCS_DIR}" 2025-07-24T04:23:06.0475877Z echo "RUNNER_DOCS_DIR=${RUNNER_DOCS_DIR}" >> "${GITHUB_ENV}" 2025-07-24T04:23:06.0500159Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:06.0500511Z env: 2025-07-24T04:23:06.0500702Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:06.0500918Z ##[endgroup] 2025-07-24T04:23:06.0695476Z ##[group]Run env | grep '^GITHUB' >> "${RUNNER_TEMP}/github_env_${GITHUB_RUN_ID}" 2025-07-24T04:23:06.0695982Z env | grep '^GITHUB' >> "${RUNNER_TEMP}/github_env_${GITHUB_RUN_ID}" 2025-07-24T04:23:06.0696398Z env | grep '^CI' >> "${RUNNER_TEMP}/github_env_${GITHUB_RUN_ID}" 2025-07-24T04:23:06.0724987Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:06.0725329Z env: 2025-07-24T04:23:06.0725520Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:06.0725893Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:06.0726432Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:06.0726921Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:06.0727290Z ##[endgroup] 2025-07-24T04:23:06.0853601Z ##[group]Run # All GPUs are visible to the runner; visibility, if needed, will be set by run_test.py. 2025-07-24T04:23:06.0854267Z # All GPUs are visible to the runner; visibility, if needed, will be set by run_test.py. 2025-07-24T04:23:06.0854730Z # Add render group for container creation. 2025-07-24T04:23:06.0855115Z render_gid=`cat /etc/group | grep render | cut -d: -f3` 2025-07-24T04:23:06.0855583Z # Ensure GPU isolation if pod is part of kubernetes setup with DEVICE_FLAG. 2025-07-24T04:23:06.0856065Z if [ -f "/etc/podinfo/gha-render-devices" ]; then 2025-07-24T04:23:06.0856472Z  DEVICE_FLAG=$(cat /etc/podinfo/gha-render-devices) 2025-07-24T04:23:06.0856804Z else 2025-07-24T04:23:06.0857017Z  DEVICE_FLAG="--device /dev/dri" 2025-07-24T04:23:06.0857273Z fi 2025-07-24T04:23:06.0857709Z # The --group-add daemon and --group-add bin are needed in the Ubuntu 24.04 and Almalinux OSs respectively. 2025-07-24T04:23:06.0858389Z # This is due to the device files (/dev/kfd & /dev/dri) being owned by video group on bare metal. 2025-07-24T04:23:06.0859004Z # This video group ID maps to subgid 1 inside the docker image due to the /etc/subgid entries. 2025-07-24T04:23:06.0859645Z # The group name corresponding to group ID 1 can change depending on the OS, so both are necessary. 2025-07-24T04:23:06.0860737Z echo "GPU_FLAG=--device=/dev/mem --device=/dev/kfd $DEVICE_FLAG --group-add video --group-add $render_gid --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host" >> "${GITHUB_ENV}" 2025-07-24T04:23:06.0885256Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:06.0885579Z env: 2025-07-24T04:23:06.0885760Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:06.0886130Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:06.0886664Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:06.0887175Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:06.0887511Z ##[endgroup] 2025-07-24T04:23:06.1029383Z ##[group]Run aws-actions/configure-aws-credentials@ececac1a45f3b08a01d2dd070d28d111c5fe6722 2025-07-24T04:23:06.1029856Z with: 2025-07-24T04:23:06.1030193Z role-to-assume: arn:aws:iam::308535385114:role/gha_workflow_s3_and_ecr_read_only 2025-07-24T04:23:06.1030585Z aws-region: us-east-1 2025-07-24T04:23:06.1030830Z role-duration-seconds: 18000 2025-07-24T04:23:06.1031085Z audience: sts.amazonaws.com 2025-07-24T04:23:06.1031318Z env: 2025-07-24T04:23:06.1031495Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:06.1031877Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:06.1032418Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:06.1032898Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:06.1033797Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:06.1034517Z ##[endgroup] 2025-07-24T04:23:06.3898538Z Assuming role with OIDC 2025-07-24T04:23:06.5493231Z Authenticated as assumedRoleId AROAUPVRELQNLLCOPFEJR:GitHubActions 2025-07-24T04:23:06.6076445Z ##[group]Run aws-actions/amazon-ecr-login@062b18b96a7aff071d4dc91bc00c4c1a7945b076 2025-07-24T04:23:06.6076912Z with: 2025-07-24T04:23:06.6077134Z mask-password: true 2025-07-24T04:23:06.6077379Z registry-type: private 2025-07-24T04:23:06.6077629Z skip-logout: false 2025-07-24T04:23:06.6077848Z env: 2025-07-24T04:23:06.6078048Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:06.6078430Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:06.6078987Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:06.6079779Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:06.6080623Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:06.6081384Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:06.6081701Z AWS_REGION: us-east-1 2025-07-24T04:23:06.6082640Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:06.6083019Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:06.6088088Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:06.6088328Z ##[endgroup] 2025-07-24T04:23:06.9429015Z Logging into registry 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-07-24T04:23:07.2351146Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2025-07-24T04:23:07.2351594Z with: 2025-07-24T04:23:07.2352246Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f 2025-07-24T04:23:07.2352993Z use-custom-docker-registry: true 2025-07-24T04:23:07.2353290Z docker-build-dir: .ci/docker 2025-07-24T04:23:07.2353566Z docker-build-script: ./build.sh 2025-07-24T04:23:07.2353823Z working-directory: . 2025-07-24T04:23:07.2354136Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-07-24T04:23:07.2354491Z force-push: false 2025-07-24T04:23:07.2354702Z env: 2025-07-24T04:23:07.2354904Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:07.2355276Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:07.2355818Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:07.2356379Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:07.2357232Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:07.2357999Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:07.2358259Z AWS_REGION: us-east-1 2025-07-24T04:23:07.2358613Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:07.2359003Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:07.2364170Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:07.2364412Z ##[endgroup] 2025-07-24T04:23:07.2383730Z ##[group]Run set -ex 2025-07-24T04:23:07.2384010Z set -ex 2025-07-24T04:23:07.2384217Z  2025-07-24T04:23:07.2384581Z # If the docker build directory or the build script doesn't exist, the action will 2025-07-24T04:23:07.2385152Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2025-07-24T04:23:07.2385662Z # job could then download the pre-built image as usual 2025-07-24T04:23:07.2386256Z if [[ -d "${DOCKER_BUILD_DIR}" ]] && [[ -f "${DOCKER_BUILD_DIR}/${DOCKER_BUILD_SCRIPT}" ]] && [[ "${USE_CUSTOM_DOCKER_REGISTRY}" == "true" ]]; then 2025-07-24T04:23:07.2386817Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2025-07-24T04:23:07.2387119Z else 2025-07-24T04:23:07.2387364Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2025-07-24T04:23:07.2387751Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2025-07-24T04:23:07.2388110Z  2025-07-24T04:23:07.2388592Z  echo "Not using custom ECR registry. Either it was not requested or there is no Docker build script in the ${REPO_NAME} repo..." 2025-07-24T04:23:07.2389132Z  exit 0 2025-07-24T04:23:07.2389331Z fi 2025-07-24T04:23:07.2389527Z  2025-07-24T04:23:07.2389832Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2025-07-24T04:23:07.2390354Z  # The docker image name already includes the ECR prefix and tag, so we can just 2025-07-24T04:23:07.2390814Z  # use it as it is, but first let's extract the tag 2025-07-24T04:23:07.2391472Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2025-07-24T04:23:07.2391917Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2025-07-24T04:23:07.2392341Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2025-07-24T04:23:07.2392699Z else 2025-07-24T04:23:07.2392950Z  if [[ "${DOCKER_IMAGE_NAME}" == *:* ]]; then 2025-07-24T04:23:07.2393297Z  CUSTOM_TAG_PREFIX=${DOCKER_IMAGE_NAME#*:} 2025-07-24T04:23:07.2393637Z  DOCKER_IMAGE_NAME=${DOCKER_IMAGE_NAME%%:*} 2025-07-24T04:23:07.2393933Z  fi 2025-07-24T04:23:07.2394513Z  DOCKER_TAG=${CUSTOM_TAG_PREFIX:+${CUSTOM_TAG_PREFIX}-}$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2025-07-24T04:23:07.2395049Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2025-07-24T04:23:07.2395591Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2025-07-24T04:23:07.2396191Z  echo "custom-tag-prefix=${CUSTOM_TAG_PREFIX}" >> "${GITHUB_OUTPUT}" 2025-07-24T04:23:07.2396566Z fi 2025-07-24T04:23:07.2424596Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:07.2424960Z env: 2025-07-24T04:23:07.2425175Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:07.2425571Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:07.2426131Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:07.2426656Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:07.2427514Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:07.2428298Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:07.2428574Z AWS_REGION: us-east-1 2025-07-24T04:23:07.2428875Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:07.2429218Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:07.2434268Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:07.2434512Z REPO_NAME: pytorch 2025-07-24T04:23:07.2435160Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f 2025-07-24T04:23:07.2435849Z DOCKER_BUILD_DIR: .ci/docker 2025-07-24T04:23:07.2436120Z DOCKER_BUILD_SCRIPT: ./build.sh 2025-07-24T04:23:07.2436460Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-07-24T04:23:07.2436812Z USE_CUSTOM_DOCKER_REGISTRY: true 2025-07-24T04:23:07.2437082Z CUSTOM_TAG_PREFIX: 2025-07-24T04:23:07.2437322Z ##[endgroup] 2025-07-24T04:23:07.2487541Z + [[ -d .ci/docker ]] 2025-07-24T04:23:07.2487856Z + [[ -f .ci/docker/./build.sh ]] 2025-07-24T04:23:07.2488143Z + [[ true == \t\r\u\e ]] 2025-07-24T04:23:07.2488435Z + echo skip=false 2025-07-24T04:23:07.2489431Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2025-07-24T04:23:07.2499319Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f 2025-07-24T04:23:07.2501120Z ++ awk -F '[:,]' '{print $2}' 2025-07-24T04:23:07.2532726Z + DOCKER_TAG=pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f 2025-07-24T04:23:07.2533870Z + echo docker-tag=pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f 2025-07-24T04:23:07.2535443Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f 2025-07-24T04:23:07.2595284Z ##[group]Run set +e 2025-07-24T04:23:07.2595818Z set +e 2025-07-24T04:23:07.2596014Z set -x 2025-07-24T04:23:07.2596212Z  2025-07-24T04:23:07.2596417Z login() { 2025-07-24T04:23:07.2596852Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2025-07-24T04:23:07.2597312Z } 2025-07-24T04:23:07.2597510Z  2025-07-24T04:23:07.2597702Z retry () { 2025-07-24T04:23:07.2597950Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2025-07-24T04:23:07.2598229Z } 2025-07-24T04:23:07.2598419Z  2025-07-24T04:23:07.2598629Z retry login "${DOCKER_REGISTRY}" 2025-07-24T04:23:07.2598901Z  2025-07-24T04:23:07.2599108Z START_TIME=$(date +%s) 2025-07-24T04:23:07.2599592Z # Wait up to 120 minutes 2025-07-24T04:23:07.2599935Z while [[ $(( $(date +%s) - 7200 )) -lt $START_TIME ]]; do 2025-07-24T04:23:07.2600358Z  # Check if image already exists, if it does then skip building it 2025-07-24T04:23:07.2600806Z  if docker manifest inspect "${DOCKER_IMAGE}"; then 2025-07-24T04:23:07.2601129Z  exit 0 2025-07-24T04:23:07.2601351Z  fi 2025-07-24T04:23:07.2601558Z  2025-07-24T04:23:07.2601893Z  # NB: This flag is used by Docker build workflow to push the image to ECR, so we can 2025-07-24T04:23:07.2602531Z  # use this to differentiate between the Docker build and regular build jobs. For the 2025-07-24T04:23:07.2603079Z  # latter, it will wait for the Docker images to become available before continuing 2025-07-24T04:23:07.2603523Z  if [ "${DOCKER_PUSH:-false}" == "true" ]; then 2025-07-24T04:23:07.2603870Z  # It's a Docker build job, let's build the image 2025-07-24T04:23:07.2604185Z  break 2025-07-24T04:23:07.2604395Z  else 2025-07-24T04:23:07.2604691Z  # It's a regular build job, wait for the image to become available 2025-07-24T04:23:07.2605048Z  sleep 300 2025-07-24T04:23:07.2605256Z  fi 2025-07-24T04:23:07.2605451Z done 2025-07-24T04:23:07.2605642Z  2025-07-24T04:23:07.2605949Z # NB: This part requires a full checkout. Otherwise, the merge base will 2025-07-24T04:23:07.2606433Z # be empty. The default action would be to continue rebuild the image 2025-07-24T04:23:07.2606888Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2025-07-24T04:23:07.2607278Z  # if we're on the base branch then use the parent commit 2025-07-24T04:23:07.2607618Z  MERGE_BASE=$(git rev-parse HEAD~) 2025-07-24T04:23:07.2607894Z else 2025-07-24T04:23:07.2608185Z  # otherwise we're on a PR, so use the most recent base commit 2025-07-24T04:23:07.2608592Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2025-07-24T04:23:07.2608897Z fi 2025-07-24T04:23:07.2609094Z  2025-07-24T04:23:07.2609316Z if [[ -z "${MERGE_BASE}" ]]; then 2025-07-24T04:23:07.2609620Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2025-07-24T04:23:07.2609913Z  2025-07-24T04:23:07.2610313Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2025-07-24T04:23:07.2610783Z  exit 0 2025-07-24T04:23:07.2610988Z fi 2025-07-24T04:23:07.2611187Z  2025-07-24T04:23:07.2611460Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2025-07-24T04:23:07.2612038Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2025-07-24T04:23:07.2612537Z  exit 1 2025-07-24T04:23:07.2612741Z fi 2025-07-24T04:23:07.2612945Z  2025-07-24T04:23:07.2613267Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2025-07-24T04:23:07.2613820Z # If no image exists but the hash is the same as the previous hash then we should error out here 2025-07-24T04:23:07.2614496Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2025-07-24T04:23:07.2615065Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2025-07-24T04:23:07.2615723Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2025-07-24T04:23:07.2616109Z fi 2025-07-24T04:23:07.2616307Z  2025-07-24T04:23:07.2616546Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2025-07-24T04:23:07.2644558Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:07.2644914Z env: 2025-07-24T04:23:07.2645120Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:07.2645677Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:07.2646223Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:07.2646740Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:07.2647582Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:07.2648333Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:07.2648591Z AWS_REGION: us-east-1 2025-07-24T04:23:07.2648892Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:07.2649255Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:07.2654289Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:07.2654541Z DOCKER_BUILD_DIR: .ci/docker 2025-07-24T04:23:07.2654839Z BASE_REVISION: 27c8ef1839e0747b36769efb192befc9308ff93c 2025-07-24T04:23:07.2655574Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f 2025-07-24T04:23:07.2656411Z DOCKER_TAG: pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f 2025-07-24T04:23:07.2656943Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-07-24T04:23:07.2657286Z DOCKER_PUSH: 2025-07-24T04:23:07.2657498Z ##[endgroup] 2025-07-24T04:23:07.2707790Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-07-24T04:23:07.2708200Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-07-24T04:23:07.2713003Z + aws ecr get-login-password --region us-east-1 2025-07-24T04:23:07.2715993Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-07-24T04:23:08.2490274Z WARNING! Your password will be stored unencrypted in /var/home/pytorchci/.docker/config.json. 2025-07-24T04:23:08.2491208Z Configure a credential helper to remove this warning. See 2025-07-24T04:23:08.2492096Z https://docs.docker.com/engine/reference/commandline/login/#credential-stores 2025-07-24T04:23:08.2492700Z 2025-07-24T04:23:08.2493978Z Login Succeeded 2025-07-24T04:23:08.2531662Z ++ date +%s 2025-07-24T04:23:08.2545016Z + START_TIME=1753330988 2025-07-24T04:23:08.2551832Z ++ date +%s 2025-07-24T04:23:08.2566688Z + [[ 1753323788 -lt 1753330988 ]] 2025-07-24T04:23:08.2567406Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f 2025-07-24T04:23:08.9034437Z { 2025-07-24T04:23:08.9034870Z "schemaVersion": 2, 2025-07-24T04:23:08.9035527Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2025-07-24T04:23:08.9036310Z "config": { 2025-07-24T04:23:08.9036923Z "mediaType": "application/vnd.docker.container.image.v1+json", 2025-07-24T04:23:08.9037740Z "size": 28426, 2025-07-24T04:23:08.9038456Z "digest": "sha256:441c23f68601012989b37ca64859d010f9dc3cbc8ea65694e4ace6ed235631cc" 2025-07-24T04:23:08.9039328Z }, 2025-07-24T04:23:08.9039678Z "layers": [ 2025-07-24T04:23:08.9040018Z { 2025-07-24T04:23:08.9040524Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9041638Z "size": 30446997, 2025-07-24T04:23:08.9042241Z "digest": "sha256:66587c81b81a58d07e40c48d900a1517516bbf58c4378c687d89d645824f5e5f" 2025-07-24T04:23:08.9043344Z }, 2025-07-24T04:23:08.9043631Z { 2025-07-24T04:23:08.9044122Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9044745Z "size": 1554, 2025-07-24T04:23:08.9045326Z "digest": "sha256:c467531473a7140e2846e8648a245301c77a465d0561b48e8dd31b30e191e523" 2025-07-24T04:23:08.9046055Z }, 2025-07-24T04:23:08.9046381Z { 2025-07-24T04:23:08.9046763Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9047172Z "size": 313286081, 2025-07-24T04:23:08.9047587Z "digest": "sha256:6edafb9a3aa208d2ac734c6205f560f7b7dcdec1a7b832bc7894e1278787f2b1" 2025-07-24T04:23:08.9048283Z }, 2025-07-24T04:23:08.9048484Z { 2025-07-24T04:23:08.9048788Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9049196Z "size": 704, 2025-07-24T04:23:08.9049697Z "digest": "sha256:67bbdb9be01f9e05df0be4996ff180f6ee99763b4e8bed028f22e8c5c687f1f3" 2025-07-24T04:23:08.9050165Z }, 2025-07-24T04:23:08.9050364Z { 2025-07-24T04:23:08.9050673Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9051091Z "size": 1214, 2025-07-24T04:23:08.9051482Z "digest": "sha256:d30866a5fbe67a4d3ed6411fd4d9040a5715bdb1f83db530ab44c7275bf0cd12" 2025-07-24T04:23:08.9051942Z }, 2025-07-24T04:23:08.9052134Z { 2025-07-24T04:23:08.9052443Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9052831Z "size": 485, 2025-07-24T04:23:08.9053213Z "digest": "sha256:ef2b71d5d5187f9bc8eb30b15d328bc65465f4b4b4c67061cfd8027aba8f72dd" 2025-07-24T04:23:08.9053679Z }, 2025-07-24T04:23:08.9053861Z { 2025-07-24T04:23:08.9054163Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9054562Z "size": 110343867, 2025-07-24T04:23:08.9054961Z "digest": "sha256:3a79f1be9328db38db5d9682fef0e9659b2d24e666d5ef2f39f2a233084e548c" 2025-07-24T04:23:08.9055423Z }, 2025-07-24T04:23:08.9055601Z { 2025-07-24T04:23:08.9055898Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9056305Z "size": 4164, 2025-07-24T04:23:08.9056682Z "digest": "sha256:ab67e3b4c18a18c51c22703a2493006d45815828baf0b1df64b12da7ff0bd2e7" 2025-07-24T04:23:08.9057081Z }, 2025-07-24T04:23:08.9057243Z { 2025-07-24T04:23:08.9057519Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9057860Z "size": 1710, 2025-07-24T04:23:08.9058200Z "digest": "sha256:035eb636c57bd7202decf2a456e8537bddc1be808adbd80f6dd75e12c13e1be3" 2025-07-24T04:23:08.9058618Z }, 2025-07-24T04:23:08.9058788Z { 2025-07-24T04:23:08.9059061Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9059409Z "size": 724, 2025-07-24T04:23:08.9059746Z "digest": "sha256:d4ec37220e3a61f253b7bff3b60dce616d51caf1327ea11d322d85cd281dbf07" 2025-07-24T04:23:08.9060146Z }, 2025-07-24T04:23:08.9060313Z { 2025-07-24T04:23:08.9060588Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9060935Z "size": 3305033147, 2025-07-24T04:23:08.9061278Z "digest": "sha256:9d9b3bb5166b5733af7605b65690330a088bd8636c36433e8e173a6659b16421" 2025-07-24T04:23:08.9061672Z }, 2025-07-24T04:23:08.9061840Z { 2025-07-24T04:23:08.9062111Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9062460Z "size": 381, 2025-07-24T04:23:08.9062789Z "digest": "sha256:554a1868b180d4e2032cc390ea00a9f7c180d8885392be12550bc3e1c5e1a23c" 2025-07-24T04:23:08.9063194Z }, 2025-07-24T04:23:08.9063348Z { 2025-07-24T04:23:08.9063625Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9063976Z "size": 233567, 2025-07-24T04:23:08.9064323Z "digest": "sha256:090ade5d5ee9d68cd421dbc7cb6e401709f6c178a61d3f87019b041c6774b2db" 2025-07-24T04:23:08.9064896Z }, 2025-07-24T04:23:08.9065060Z { 2025-07-24T04:23:08.9065327Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9065676Z "size": 792, 2025-07-24T04:23:08.9066004Z "digest": "sha256:a998e898d9939b4c1d30567427b67cc709d6852ee65ef64fa2d9c318db592126" 2025-07-24T04:23:08.9066406Z }, 2025-07-24T04:23:08.9066565Z { 2025-07-24T04:23:08.9066836Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9067181Z "size": 106, 2025-07-24T04:23:08.9067529Z "digest": "sha256:cfaabb19fb471ea7d79bd4c76b041e6a316873783814724e35c08c71bf89cd2c" 2025-07-24T04:23:08.9067932Z }, 2025-07-24T04:23:08.9068096Z { 2025-07-24T04:23:08.9068363Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9068829Z "size": 1495, 2025-07-24T04:23:08.9069177Z "digest": "sha256:53b92662290aaff40a4779c14e8bf37ca7e400c814eadabbdeea922a157f26f0" 2025-07-24T04:23:08.9069587Z }, 2025-07-24T04:23:08.9069757Z { 2025-07-24T04:23:08.9070017Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9070369Z "size": 455235035, 2025-07-24T04:23:08.9070717Z "digest": "sha256:95b6332c3b55e9d6fe681de21373e3993ddc9b286515039dc2f7ed9914277167" 2025-07-24T04:23:08.9071113Z }, 2025-07-24T04:23:08.9071280Z { 2025-07-24T04:23:08.9071550Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9071899Z "size": 163, 2025-07-24T04:23:08.9072248Z "digest": "sha256:b67085f0f4fdd797574ea5df363a0cbde9ca856b230bbdd7aae29b90ad5e79cb" 2025-07-24T04:23:08.9072657Z }, 2025-07-24T04:23:08.9072822Z { 2025-07-24T04:23:08.9073100Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9073440Z "size": 2438, 2025-07-24T04:23:08.9073783Z "digest": "sha256:5e2e975cce85260c1165f6d3d8280f9d678fed13e21fc8b6efba9f1a2bca97bc" 2025-07-24T04:23:08.9074191Z }, 2025-07-24T04:23:08.9074360Z { 2025-07-24T04:23:08.9074641Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9074992Z "size": 7164124491, 2025-07-24T04:23:08.9075347Z "digest": "sha256:90308017eb7efa2434edc95a07bc92d8710942262bf3d76dda289feca59fb5f5" 2025-07-24T04:23:08.9075752Z }, 2025-07-24T04:23:08.9075919Z { 2025-07-24T04:23:08.9076192Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9076548Z "size": 105, 2025-07-24T04:23:08.9076881Z "digest": "sha256:7f8c898d8b8417ffa2b0a975a6e9a7ccaee952d8e8acb7e90906c53f9f41a7a6" 2025-07-24T04:23:08.9077289Z }, 2025-07-24T04:23:08.9077455Z { 2025-07-24T04:23:08.9077717Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9078060Z "size": 612, 2025-07-24T04:23:08.9078392Z "digest": "sha256:5b1565991f5c0f1defeb2448eaac6383107d330f11fa60e3c9f8488752ee12e8" 2025-07-24T04:23:08.9078795Z }, 2025-07-24T04:23:08.9078951Z { 2025-07-24T04:23:08.9079220Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9079572Z "size": 677677324, 2025-07-24T04:23:08.9079928Z "digest": "sha256:cfa6c02ce350f691ca01e9bbc91554b3448765199582f80d0dc44b8b4d226f46" 2025-07-24T04:23:08.9080514Z }, 2025-07-24T04:23:08.9080826Z { 2025-07-24T04:23:08.9081114Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9081464Z "size": 111, 2025-07-24T04:23:08.9081813Z "digest": "sha256:4690cd0b661d36beb72956d1933777b5e2eb63ac53f3549cbe117619cbed11ab" 2025-07-24T04:23:08.9082221Z }, 2025-07-24T04:23:08.9082486Z { 2025-07-24T04:23:08.9082761Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9083116Z "size": 1557, 2025-07-24T04:23:08.9083478Z "digest": "sha256:8dab92a6cabfe1c1cc4cf921541571080c1db7eeaa5d6fb5e1dc421b7627954e" 2025-07-24T04:23:08.9083907Z }, 2025-07-24T04:23:08.9084079Z { 2025-07-24T04:23:08.9084347Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9084880Z "size": 107, 2025-07-24T04:23:08.9085236Z "digest": "sha256:741b9f0a45dbdd117063541fe5928c8656c711df99ce2f6c93fb837df6c54000" 2025-07-24T04:23:08.9085658Z }, 2025-07-24T04:23:08.9085836Z { 2025-07-24T04:23:08.9086125Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9086499Z "size": 166, 2025-07-24T04:23:08.9086851Z "digest": "sha256:78a6125437f3d51017ccb8735f1c3fce4ce00aa14f7490ba2d1f3a0254101595" 2025-07-24T04:23:08.9087261Z }, 2025-07-24T04:23:08.9087438Z { 2025-07-24T04:23:08.9087708Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9088087Z "size": 2801110, 2025-07-24T04:23:08.9088454Z "digest": "sha256:3be44072ab4e7975f53fa3962eb807664f771ebafad2dae86f07404fb56be803" 2025-07-24T04:23:08.9089008Z }, 2025-07-24T04:23:08.9089195Z { 2025-07-24T04:23:08.9089488Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9089868Z "size": 107, 2025-07-24T04:23:08.9090216Z "digest": "sha256:04044610df175f1bf00fb5ea1215ba9784924b5a20293aafe94a155cfc75d56d" 2025-07-24T04:23:08.9090622Z }, 2025-07-24T04:23:08.9090803Z { 2025-07-24T04:23:08.9091086Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9091435Z "size": 802, 2025-07-24T04:23:08.9091781Z "digest": "sha256:152aa5a5158e010217f23fd8fbb0d9b0826c39bb13e6d8652c6965cd56b5fc7d" 2025-07-24T04:23:08.9092190Z }, 2025-07-24T04:23:08.9092365Z { 2025-07-24T04:23:08.9092644Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9093001Z "size": 26113617, 2025-07-24T04:23:08.9093354Z "digest": "sha256:63f73895622ecab7d061d93953419fbdf7c48814c5008105fc0b432177c033a0" 2025-07-24T04:23:08.9093758Z }, 2025-07-24T04:23:08.9093931Z { 2025-07-24T04:23:08.9094208Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9094573Z "size": 104, 2025-07-24T04:23:08.9094918Z "digest": "sha256:5af6f2f093ea5798a44a5decddf46cb875cbf4b54e5e938d3ad6d5a0d3b61967" 2025-07-24T04:23:08.9095346Z }, 2025-07-24T04:23:08.9095519Z { 2025-07-24T04:23:08.9095795Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9096161Z "size": 425, 2025-07-24T04:23:08.9096515Z "digest": "sha256:fda40cf9ab8372afeefe16b77d0233d4b63b678cace2059714f7c35d772e1637" 2025-07-24T04:23:08.9096933Z }, 2025-07-24T04:23:08.9097107Z { 2025-07-24T04:23:08.9097385Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9097744Z "size": 19308731, 2025-07-24T04:23:08.9098113Z "digest": "sha256:5aae7747cc25bf67f6bfe370770b12dccc584ea9afb3f8b825b03b0cb15918b1" 2025-07-24T04:23:08.9098535Z }, 2025-07-24T04:23:08.9098710Z { 2025-07-24T04:23:08.9098999Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9099366Z "size": 425, 2025-07-24T04:23:08.9099716Z "digest": "sha256:5b62e7dff82033606f1b96c42c49540ebbe46772aeb12467df1e1bb07eb2e175" 2025-07-24T04:23:08.9100137Z }, 2025-07-24T04:23:08.9100317Z { 2025-07-24T04:23:08.9100597Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9100957Z "size": 724, 2025-07-24T04:23:08.9101308Z "digest": "sha256:d4ec37220e3a61f253b7bff3b60dce616d51caf1327ea11d322d85cd281dbf07" 2025-07-24T04:23:08.9101729Z }, 2025-07-24T04:23:08.9101895Z { 2025-07-24T04:23:08.9102182Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9102578Z "size": 141, 2025-07-24T04:23:08.9102938Z "digest": "sha256:02460b2a4f6a2d3bc23ff65db1afd3750ee9ae49a35ce9ddb8e74f3c46b5705f" 2025-07-24T04:23:08.9103362Z }, 2025-07-24T04:23:08.9103538Z { 2025-07-24T04:23:08.9103822Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9104189Z "size": 136, 2025-07-24T04:23:08.9104536Z "digest": "sha256:344aa86a61d68f5f8573f99122b4efe5c555bdceaeb3a1f1b5502715dbaee4c7" 2025-07-24T04:23:08.9105099Z }, 2025-07-24T04:23:08.9105267Z { 2025-07-24T04:23:08.9105559Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9105925Z "size": 5098992948, 2025-07-24T04:23:08.9106303Z "digest": "sha256:04b592ce3217c257cc0b889e15f3c3731d22baca68e5760ec146ef749d0ff1f1" 2025-07-24T04:23:08.9106713Z }, 2025-07-24T04:23:08.9106885Z { 2025-07-24T04:23:08.9107162Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9107526Z "size": 195, 2025-07-24T04:23:08.9107881Z "digest": "sha256:e3aedebec93030bc06d19aacab68750c6e085bb1c27dc358d16858848935cf71" 2025-07-24T04:23:08.9108308Z }, 2025-07-24T04:23:08.9108486Z { 2025-07-24T04:23:08.9108761Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9109253Z "size": 346, 2025-07-24T04:23:08.9109607Z "digest": "sha256:2b5211eebec64099919bea1197c9c43983976b5037fd1ff6236509f49dd4a979" 2025-07-24T04:23:08.9110014Z }, 2025-07-24T04:23:08.9110199Z { 2025-07-24T04:23:08.9110487Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9110849Z "size": 88303, 2025-07-24T04:23:08.9111203Z "digest": "sha256:88aac01ec1db72a960d19e691f39b315730a97c2330390a8e6d0cbd2f80d3ca0" 2025-07-24T04:23:08.9111619Z }, 2025-07-24T04:23:08.9111795Z { 2025-07-24T04:23:08.9112071Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9112416Z "size": 106, 2025-07-24T04:23:08.9112766Z "digest": "sha256:f154dd81b32fa92d4faa8040506993076e2912a919c2afe6dded50740d86d5be" 2025-07-24T04:23:08.9113177Z }, 2025-07-24T04:23:08.9113347Z { 2025-07-24T04:23:08.9113624Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9113993Z "size": 1666, 2025-07-24T04:23:08.9114343Z "digest": "sha256:089f2260c49023e792a2bee3867520ef852c8a55f37ce04ea81b07b7dfc2dae7" 2025-07-24T04:23:08.9114749Z }, 2025-07-24T04:23:08.9114922Z { 2025-07-24T04:23:08.9115213Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9115570Z "size": 724, 2025-07-24T04:23:08.9115909Z "digest": "sha256:d4ec37220e3a61f253b7bff3b60dce616d51caf1327ea11d322d85cd281dbf07" 2025-07-24T04:23:08.9116319Z }, 2025-07-24T04:23:08.9116502Z { 2025-07-24T04:23:08.9116777Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9117137Z "size": 137, 2025-07-24T04:23:08.9117487Z "digest": "sha256:ca8b54819584ab34ad98cd66be3fceb44c157dc0b7f140070c4584816919bbe5" 2025-07-24T04:23:08.9117909Z }, 2025-07-24T04:23:08.9118090Z { 2025-07-24T04:23:08.9118376Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9118739Z "size": 120, 2025-07-24T04:23:08.9119089Z "digest": "sha256:76870a4c3f8fd6d57b2ec17e72b7a85a29d885057ef18bf331afb97a748d6d40" 2025-07-24T04:23:08.9119491Z }, 2025-07-24T04:23:08.9119671Z { 2025-07-24T04:23:08.9119949Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9120322Z "size": 5356440017, 2025-07-24T04:23:08.9120697Z "digest": "sha256:f4a9f30fcdabff1a1167cb66c7560f21b3668899a96b35b9fc4fde046e4ef265" 2025-07-24T04:23:08.9121112Z }, 2025-07-24T04:23:08.9121286Z { 2025-07-24T04:23:08.9121561Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9121921Z "size": 175, 2025-07-24T04:23:08.9122267Z "digest": "sha256:110b5a00f043de5141a35fc11bdd3d6541ec027ba726178f89e3bd5b44f18d8b" 2025-07-24T04:23:08.9122747Z }, 2025-07-24T04:23:08.9122907Z { 2025-07-24T04:23:08.9123176Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9123534Z "size": 1897, 2025-07-24T04:23:08.9123885Z "digest": "sha256:83772e66e52fac49c784a3a4dee3cad95c1653ee093350b48399492da564b335" 2025-07-24T04:23:08.9124292Z }, 2025-07-24T04:23:08.9124461Z { 2025-07-24T04:23:08.9124732Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9125252Z "size": 197493594, 2025-07-24T04:23:08.9125616Z "digest": "sha256:eee6d9689c2d5b11deada1d789902ffa6c32c42bd82c1c99df6264532d0af8d8" 2025-07-24T04:23:08.9126026Z }, 2025-07-24T04:23:08.9126181Z { 2025-07-24T04:23:08.9126451Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9126815Z "size": 303, 2025-07-24T04:23:08.9127160Z "digest": "sha256:40f61293f85973eb3d6df45c40671ffd92439d1bc29f39c887e4342e5bb05461" 2025-07-24T04:23:08.9127566Z }, 2025-07-24T04:23:08.9127746Z { 2025-07-24T04:23:08.9128025Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9128386Z "size": 32, 2025-07-24T04:23:08.9128735Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-07-24T04:23:08.9129313Z }, 2025-07-24T04:23:08.9129484Z { 2025-07-24T04:23:08.9129753Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9130110Z "size": 108, 2025-07-24T04:23:08.9130466Z "digest": "sha256:dfc71386efac7d21b798410f8f2ead070171f7f687d570ebf35eed00ecaf48c8" 2025-07-24T04:23:08.9130884Z }, 2025-07-24T04:23:08.9131061Z { 2025-07-24T04:23:08.9131337Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-07-24T04:23:08.9131699Z "size": 54145699, 2025-07-24T04:23:08.9132057Z "digest": "sha256:f4813cdb60f2c95fd11144ffcc01f5908a02cabb3606fc1f84f9b6bdf44177e0" 2025-07-24T04:23:08.9132475Z } 2025-07-24T04:23:08.9132652Z ] 2025-07-24T04:23:08.9132830Z } 2025-07-24T04:23:08.9133015Z + exit 0 2025-07-24T04:23:08.9158796Z ##[group]Run set -eux 2025-07-24T04:23:08.9159048Z set -eux 2025-07-24T04:23:08.9159798Z aws secretsmanager get-secret-value --secret-id docker_hub_readonly_token | jq --raw-output '.SecretString' | jq -r .docker_hub_readonly_token | docker login --username pytorchbot --password-stdin 2025-07-24T04:23:08.9185606Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:08.9185964Z env: 2025-07-24T04:23:08.9186173Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:08.9186557Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:08.9187121Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:08.9187637Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:08.9188483Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:08.9189244Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:08.9189515Z AWS_REGION: us-east-1 2025-07-24T04:23:08.9189852Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:08.9190204Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:08.9195256Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:08.9195501Z ##[endgroup] 2025-07-24T04:23:08.9250552Z + aws secretsmanager get-secret-value --secret-id docker_hub_readonly_token 2025-07-24T04:23:08.9251507Z + jq --raw-output .SecretString 2025-07-24T04:23:08.9254245Z + jq -r .docker_hub_readonly_token 2025-07-24T04:23:08.9256017Z + docker login --username pytorchbot --password-stdin 2025-07-24T04:23:09.5519288Z 2025-07-24T04:23:09.5521715Z An error occurred (AccessDeniedException) when calling the GetSecretValue operation: User: arn:aws:sts::308535385114:assumed-role/gha_workflow_s3_and_ecr_read_only/GitHubActions is not authorized to perform: secretsmanager:GetSecretValue on resource: docker_hub_readonly_token because no identity-based policy allows the secretsmanager:GetSecretValue action 2025-07-24T04:23:09.6573722Z Error: Cannot perform an interactive login from a non TTY device 2025-07-24T04:23:09.6625544Z ##[error]Process completed with exit code 1. 2025-07-24T04:23:09.6730606Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2025-07-24T04:23:09.6731171Z with: 2025-07-24T04:23:09.6732002Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f 2025-07-24T04:23:09.6733174Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-07-24T04:23:09.6733606Z env: 2025-07-24T04:23:09.6734037Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:09.6734527Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:09.6735230Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:09.6735888Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:09.6737000Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:09.6737921Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:09.6738287Z AWS_REGION: us-east-1 2025-07-24T04:23:09.6738824Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:09.6763319Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:09.6768422Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:09.6768661Z ##[endgroup] 2025-07-24T04:23:09.6784693Z ##[group]Run set -x 2025-07-24T04:23:09.6784961Z set -x 2025-07-24T04:23:09.6785164Z set +e 2025-07-24T04:23:09.6785364Z  2025-07-24T04:23:09.6785561Z login() { 2025-07-24T04:23:09.6785976Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2025-07-24T04:23:09.6786436Z } 2025-07-24T04:23:09.6786637Z  2025-07-24T04:23:09.6786833Z retry () { 2025-07-24T04:23:09.6787078Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2025-07-24T04:23:09.6787363Z } 2025-07-24T04:23:09.6787556Z  2025-07-24T04:23:09.6787779Z retry login "${DOCKER_REGISTRY}" 2025-07-24T04:23:09.6788055Z  2025-07-24T04:23:09.6788482Z IMAGE_SIZE=$(docker manifest inspect "${DOCKER_IMAGE}" | jq '[.layers[].size, .config.size] | add / 1024 / 1024') 2025-07-24T04:23:09.6789049Z echo "Compressed size of image in MB: ${IMAGE_SIZE}" 2025-07-24T04:23:09.6789377Z  2025-07-24T04:23:09.6789566Z set -e 2025-07-24T04:23:09.6789867Z # ignore output since only exit code is used for conditional 2025-07-24T04:23:09.6790297Z # only pull docker image if it's not available locally 2025-07-24T04:23:09.6790772Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2025-07-24T04:23:09.6791208Z  retry docker pull "${DOCKER_IMAGE}" 2025-07-24T04:23:09.6791492Z fi 2025-07-24T04:23:09.6821100Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:09.6821446Z env: 2025-07-24T04:23:09.6821653Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:09.6822036Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:09.6822580Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:09.6823095Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:09.6823924Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:09.6824687Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:09.6824947Z AWS_REGION: us-east-1 2025-07-24T04:23:09.6825230Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:09.6825576Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:09.6830599Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:09.6831465Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f 2025-07-24T04:23:09.6832216Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-07-24T04:23:09.6832561Z ##[endgroup] 2025-07-24T04:23:09.6882147Z + set +e 2025-07-24T04:23:09.6882771Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-07-24T04:23:09.6883168Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-07-24T04:23:09.6888555Z + aws ecr get-login-password --region us-east-1 2025-07-24T04:23:09.6890777Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-07-24T04:23:10.7047898Z WARNING! Your password will be stored unencrypted in /var/home/pytorchci/.docker/config.json. 2025-07-24T04:23:10.7048899Z Configure a credential helper to remove this warning. See 2025-07-24T04:23:10.7049781Z https://docs.docker.com/engine/reference/commandline/login/#credential-stores 2025-07-24T04:23:10.7050484Z 2025-07-24T04:23:10.7057352Z Login Succeeded 2025-07-24T04:23:10.7104557Z ++ docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f 2025-07-24T04:23:10.7105956Z ++ jq '[.layers[].size, .config.size] | add / 1024 / 1024' 2025-07-24T04:23:11.3763803Z + IMAGE_SIZE=21755.0470495224 2025-07-24T04:23:11.3764278Z + echo 'Compressed size of image in MB: 21755.0470495224' 2025-07-24T04:23:11.3764687Z + set -e 2025-07-24T04:23:11.3764987Z Compressed size of image in MB: 21755.0470495224 2025-07-24T04:23:11.3766032Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f 2025-07-24T04:23:11.4034406Z Prepare all required actions 2025-07-24T04:23:11.4063461Z ##[group]Run ./.github/actions/get-workflow-job-id 2025-07-24T04:23:11.4063784Z with: 2025-07-24T04:23:11.4064214Z github-token: *** 2025-07-24T04:23:11.4064439Z env: 2025-07-24T04:23:11.4064643Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:11.4065031Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:11.4065586Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:11.4066119Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:11.4066983Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:11.4067747Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:11.4068011Z AWS_REGION: us-east-1 2025-07-24T04:23:11.4068338Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:11.4068683Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:11.4073761Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:11.4074004Z ##[endgroup] 2025-07-24T04:23:11.4089075Z ##[group]Run set -eux 2025-07-24T04:23:11.4089337Z set -eux 2025-07-24T04:23:11.4089728Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2025-07-24T04:23:11.4114557Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:11.4114911Z env: 2025-07-24T04:23:11.4115117Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:11.4115516Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:11.4116075Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:11.4116580Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:11.4117453Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:11.4118236Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:11.4118497Z AWS_REGION: us-east-1 2025-07-24T04:23:11.4118785Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:11.4119169Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:11.4124379Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:11.4124779Z GITHUB_TOKEN: *** 2025-07-24T04:23:11.4125001Z ##[endgroup] 2025-07-24T04:23:11.4174105Z + python3 .github/scripts/get_workflow_job_id.py 16487210679 gpud4f2 2025-07-24T04:23:12.3522073Z Setting output job-id=46615439117 2025-07-24T04:23:12.3523096Z Setting output job-name=linux-jammy-rocm-py3.10 / test (default, 1, 2, linux.rocm.gpu.2) 2025-07-24T04:23:12.3712261Z Prepare all required actions 2025-07-24T04:23:12.3712684Z Getting action download info 2025-07-24T04:23:12.5219832Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2025-07-24T04:23:13.0820452Z Download action repository 'actions/download-artifact@v4' (SHA:d3f86a106a0bac45b974a628896c90dbdf5c8093) 2025-07-24T04:23:13.7890886Z ##[group]Run ./.github/actions/download-build-artifacts 2025-07-24T04:23:13.7891221Z with: 2025-07-24T04:23:13.7891437Z name: linux-jammy-rocm-py3.10 2025-07-24T04:23:13.7891708Z s3-bucket: gha-artifacts 2025-07-24T04:23:13.7891939Z env: 2025-07-24T04:23:13.7892136Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:13.7892504Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:13.7893096Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:13.7893649Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:13.7894499Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:13.7895258Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:13.7895502Z AWS_REGION: us-east-1 2025-07-24T04:23:13.7895783Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:13.7896117Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:13.7901153Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:13.7901387Z ##[endgroup] 2025-07-24T04:23:13.7930029Z ##[group]Run seemethere/download-artifact-s3@v4 2025-07-24T04:23:13.7930328Z with: 2025-07-24T04:23:13.7930549Z name: linux-jammy-rocm-py3.10 2025-07-24T04:23:13.7930817Z s3-bucket: gha-artifacts 2025-07-24T04:23:13.7931057Z region: us-east-1 2025-07-24T04:23:13.7931272Z env: 2025-07-24T04:23:13.7931473Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:13.7931853Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:13.7932394Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:13.7932892Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:13.7933734Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:13.7934481Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:13.7934723Z AWS_REGION: us-east-1 2025-07-24T04:23:13.7935001Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:13.7935331Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:13.7940383Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:13.7940621Z ##[endgroup] 2025-07-24T04:23:14.2843555Z (node:3024221) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2025-07-24T04:23:14.2844131Z 2025-07-24T04:23:14.2844352Z Please migrate your code to use AWS SDK for JavaScript (v3). 2025-07-24T04:23:14.2844939Z For more information, check the migration guide at https://a.co/7PzMCcy 2025-07-24T04:23:14.4241813Z (Use `node --trace-warnings ...` to show where the warning was created) 2025-07-24T04:23:14.4242846Z Found 1 objects with prefix pytorch/pytorch/16487210679/linux-jammy-rocm-py3.10/ 2025-07-24T04:23:14.4243917Z Starting download (1/1): /var/home/pytorchci/actions-runner/_work/pytorch/pytorch/artifacts.zip 2025-07-24T04:23:38.5594099Z Finished download (1/1): /var/home/pytorchci/actions-runner/_work/pytorch/pytorch/artifacts.zip 2025-07-24T04:23:38.5604358Z Artifact download has finished successfully 2025-07-24T04:23:38.6028446Z ##[group]Run unzip -o artifacts.zip 2025-07-24T04:23:38.6028771Z unzip -o artifacts.zip 2025-07-24T04:23:38.6064437Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:38.6064794Z env: 2025-07-24T04:23:38.6065003Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:38.6065643Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:38.6066226Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:38.6066737Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:38.6067576Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:38.6068357Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:38.6068599Z AWS_REGION: us-east-1 2025-07-24T04:23:38.6068905Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:38.6069237Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:38.6074280Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:38.6074515Z ##[endgroup] 2025-07-24T04:23:38.6146595Z Archive: artifacts.zip 2025-07-24T04:23:38.6147694Z creating: dist/ 2025-07-24T04:23:44.9213686Z inflating: dist/torch-2.9.0a0+git27c8ef1-cp310-cp310-linux_x86_64.whl 2025-07-24T04:23:44.9366313Z inflating: dist/.ninja_log 2025-07-24T04:23:44.9367060Z creating: build/custom_test_artifacts/ 2025-07-24T04:23:44.9367676Z creating: build/custom_test_artifacts/custom-op-build/ 2025-07-24T04:23:44.9368410Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2025-07-24T04:23:44.9369277Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2025-07-24T04:23:44.9373485Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2025-07-24T04:23:44.9374460Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/ 2025-07-24T04:23:44.9375419Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CMakeSystem.cmake 2025-07-24T04:23:44.9376428Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CompilerIdC/ 2025-07-24T04:23:44.9377184Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CompilerIdC/tmp/ 2025-07-24T04:23:44.9377859Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CompilerIdC/CMakeCCompilerId.c 2025-07-24T04:23:44.9379562Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CompilerIdC/a.out 2025-07-24T04:23:44.9380470Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CMakeCCompiler.cmake 2025-07-24T04:23:44.9381092Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CompilerIdCXX/ 2025-07-24T04:23:44.9381681Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CompilerIdCXX/tmp/ 2025-07-24T04:23:44.9384010Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CompilerIdCXX/CMakeCXXCompilerId.cpp 2025-07-24T04:23:44.9385488Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CompilerIdCXX/a.out 2025-07-24T04:23:44.9386397Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CMakeCXXCompiler.cmake 2025-07-24T04:23:44.9388072Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CMakeDetermineCompilerABI_C.bin 2025-07-24T04:23:44.9389602Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CMakeDetermineCompilerABI_CXX.bin 2025-07-24T04:23:44.9390266Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2025-07-24T04:23:44.9390780Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2025-07-24T04:23:44.9391326Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2025-07-24T04:23:44.9391881Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2025-07-24T04:23:44.9392504Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2025-07-24T04:23:44.9393700Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2025-07-24T04:23:44.9394661Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2025-07-24T04:23:44.9395306Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2025-07-24T04:23:44.9395944Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2025-07-24T04:23:44.9396601Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2025-07-24T04:23:44.9397249Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2025-07-24T04:23:44.9397886Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2025-07-24T04:23:44.9398543Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2025-07-24T04:23:44.9421013Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2025-07-24T04:23:44.9632767Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2025-07-24T04:23:44.9633795Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2025-07-24T04:23:44.9634906Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2025-07-24T04:23:44.9636183Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2025-07-24T04:23:44.9637285Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2025-07-24T04:23:44.9637934Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2025-07-24T04:23:44.9638618Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2025-07-24T04:23:44.9639316Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2025-07-24T04:23:44.9640007Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2025-07-24T04:23:44.9640686Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2025-07-24T04:23:44.9641357Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2025-07-24T04:23:44.9662739Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2025-07-24T04:23:44.9751817Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2025-07-24T04:23:44.9753054Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2025-07-24T04:23:44.9754180Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2025-07-24T04:23:44.9755200Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2025-07-24T04:23:44.9756135Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2025-07-24T04:23:44.9757076Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2025-07-24T04:23:44.9757772Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/InstallScripts.json 2025-07-24T04:23:44.9758376Z inflating: build/custom_test_artifacts/custom-op-build/hipblaslt_test_outer_vec.cc 2025-07-24T04:23:44.9758947Z inflating: build/custom_test_artifacts/custom-op-build/hipblaslt_test_vec_ext.cc 2025-07-24T04:23:44.9759478Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2025-07-24T04:23:44.9760062Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2025-07-24T04:23:44.9760701Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2025-07-24T04:23:44.9939181Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2025-07-24T04:23:44.9999827Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2025-07-24T04:23:45.0000591Z creating: build/custom_test_artifacts/jit-hook-build/ 2025-07-24T04:23:45.0001268Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2025-07-24T04:23:45.0002082Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2025-07-24T04:23:45.0005712Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2025-07-24T04:23:45.0006652Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/ 2025-07-24T04:23:45.0007577Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CMakeSystem.cmake 2025-07-24T04:23:45.0008274Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CompilerIdC/ 2025-07-24T04:23:45.0008842Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CompilerIdC/tmp/ 2025-07-24T04:23:45.0009725Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CompilerIdC/CMakeCCompilerId.c 2025-07-24T04:23:45.0011352Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CompilerIdC/a.out 2025-07-24T04:23:45.0011988Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CMakeCCompiler.cmake 2025-07-24T04:23:45.0012595Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CompilerIdCXX/ 2025-07-24T04:23:45.0013175Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CompilerIdCXX/tmp/ 2025-07-24T04:23:45.0015554Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CompilerIdCXX/CMakeCXXCompilerId.cpp 2025-07-24T04:23:45.0017025Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CompilerIdCXX/a.out 2025-07-24T04:23:45.0017957Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CMakeCXXCompiler.cmake 2025-07-24T04:23:45.0019605Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CMakeDetermineCompilerABI_C.bin 2025-07-24T04:23:45.0021156Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CMakeDetermineCompilerABI_CXX.bin 2025-07-24T04:23:45.0021788Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2025-07-24T04:23:45.0022291Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2025-07-24T04:23:45.0022817Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2025-07-24T04:23:45.0023366Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2025-07-24T04:23:45.0023989Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2025-07-24T04:23:45.0024713Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2025-07-24T04:23:45.0025415Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2025-07-24T04:23:45.0026051Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2025-07-24T04:23:45.0026705Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2025-07-24T04:23:45.0027364Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2025-07-24T04:23:45.0028024Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2025-07-24T04:23:45.0028683Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2025-07-24T04:23:45.0029343Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2025-07-24T04:23:45.0052314Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2025-07-24T04:23:45.0122760Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2025-07-24T04:23:45.0124007Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2025-07-24T04:23:45.0125122Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2025-07-24T04:23:45.0126129Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2025-07-24T04:23:45.0127063Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2025-07-24T04:23:45.0127959Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2025-07-24T04:23:45.0128565Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/InstallScripts.json 2025-07-24T04:23:45.0129135Z inflating: build/custom_test_artifacts/jit-hook-build/hipblaslt_test_outer_vec.cc 2025-07-24T04:23:45.0129680Z inflating: build/custom_test_artifacts/jit-hook-build/hipblaslt_test_vec_ext.cc 2025-07-24T04:23:45.0130199Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2025-07-24T04:23:45.0130689Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2025-07-24T04:23:45.0131340Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2025-07-24T04:23:45.0173181Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2025-07-24T04:23:45.0173958Z creating: build/custom_test_artifacts/custom-backend-build/ 2025-07-24T04:23:45.0174716Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2025-07-24T04:23:45.0175618Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2025-07-24T04:23:45.0178945Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2025-07-24T04:23:45.0179564Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/ 2025-07-24T04:23:45.0180158Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CMakeSystem.cmake 2025-07-24T04:23:45.0180790Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CompilerIdC/ 2025-07-24T04:23:45.0181404Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CompilerIdC/tmp/ 2025-07-24T04:23:45.0183354Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CompilerIdC/CMakeCCompilerId.c 2025-07-24T04:23:45.0184808Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CompilerIdC/a.out 2025-07-24T04:23:45.0185494Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CMakeCCompiler.cmake 2025-07-24T04:23:45.0186151Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CompilerIdCXX/ 2025-07-24T04:23:45.0186780Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CompilerIdCXX/tmp/ 2025-07-24T04:23:45.0189051Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CompilerIdCXX/CMakeCXXCompilerId.cpp 2025-07-24T04:23:45.0190547Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CompilerIdCXX/a.out 2025-07-24T04:23:45.0191463Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CMakeCXXCompiler.cmake 2025-07-24T04:23:45.0193108Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CMakeDetermineCompilerABI_C.bin 2025-07-24T04:23:45.0194614Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CMakeDetermineCompilerABI_CXX.bin 2025-07-24T04:23:45.0195293Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2025-07-24T04:23:45.0195836Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2025-07-24T04:23:45.0196611Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2025-07-24T04:23:45.0197404Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2025-07-24T04:23:45.0198097Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2025-07-24T04:23:45.0198862Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2025-07-24T04:23:45.0199600Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2025-07-24T04:23:45.0200294Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2025-07-24T04:23:45.0201011Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2025-07-24T04:23:45.0201739Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2025-07-24T04:23:45.0202566Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2025-07-24T04:23:45.0203293Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2025-07-24T04:23:45.0204009Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2025-07-24T04:23:45.0206746Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2025-07-24T04:23:45.0339713Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2025-07-24T04:23:45.0340947Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2025-07-24T04:23:45.0342174Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2025-07-24T04:23:45.0343574Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2025-07-24T04:23:45.0344905Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2025-07-24T04:23:45.0346160Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2025-07-24T04:23:45.0347433Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2025-07-24T04:23:45.0348636Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2025-07-24T04:23:45.0349390Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2025-07-24T04:23:45.0350147Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2025-07-24T04:23:45.0350882Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2025-07-24T04:23:45.0369594Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2025-07-24T04:23:45.0429493Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2025-07-24T04:23:45.0430864Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2025-07-24T04:23:45.0432046Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2025-07-24T04:23:45.0433115Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2025-07-24T04:23:45.0434110Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2025-07-24T04:23:45.0435076Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2025-07-24T04:23:45.0436449Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/InstallScripts.json 2025-07-24T04:23:45.0437753Z inflating: build/custom_test_artifacts/custom-backend-build/hipblaslt_test_outer_vec.cc 2025-07-24T04:23:45.0438573Z inflating: build/custom_test_artifacts/custom-backend-build/hipblaslt_test_vec_ext.cc 2025-07-24T04:23:45.0439116Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2025-07-24T04:23:45.0439618Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2025-07-24T04:23:45.0440125Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2025-07-24T04:23:45.0548732Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2025-07-24T04:23:45.0591201Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2025-07-24T04:23:45.0591891Z creating: build/lib/ 2025-07-24T04:23:45.0681336Z inflating: build/lib/libprotobuf-lite.a 2025-07-24T04:23:45.1161929Z inflating: build/lib/libprotobuf.a 2025-07-24T04:23:45.1698314Z inflating: build/lib/libprotoc.a 2025-07-24T04:23:45.1707815Z inflating: build/lib/libpthreadpool.a 2025-07-24T04:23:45.1716410Z inflating: build/lib/libcpuinfo.a 2025-07-24T04:23:45.1724899Z inflating: build/lib/libcpuinfo_internals.a 2025-07-24T04:23:45.1725767Z inflating: build/lib/libclog.a 2025-07-24T04:23:45.1746214Z inflating: build/lib/libpytorch_qnnpack.a 2025-07-24T04:23:45.1748469Z inflating: build/lib/libnnpack_reference_layers.a 2025-07-24T04:23:45.1767711Z inflating: build/lib/libnnpack.a 2025-07-24T04:23:45.1968256Z inflating: build/lib/libmicrokernels-prod.a 2025-07-24T04:23:45.2920073Z inflating: build/lib/libmicrokernels-all.a 2025-07-24T04:23:45.2995777Z inflating: build/lib/libgtest.a 2025-07-24T04:23:45.3014360Z inflating: build/lib/libgmock.a 2025-07-24T04:23:45.3015068Z inflating: build/lib/libgtest_main.a 2025-07-24T04:23:45.3112508Z inflating: build/lib/libXNNPACK.a 2025-07-24T04:23:45.3113101Z inflating: build/lib/libgmock_main.a 2025-07-24T04:23:45.3194417Z inflating: build/lib/libbenchmark.a 2025-07-24T04:23:45.3195012Z inflating: build/lib/libbenchmark_main.a 2025-07-24T04:23:45.3203838Z inflating: build/lib/libittnotify.a 2025-07-24T04:23:45.3204510Z inflating: build/lib/libjitprofiling.a 2025-07-24T04:23:45.3273815Z inflating: build/lib/libasmjit.a 2025-07-24T04:23:45.4585588Z inflating: build/lib/libfbgemm.a 2025-07-24T04:23:45.4614650Z inflating: build/lib/libtensorpipe_uv.a 2025-07-24T04:23:45.5219165Z inflating: build/lib/libtensorpipe.a 2025-07-24T04:23:45.5351157Z inflating: build/lib/libgloo.a 2025-07-24T04:23:45.5402779Z inflating: build/lib/libonnx_proto.a 2025-07-24T04:23:45.5880718Z inflating: build/lib/libgloo_hip.a 2025-07-24T04:23:45.6662746Z inflating: build/lib/libonnx.a 2025-07-24T04:23:46.7950397Z inflating: build/lib/libdnnl.a 2025-07-24T04:23:46.7970218Z inflating: build/lib/libfmt.a 2025-07-24T04:23:46.8282249Z inflating: build/lib/libkineto.a 2025-07-24T04:23:46.8397294Z inflating: build/lib/libc10.so 2025-07-24T04:23:46.8398652Z inflating: build/lib/libtorch_global_deps.so 2025-07-24T04:23:46.8453590Z inflating: build/lib/libc10_hip.so 2025-07-24T04:23:46.8455269Z inflating: build/lib/libcaffe2_nvrtc.so 2025-07-24T04:23:50.4668227Z inflating: build/lib/libtorch_cpu.so 2025-07-24T04:23:50.4672357Z inflating: build/lib/libshm.so 2025-07-24T04:23:51.3915615Z inflating: build/lib/libtorch_hip.so 2025-07-24T04:23:51.3916492Z inflating: build/lib/libtorch.so 2025-07-24T04:23:51.3936314Z inflating: build/lib/libjitbackend_test.so 2025-07-24T04:23:51.3962111Z inflating: build/lib/libbackend_with_compiler.so 2025-07-24T04:23:51.4037669Z inflating: build/lib/libtorchbind_test.so 2025-07-24T04:23:51.4066167Z inflating: build/lib/libaoti_custom_ops.so 2025-07-24T04:23:51.6261619Z inflating: build/lib/libtorch_python.so 2025-07-24T04:23:51.6301693Z inflating: build/lib/libnnapi_backend.so 2025-07-24T04:23:51.6302209Z creating: build/bin/ 2025-07-24T04:23:51.6303076Z creating: build/bin/CMakeFiles/ 2025-07-24T04:23:51.6303594Z inflating: build/bin/cmake_install.cmake 2025-07-24T04:23:51.6304114Z inflating: build/bin/CTestTestfile.cmake 2025-07-24T04:23:51.6790268Z inflating: build/bin/protoc-3.13.0.0 2025-07-24T04:23:51.7276185Z inflating: build/bin/protoc 2025-07-24T04:23:51.7335221Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2025-07-24T04:23:51.7395794Z inflating: build/bin/c10_DeviceGuard_test 2025-07-24T04:23:51.7456782Z inflating: build/bin/c10_Device_test 2025-07-24T04:23:51.7526318Z inflating: build/bin/c10_DispatchKeySet_test 2025-07-24T04:23:51.7589790Z inflating: build/bin/c10_Scalar_test 2025-07-24T04:23:51.7647809Z inflating: build/bin/c10_StreamGuard_test 2025-07-24T04:23:51.7707982Z inflating: build/bin/c10_SymInt_test 2025-07-24T04:23:51.7771926Z inflating: build/bin/c10_InlineDeviceGuard_test 2025-07-24T04:23:51.7837480Z inflating: build/bin/c10_InlineStreamGuard_test 2025-07-24T04:23:51.7895379Z inflating: build/bin/c10_ConstexprCrc_test 2025-07-24T04:23:51.7961134Z inflating: build/bin/c10_SizesAndStrides_test 2025-07-24T04:23:51.8042692Z inflating: build/bin/c10_cow_test 2025-07-24T04:23:51.8101278Z inflating: build/bin/c10_DeadlockDetection_test 2025-07-24T04:23:51.8163813Z inflating: build/bin/c10_Bitset_test 2025-07-24T04:23:51.8223085Z inflating: build/bin/c10_ArrayRef_test 2025-07-24T04:23:51.8290060Z inflating: build/bin/c10_Enumerate_test 2025-07-24T04:23:51.8352299Z inflating: build/bin/c10_IntrusiveList_test 2025-07-24T04:23:51.8417763Z inflating: build/bin/c10_Metaprogramming_test 2025-07-24T04:23:51.8477509Z inflating: build/bin/c10_Half_test 2025-07-24T04:23:51.8543214Z inflating: build/bin/c10_LeftRight_test 2025-07-24T04:23:51.8606494Z inflating: build/bin/c10_NetworkFlow_test 2025-07-24T04:23:51.8665163Z inflating: build/bin/c10_Semaphore_test 2025-07-24T04:23:51.8724194Z inflating: build/bin/c10_Synchronized_test 2025-07-24T04:23:51.8789339Z inflating: build/bin/c10_ThreadLocal_test 2025-07-24T04:23:51.8850605Z inflating: build/bin/c10_TypeIndex_test 2025-07-24T04:23:51.8908538Z inflating: build/bin/c10_TypeTraits_test 2025-07-24T04:23:51.8968713Z inflating: build/bin/c10_TypeList_test 2025-07-24T04:23:51.9029460Z inflating: build/bin/c10_accumulate_test 2025-07-24T04:23:51.9095040Z inflating: build/bin/c10_bfloat16_test 2025-07-24T04:23:51.9154756Z inflating: build/bin/c10_bit_cast_test 2025-07-24T04:23:51.9221275Z inflating: build/bin/c10_complex_math_test 2025-07-24T04:23:51.9279731Z inflating: build/bin/c10_error_test 2025-07-24T04:23:51.9344544Z inflating: build/bin/c10_complex_test 2025-07-24T04:23:51.9406278Z inflating: build/bin/c10_exception_test 2025-07-24T04:23:51.9465697Z inflating: build/bin/c10_flags_test 2025-07-24T04:23:51.9525230Z inflating: build/bin/c10_generic_math_test 2025-07-24T04:23:51.9585438Z inflating: build/bin/c10_irange_test 2025-07-24T04:23:51.9648297Z inflating: build/bin/c10_lazy_test 2025-07-24T04:23:51.9837012Z inflating: build/bin/c10_intrusive_ptr_test 2025-07-24T04:23:51.9904065Z inflating: build/bin/c10_logging_test 2025-07-24T04:23:51.9991709Z inflating: build/bin/c10_optional_test 2025-07-24T04:23:52.0063984Z inflating: build/bin/c10_ordered_preserving_dict_test 2025-07-24T04:23:52.0126678Z inflating: build/bin/c10_registry_test 2025-07-24T04:23:52.0187385Z inflating: build/bin/c10_ssize_test 2025-07-24T04:23:52.0365214Z inflating: build/bin/c10_small_vector_test 2025-07-24T04:23:52.0431630Z inflating: build/bin/c10_string_util_test 2025-07-24T04:23:52.0490825Z inflating: build/bin/c10_tempfile_test 2025-07-24T04:23:52.0548914Z inflating: build/bin/c10_string_view_test 2025-07-24T04:23:52.0600291Z inflating: build/bin/c10_intrusive_ptr_benchmark 2025-07-24T04:23:52.0666293Z inflating: build/bin/c10_typeid_test 2025-07-24T04:23:52.0724800Z inflating: build/bin/c10_hip_HIPAssertionsTest_1_var_test 2025-07-24T04:23:52.0782445Z inflating: build/bin/c10_hip_HIPAssertionsTest_catches_stream 2025-07-24T04:23:52.0840451Z inflating: build/bin/c10_hip_HIPAssertionsTest_catches_thread_and_block_and_device 2025-07-24T04:23:52.0898271Z inflating: build/bin/c10_hip_HIPAssertionsTest_from_2_processes 2025-07-24T04:23:52.0956280Z inflating: build/bin/c10_hip_HIPAssertionsTest_multiple_writes_from_blocks_and_threads 2025-07-24T04:23:52.1014412Z inflating: build/bin/c10_hip_HIPAssertionsTest_multiple_writes_from_multiple_blocks 2025-07-24T04:23:52.1072588Z inflating: build/bin/c10_hip_HIPTest 2025-07-24T04:23:52.1130846Z inflating: build/bin/c10_hip_HIPAssertionsTest_multiple_writes_from_same_block 2025-07-24T04:23:52.2370242Z inflating: build/bin/vec_test_all_types_DEFAULT 2025-07-24T04:23:52.3601981Z inflating: build/bin/vec_test_all_types_AVX512 2025-07-24T04:23:52.4839462Z inflating: build/bin/vec_test_all_types_AVX2 2025-07-24T04:23:52.4901335Z inflating: build/bin/BackoffTest 2025-07-24T04:23:52.4964328Z inflating: build/bin/FileStoreTest 2025-07-24T04:23:52.5030973Z inflating: build/bin/TCPStoreTest 2025-07-24T04:23:52.5094334Z inflating: build/bin/HashStoreTest 2025-07-24T04:23:52.5171227Z inflating: build/bin/ProcessGroupGlooTest 2025-07-24T04:23:52.5174355Z inflating: build/bin/example_allreduce 2025-07-24T04:23:52.5178612Z inflating: build/bin/torch_shm_manager 2025-07-24T04:23:52.5241226Z inflating: build/bin/static_runtime_bench 2025-07-24T04:23:52.5524710Z inflating: build/bin/static_runtime_test 2025-07-24T04:23:52.5610732Z inflating: build/bin/Dict_test 2025-07-24T04:23:52.5672412Z inflating: build/bin/Dimname_test 2025-07-24T04:23:52.5748861Z inflating: build/bin/MaybeOwned_test 2025-07-24T04:23:52.5815628Z inflating: build/bin/NamedTensor_test 2025-07-24T04:23:52.5884136Z inflating: build/bin/apply_utils_test 2025-07-24T04:23:52.5953125Z inflating: build/bin/atest 2025-07-24T04:23:52.6027554Z inflating: build/bin/basic 2025-07-24T04:23:52.6092609Z inflating: build/bin/broadcast_test 2025-07-24T04:23:52.6152649Z inflating: build/bin/cpu_allocator_test 2025-07-24T04:23:52.6220433Z inflating: build/bin/cpu_generator_test 2025-07-24T04:23:52.6282921Z inflating: build/bin/cpu_profiling_allocator_test 2025-07-24T04:23:52.6389461Z inflating: build/bin/cpu_rng_test 2025-07-24T04:23:52.6449752Z inflating: build/bin/dlconvertor_test 2025-07-24T04:23:52.6516800Z inflating: build/bin/extension_backend_test 2025-07-24T04:23:52.6609740Z inflating: build/bin/half_test 2025-07-24T04:23:52.6719618Z inflating: build/bin/ivalue_test 2025-07-24T04:23:52.6778444Z inflating: build/bin/lazy_tensor_test 2025-07-24T04:23:52.6841445Z inflating: build/bin/math_kernel_test 2025-07-24T04:23:52.6904575Z inflating: build/bin/memory_format_test 2025-07-24T04:23:52.6967575Z inflating: build/bin/memory_overlapping_test 2025-07-24T04:23:52.7030096Z inflating: build/bin/mobile_memory_cleanup 2025-07-24T04:23:52.7090314Z inflating: build/bin/operator_name_test 2025-07-24T04:23:52.7156019Z inflating: build/bin/native_test 2025-07-24T04:23:52.7215976Z inflating: build/bin/operators_test 2025-07-24T04:23:52.7277293Z inflating: build/bin/packedtensoraccessor_test 2025-07-24T04:23:52.7344606Z inflating: build/bin/quantized_test 2025-07-24T04:23:52.7422779Z inflating: build/bin/pow_test 2025-07-24T04:23:52.7481906Z inflating: build/bin/reduce_ops_test 2025-07-24T04:23:52.7542055Z inflating: build/bin/reportMemoryUsage_test 2025-07-24T04:23:52.7602637Z inflating: build/bin/StorageUtils_test 2025-07-24T04:23:52.7668900Z inflating: build/bin/scalar_tensor_test 2025-07-24T04:23:52.7737681Z inflating: build/bin/scalar_test 2025-07-24T04:23:52.7799143Z inflating: build/bin/stride_properties_test 2025-07-24T04:23:52.7892295Z inflating: build/bin/tensor_iterator_test 2025-07-24T04:23:52.7957629Z inflating: build/bin/type_ptr_test 2025-07-24T04:23:52.7960242Z inflating: build/bin/thread_init_test 2025-07-24T04:23:52.8024191Z inflating: build/bin/test_parallel 2025-07-24T04:23:52.8093964Z inflating: build/bin/type_test 2025-07-24T04:23:52.8156691Z inflating: build/bin/undefined_tensor_test 2025-07-24T04:23:52.8215353Z inflating: build/bin/verify_api_visibility 2025-07-24T04:23:52.8296571Z inflating: build/bin/legacy_vmap_test 2025-07-24T04:23:52.8357001Z inflating: build/bin/weakref_test 2025-07-24T04:23:52.8426882Z inflating: build/bin/IListRef_test 2025-07-24T04:23:52.8487622Z inflating: build/bin/wrapdim_test 2025-07-24T04:23:52.8548481Z inflating: build/bin/xla_tensor_test 2025-07-24T04:23:52.8671197Z inflating: build/bin/List_test 2025-07-24T04:23:52.8808050Z inflating: build/bin/kernel_function_legacy_test 2025-07-24T04:23:52.8951584Z inflating: build/bin/kernel_lambda_legacy_test 2025-07-24T04:23:52.9061018Z inflating: build/bin/kernel_function_test 2025-07-24T04:23:52.9138777Z inflating: build/bin/KernelFunction_test 2025-07-24T04:23:52.9256232Z inflating: build/bin/kernel_lambda_test 2025-07-24T04:23:52.9327065Z inflating: build/bin/kernel_stackbased_test 2025-07-24T04:23:52.9435859Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2025-07-24T04:23:52.9496368Z inflating: build/bin/CppSignature_test 2025-07-24T04:23:52.9554399Z inflating: build/bin/op_allowlist_test 2025-07-24T04:23:52.9906573Z inflating: build/bin/op_registration_test 2025-07-24T04:23:52.9984596Z inflating: build/bin/inline_container_test 2025-07-24T04:23:53.0049478Z inflating: build/bin/backend_fallback_test 2025-07-24T04:23:53.0107543Z inflating: build/bin/hip_complex_math_test 2025-07-24T04:23:53.0165410Z inflating: build/bin/hip_complex_test 2025-07-24T04:23:53.0227816Z inflating: build/bin/hip_apply_test 2025-07-24T04:23:53.0285931Z inflating: build/bin/hip_distributions_test 2025-07-24T04:23:53.0343764Z inflating: build/bin/hip_generator_test 2025-07-24T04:23:53.0401805Z inflating: build/bin/hip_half_test 2025-07-24T04:23:53.0460369Z inflating: build/bin/hip_integer_divider_test 2025-07-24T04:23:53.0518322Z inflating: build/bin/hip_optional_test 2025-07-24T04:23:53.0576379Z inflating: build/bin/hip_packedtensoraccessor_test 2025-07-24T04:23:53.0634733Z inflating: build/bin/hip_vectorized_test 2025-07-24T04:23:53.0695713Z inflating: build/bin/hip_dlconvertor_test 2025-07-24T04:23:53.1905096Z inflating: build/bin/test_jit 2025-07-24T04:23:53.2214226Z inflating: build/bin/test_nativert 2025-07-24T04:23:53.3081150Z inflating: build/bin/test_tensorexpr 2025-07-24T04:23:53.3095137Z inflating: build/bin/tutorial_tensorexpr 2025-07-24T04:23:53.3159627Z inflating: build/bin/test_dist_autograd 2025-07-24T04:23:53.3238312Z inflating: build/bin/test_cpp_rpc 2025-07-24T04:23:53.3240952Z inflating: build/bin/parallel_benchmark 2025-07-24T04:23:53.4513665Z inflating: build/bin/test_api 2025-07-24T04:23:53.4590901Z inflating: build/bin/test_mobile_nnc 2025-07-24T04:23:53.4600208Z inflating: build/bin/aot_model_compiler_test 2025-07-24T04:23:53.4989823Z inflating: build/bin/test_lazy 2025-07-24T04:23:53.4990295Z creating: .additional_ci_files/ 2025-07-24T04:23:53.5164754Z inflating: .additional_ci_files/test-times.json 2025-07-24T04:23:53.5830284Z inflating: .additional_ci_files/test-class-times.json 2025-07-24T04:23:53.5870791Z ##[group]Run rm artifacts.zip 2025-07-24T04:23:53.5871078Z rm artifacts.zip 2025-07-24T04:23:53.5900428Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:53.5900774Z env: 2025-07-24T04:23:53.5900976Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:53.5901353Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:53.5902127Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:53.5902872Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:53.5903737Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:53.5904504Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:53.5904755Z AWS_REGION: us-east-1 2025-07-24T04:23:53.5905061Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:53.5905401Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:53.5910446Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:53.5910676Z ##[endgroup] 2025-07-24T04:23:53.9243949Z ##[group]Run df -H 2025-07-24T04:23:53.9244180Z df -H 2025-07-24T04:23:53.9274044Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:53.9274413Z env: 2025-07-24T04:23:53.9274613Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:53.9274994Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:53.9275547Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:53.9276082Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:53.9276927Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:53.9277682Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:53.9277934Z AWS_REGION: us-east-1 2025-07-24T04:23:53.9278224Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:53.9278561Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:53.9283705Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:53.9283934Z ##[endgroup] 2025-07-24T04:23:53.9372177Z Filesystem Size Used Avail Use% Mounted on 2025-07-24T04:23:53.9372722Z tmpfs 109G 33M 109G 1% /run 2025-07-24T04:23:53.9373217Z /dev/nvme0n1p2 944G 75G 822G 9% / 2025-07-24T04:23:53.9373677Z tmpfs 542G 33k 542G 1% /dev/shm 2025-07-24T04:23:53.9374092Z tmpfs 5.3M 0 5.3M 0% /run/lock 2025-07-24T04:23:53.9374542Z /dev/nvme0n1p1 536M 6.4M 530M 2% /boot/efi 2025-07-24T04:23:53.9375020Z /dev/nvme1n1p1 3.8T 2.4T 1.3T 65% /media/4TB 2025-07-24T04:23:53.9375499Z tmpfs 109G 33k 109G 1% /run/user/1307800118 2025-07-24T04:23:53.9376023Z 172.18.148.8:/export/amd2 5.5T 278G 5.3T 6% /mnt 2025-07-24T04:23:53.9376679Z pure1.jax.cs.cpe.ice.amd.com:/homes/amd-pytorch 108G 327M 108G 1% /home/amd-pytorch 2025-07-24T04:23:53.9377256Z 172.18.148.15:/GroupStorage 165T 127T 39T 77% /groups 2025-07-24T04:23:53.9377728Z 172.18.148.15:/GroupStorage/Scratch 5.5T 1.8T 3.8T 33% /scratch 2025-07-24T04:23:53.9378218Z pure1.jax.cs.cpe.ice.amd.com:/homes/jlay 108G 19G 89G 18% /home/jlay 2025-07-24T04:23:53.9410907Z Prepare all required actions 2025-07-24T04:23:53.9411283Z Getting action download info 2025-07-24T04:23:54.0908090Z ##[group]Run ./.github/actions/download-td-artifacts 2025-07-24T04:23:54.0908414Z with: 2025-07-24T04:23:54.0908610Z env: 2025-07-24T04:23:54.0908809Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:54.0909193Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:54.0909742Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:54.0910263Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:54.0911115Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:54.0912098Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:54.0912349Z AWS_REGION: us-east-1 2025-07-24T04:23:54.0912679Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:54.0913014Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:54.0918069Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:54.0918305Z ##[endgroup] 2025-07-24T04:23:54.0944816Z ##[group]Run seemethere/download-artifact-s3@v4 2025-07-24T04:23:54.0945112Z with: 2025-07-24T04:23:54.0945294Z name: td_results 2025-07-24T04:23:54.0945512Z s3-bucket: gha-artifacts 2025-07-24T04:23:54.0945748Z region: us-east-1 2025-07-24T04:23:54.0945952Z env: 2025-07-24T04:23:54.0946142Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:54.0946515Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:54.0947057Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:54.0947571Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:54.0948425Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:54.0949166Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:54.0949419Z AWS_REGION: us-east-1 2025-07-24T04:23:54.0949693Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:54.0950024Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:54.0955084Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:54.0955314Z ##[endgroup] 2025-07-24T04:23:54.5868851Z (node:3024284) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2025-07-24T04:23:54.5869597Z 2025-07-24T04:23:54.5869884Z Please migrate your code to use AWS SDK for JavaScript (v3). 2025-07-24T04:23:54.5870695Z For more information, check the migration guide at https://a.co/7PzMCcy 2025-07-24T04:23:54.5871515Z (Use `node --trace-warnings ...` to show where the warning was created) 2025-07-24T04:23:54.7367402Z Found 1 objects with prefix pytorch/pytorch/16487210679/td_results/ 2025-07-24T04:23:54.7368462Z Starting download (1/1): /var/home/pytorchci/actions-runner/_work/pytorch/pytorch/td_results.json 2025-07-24T04:23:55.0145861Z Finished download (1/1): /var/home/pytorchci/actions-runner/_work/pytorch/pytorch/td_results.json 2025-07-24T04:23:55.0151676Z Artifact download has finished successfully 2025-07-24T04:23:55.0529137Z ##[group]Run mkdir -p .additional_ci_files 2025-07-24T04:23:55.0529480Z mkdir -p .additional_ci_files 2025-07-24T04:23:55.0529858Z mv td_results.json .additional_ci_files/td_results.json || true 2025-07-24T04:23:55.0559739Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:55.0560095Z env: 2025-07-24T04:23:55.0560303Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:55.0560701Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:55.0561238Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:55.0561741Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:55.0562946Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:55.0563728Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:55.0563989Z AWS_REGION: us-east-1 2025-07-24T04:23:55.0564288Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:55.0564623Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:55.0569660Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:55.0569892Z ##[endgroup] 2025-07-24T04:23:55.0696620Z ##[group]Run .github/scripts/parse_ref.py 2025-07-24T04:23:55.0696958Z .github/scripts/parse_ref.py 2025-07-24T04:23:55.0729192Z shell: /usr/bin/bash -e {0} 2025-07-24T04:23:55.0729444Z env: 2025-07-24T04:23:55.0729649Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:55.0730035Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:55.0730579Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:55.0731092Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:55.0731925Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:55.0732670Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:55.0732914Z AWS_REGION: us-east-1 2025-07-24T04:23:55.0733195Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:55.0733559Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:55.0738598Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:55.0738829Z ##[endgroup] 2025-07-24T04:23:55.0977741Z Setting output tag=ciflow/trunk/149961 2025-07-24T04:23:55.1096305Z Prepare all required actions 2025-07-24T04:23:55.1096710Z Getting action download info 2025-07-24T04:23:55.2837277Z ##[group]Run ./.github/actions/filter-test-configs 2025-07-24T04:23:55.2837583Z with: 2025-07-24T04:23:55.2837938Z github-token: *** 2025-07-24T04:23:55.2838737Z test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 2, "runner": "linux.rocm.gpu.2"}, {"config": "default", "shard": 2, "num_shards": 2, "runner": "linux.rocm.gpu.2"}, {"config": "distributed", "shard": 1, "num_shards": 1, "runner": "linux.rocm.gpu.4"}]} 2025-07-24T04:23:55.2839674Z job-name: linux-jammy-rocm-py3.10 / test (default, 1, 2, linux.rocm.gpu.2) 2025-07-24T04:23:55.2840061Z env: 2025-07-24T04:23:55.2840260Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:55.2840632Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:55.2841172Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:55.2841720Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:55.2842640Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:55.2843392Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:55.2843643Z AWS_REGION: us-east-1 2025-07-24T04:23:55.2843926Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:55.2844268Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:55.2849307Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:55.2849530Z ##[endgroup] 2025-07-24T04:23:55.2881220Z ##[group]Run nick-fields/retry@v3.0.0 2025-07-24T04:23:55.2881486Z with: 2025-07-24T04:23:55.2881674Z shell: bash 2025-07-24T04:23:55.2881874Z timeout_minutes: 10 2025-07-24T04:23:55.2882089Z max_attempts: 5 2025-07-24T04:23:55.2882298Z retry_wait_seconds: 30 2025-07-24T04:23:55.2883062Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2025-07-24T04:23:55.2883756Z polling_interval_seconds: 1 2025-07-24T04:23:55.2884006Z warning_on_retry: true 2025-07-24T04:23:55.2884229Z continue_on_error: false 2025-07-24T04:23:55.2884452Z env: 2025-07-24T04:23:55.2884641Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:55.2885005Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:55.2885546Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:55.2886049Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:55.2886908Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:55.2887874Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:55.2888132Z AWS_REGION: us-east-1 2025-07-24T04:23:55.2888404Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:55.2888760Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:55.2893807Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:55.2894146Z GITHUB_TOKEN: *** 2025-07-24T04:23:55.2894364Z ##[endgroup] 2025-07-24T04:23:55.3706341Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2025-07-24T04:23:55.6721979Z Defaulting to user installation because normal site-packages is not writeable 2025-07-24T04:23:55.7831625Z Requirement already satisfied: requests==2.27.1 in /var/home/pytorchci/.local/lib/python3.10/site-packages (2.27.1) 2025-07-24T04:23:55.7836775Z Requirement already satisfied: pyyaml==6.0.1 in /var/home/pytorchci/.local/lib/python3.10/site-packages (6.0.1) 2025-07-24T04:23:55.7938268Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3/dist-packages (from requests==2.27.1) (1.26.5) 2025-07-24T04:23:55.7943201Z Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests==2.27.1) (2020.6.20) 2025-07-24T04:23:55.7952178Z Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests==2.27.1) (3.3) 2025-07-24T04:23:55.7963640Z Requirement already satisfied: charset-normalizer~=2.0.0 in /var/home/pytorchci/.local/lib/python3.10/site-packages (from requests==2.27.1) (2.0.12) 2025-07-24T04:23:56.3706609Z Command completed after 1 attempt(s). 2025-07-24T04:23:56.3787127Z ##[group]Run set -x 2025-07-24T04:23:56.3802443Z set -x 2025-07-24T04:23:56.3802693Z  2025-07-24T04:23:56.3803053Z # Use relative path here as this could be checked out anywhere, not necessarily 2025-07-24T04:23:56.3803483Z # in runner workspace 2025-07-24T04:23:56.3803848Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2025-07-24T04:23:56.3831709Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:56.3832081Z env: 2025-07-24T04:23:56.3832289Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:56.3832673Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:56.3833224Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:56.3833746Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:56.3834605Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:56.3835362Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:56.3835609Z AWS_REGION: us-east-1 2025-07-24T04:23:56.3835889Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:56.3836226Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:56.3841266Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:56.3841497Z ##[endgroup] 2025-07-24T04:23:56.3895599Z + python3 /var/home/pytorchci/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2025-07-24T04:23:56.4068559Z Setting output tag=ciflow/trunk/149961 2025-07-24T04:23:56.4125580Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2025-07-24T04:23:56.4125940Z echo "Workflow: ${GITHUB_WORKFLOW}" 2025-07-24T04:23:56.4126242Z echo "Job name: ${JOB_NAME}" 2025-07-24T04:23:56.4126505Z  2025-07-24T04:23:56.4126839Z # Use relative path here as this could be checked out anywhere, not necessarily 2025-07-24T04:23:56.4127269Z # in runner workspace 2025-07-24T04:23:56.4127642Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2025-07-24T04:23:56.4128057Z  --workflow "${GITHUB_WORKFLOW}" \ 2025-07-24T04:23:56.4128357Z  --job-name "${JOB_NAME}" \ 2025-07-24T04:23:56.4129220Z  --test-matrix "{"include": [{"config": "default", "shard": 1, "num_shards": 2, "runner": "linux.rocm.gpu.2"}, {"config": "default", "shard": 2, "num_shards": 2, "runner": "linux.rocm.gpu.2"}, {"config": "distributed", "shard": 1, "num_shards": 1, "runner": "linux.rocm.gpu.4"}]}" \ 2025-07-24T04:23:56.4130321Z  --selected-test-configs "" \ 2025-07-24T04:23:56.4130617Z  --pr-number "${PR_NUMBER}" \ 2025-07-24T04:23:56.4130895Z  --tag "${TAG}" \ 2025-07-24T04:23:56.4131157Z  --event-name "${EVENT_NAME}" \ 2025-07-24T04:23:56.4131491Z  --schedule "${SCHEDULE}" \ 2025-07-24T04:23:56.4131845Z  --branch "${HEAD_BRANCH}" 2025-07-24T04:23:56.4161076Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:56.4161414Z env: 2025-07-24T04:23:56.4161622Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:56.4161994Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:56.4162605Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:56.4163111Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:56.4164187Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:56.4164942Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:56.4165200Z AWS_REGION: us-east-1 2025-07-24T04:23:56.4165502Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:56.4165844Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:56.4170920Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:56.4171289Z GITHUB_TOKEN: *** 2025-07-24T04:23:56.4171638Z JOB_NAME: linux-jammy-rocm-py3.10 / test (default, 1, 2, linux.rocm.gpu.2) 2025-07-24T04:23:56.4172022Z PR_NUMBER: 2025-07-24T04:23:56.4172238Z TAG: ciflow/trunk/149961 2025-07-24T04:23:56.4172474Z EVENT_NAME: push 2025-07-24T04:23:56.4172673Z SCHEDULE: 2025-07-24T04:23:56.4172858Z HEAD_BRANCH: 2025-07-24T04:23:56.4173047Z ##[endgroup] 2025-07-24T04:23:56.4222962Z Workflow: trunk 2025-07-24T04:23:56.4223372Z Job name: linux-jammy-rocm-py3.10 / test (default, 1, 2, linux.rocm.gpu.2) 2025-07-24T04:23:56.7100344Z INFO:root:Found no test-config label on the PR, so all test configs are included 2025-07-24T04:23:57.0365347Z Setting output keep-going=False 2025-07-24T04:23:57.0365912Z Setting output ci-verbose-test-logs=False 2025-07-24T04:23:57.0366459Z Setting output ci-test-showlocals=False 2025-07-24T04:23:57.0366955Z Setting output ci-no-test-timeout=False 2025-07-24T04:23:57.0367442Z Setting output ci-no-td=False 2025-07-24T04:23:57.0367909Z Setting output ci-td-distributed=False 2025-07-24T04:23:57.0368390Z Setting output is-unstable=False 2025-07-24T04:23:57.0368838Z Setting output reenabled-issues= 2025-07-24T04:23:57.0370335Z Setting output test-matrix={"include": [{"config": "default", "shard": 1, "num_shards": 2, "runner": "linux.rocm.gpu.2"}, {"config": "default", "shard": 2, "num_shards": 2, "runner": "linux.rocm.gpu.2"}, {"config": "distributed", "shard": 1, "num_shards": 1, "runner": "linux.rocm.gpu.4"}]} 2025-07-24T04:23:57.0371889Z Setting output is-test-matrix-empty=False 2025-07-24T04:23:57.0539277Z ##[group]Run echo "Filtered matrix:" 2025-07-24T04:23:57.0539593Z echo "Filtered matrix:" 2025-07-24T04:23:57.0540403Z echo "{"include": [{"config": "default", "shard": 1, "num_shards": 2, "runner": "linux.rocm.gpu.2"}, {"config": "default", "shard": 2, "num_shards": 2, "runner": "linux.rocm.gpu.2"}, {"config": "distributed", "shard": 1, "num_shards": 1, "runner": "linux.rocm.gpu.4"}]}" 2025-07-24T04:23:57.0541204Z  2025-07-24T04:23:57.0541389Z echo 2025-07-24T04:23:57.0541631Z echo "Is the current job unstable? False" 2025-07-24T04:23:57.0541919Z  2025-07-24T04:23:57.0542094Z echo 2025-07-24T04:23:57.0542320Z echo "Is keep-going label set? False" 2025-07-24T04:23:57.0542582Z  2025-07-24T04:23:57.0542762Z echo 2025-07-24T04:23:57.0542971Z echo "Reenabled issues? " 2025-07-24T04:23:57.0572481Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:57.0572827Z env: 2025-07-24T04:23:57.0573033Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:57.0573423Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:57.0573971Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:57.0574512Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:57.0575367Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:57.0576128Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:57.0576376Z AWS_REGION: us-east-1 2025-07-24T04:23:57.0576656Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:57.0576997Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:57.0582051Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:57.0582276Z ##[endgroup] 2025-07-24T04:23:57.0628476Z Filtered matrix: 2025-07-24T04:23:57.0629255Z {include: [{config: default, shard: 1, num_shards: 2, runner: linux.rocm.gpu.2}, {config: default, shard: 2, num_shards: 2, runner: linux.rocm.gpu.2}, {config: distributed, shard: 1, num_shards: 1, runner: linux.rocm.gpu.4}]} 2025-07-24T04:23:57.0629938Z 2025-07-24T04:23:57.0630041Z Is the current job unstable? False 2025-07-24T04:23:57.0630223Z 2025-07-24T04:23:57.0630319Z Is keep-going label set? False 2025-07-24T04:23:57.0630486Z 2025-07-24T04:23:57.0630754Z Reenabled issues? 2025-07-24T04:23:57.0671126Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2025-07-24T04:23:57.0671613Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2025-07-24T04:23:57.0700763Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:23:57.0701119Z env: 2025-07-24T04:23:57.0701331Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:57.0701725Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:57.0702276Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:57.0702794Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:57.0703631Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:57.0704380Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:57.0704671Z AWS_REGION: us-east-1 2025-07-24T04:23:57.0704955Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:57.0705288Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:57.0710348Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:57.0710579Z JOB_TIMEOUT: 300 2025-07-24T04:23:57.0710785Z ##[endgroup] 2025-07-24T04:23:57.0810700Z ##[group]Run set -x 2025-07-24T04:23:57.0811001Z set -x 2025-07-24T04:23:57.0811229Z  2025-07-24T04:23:57.0811453Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2025-07-24T04:23:57.0811805Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2025-07-24T04:23:57.0812150Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2025-07-24T04:23:57.0812469Z  TEST_COMMAND=.ci/caffe2/test.sh 2025-07-24T04:23:57.0812732Z else 2025-07-24T04:23:57.0812949Z  TEST_COMMAND=.ci/pytorch/test.sh 2025-07-24T04:23:57.0813207Z fi 2025-07-24T04:23:57.0813379Z  2025-07-24T04:23:57.0813673Z # detached container should get cleaned up by teardown_ec2_linux 2025-07-24T04:23:57.0814127Z # TODO: Stop building test binaries as part of the build phase 2025-07-24T04:23:57.0814520Z # Used for GPU_FLAG since that doesn't play nice 2025-07-24T04:23:57.0814883Z # shellcheck disable=SC2086,SC2090 2025-07-24T04:23:57.0815175Z container_name=$(docker run \ 2025-07-24T04:23:57.0815679Z  ${GPU_FLAG:-} \ 2025-07-24T04:23:57.0815926Z  -e BUILD_ENVIRONMENT \ 2025-07-24T04:23:57.0816178Z  -e PR_NUMBER \ 2025-07-24T04:23:57.0816415Z  -e GITHUB_ACTIONS \ 2025-07-24T04:23:57.0816665Z  -e GITHUB_REPOSITORY \ 2025-07-24T04:23:57.0816920Z  -e GITHUB_WORKFLOW \ 2025-07-24T04:23:57.0817166Z  -e GITHUB_JOB \ 2025-07-24T04:23:57.0817397Z  -e GITHUB_RUN_ID \ 2025-07-24T04:23:57.0817639Z  -e GITHUB_RUN_NUMBER \ 2025-07-24T04:23:57.0817892Z  -e GITHUB_RUN_ATTEMPT \ 2025-07-24T04:23:57.0818143Z  -e JOB_ID \ 2025-07-24T04:23:57.0818366Z  -e JOB_NAME \ 2025-07-24T04:23:57.0818589Z  -e BRANCH \ 2025-07-24T04:23:57.0818794Z  -e SHA1 \ 2025-07-24T04:23:57.0819013Z  -e AWS_DEFAULT_REGION \ 2025-07-24T04:23:57.0819268Z  -e IN_WHEEL_TEST \ 2025-07-24T04:23:57.0819507Z  -e SHARD_NUMBER \ 2025-07-24T04:23:57.0819742Z  -e TEST_CONFIG \ 2025-07-24T04:23:57.0819982Z  -e NUM_TEST_SHARDS \ 2025-07-24T04:23:57.0820231Z  -e REENABLED_ISSUES \ 2025-07-24T04:23:57.0820492Z  -e CONTINUE_THROUGH_ERROR \ 2025-07-24T04:23:57.0820754Z  -e VERBOSE_TEST_LOGS \ 2025-07-24T04:23:57.0821001Z  -e TEST_SHOWLOCALS \ 2025-07-24T04:23:57.0821245Z  -e NO_TEST_TIMEOUT \ 2025-07-24T04:23:57.0821470Z  -e NO_TD \ 2025-07-24T04:23:57.0821717Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2025-07-24T04:23:57.0822029Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2025-07-24T04:23:57.0822342Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2025-07-24T04:23:57.0822646Z  -e TESTS_TO_INCLUDE \ 2025-07-24T04:23:57.0822899Z  -e DASHBOARD_TAG \ 2025-07-24T04:23:57.0823221Z  --env-file="${RUNNER_TEMP}/github_env_${GITHUB_RUN_ID}" \ 2025-07-24T04:23:57.0823573Z  --ulimit stack=10485760:83886080 \ 2025-07-24T04:23:57.0823848Z  --ulimit core=0 \ 2025-07-24T04:23:57.0824113Z  --security-opt seccomp=unconfined \ 2025-07-24T04:23:57.0824402Z  --cap-add=SYS_PTRACE \ 2025-07-24T04:23:57.0824655Z  --shm-size="8g" \ 2025-07-24T04:23:57.0824885Z  --tty \ 2025-07-24T04:23:57.0825084Z  --detach \ 2025-07-24T04:23:57.0825312Z  --name="${container_name}" \ 2025-07-24T04:23:57.0825579Z  --user jenkins \ 2025-07-24T04:23:57.0825876Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2025-07-24T04:23:57.0826215Z  -w /var/lib/jenkins/workspace \ 2025-07-24T04:23:57.0826487Z  "${DOCKER_IMAGE}" 2025-07-24T04:23:57.0826716Z ) 2025-07-24T04:23:57.0826939Z # save container name for later step 2025-07-24T04:23:57.0827488Z echo "CONTAINER_NAME=${container_name}" >> "$GITHUB_ENV" 2025-07-24T04:23:57.0828119Z # jenkins user does not have write permission to mounted workspace; work-around by copying within container to jenkins home 2025-07-24T04:23:57.0828917Z docker exec -t "${container_name}" sh -c "cd .. && cp -R workspace pytorch && cd pytorch && pip install dist/*.whl && ${TEST_COMMAND}" 2025-07-24T04:23:57.0855312Z shell: /usr/bin/bash -e {0} 2025-07-24T04:23:57.0855566Z env: 2025-07-24T04:23:57.0855765Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:23:57.0856152Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:23:57.0856707Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:23:57.0857208Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:23:57.0858049Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:23:57.0858800Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:23:57.0859231Z AWS_REGION: us-east-1 2025-07-24T04:23:57.0859502Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:23:57.0859829Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:23:57.0864867Z AWS_SESSION_TOKEN: *** 2025-07-24T04:23:57.0865136Z BUILD_ENVIRONMENT: linux-jammy-rocm-py3.10 2025-07-24T04:23:57.0865418Z PR_NUMBER: 2025-07-24T04:23:57.0865634Z GITHUB_REPOSITORY: pytorch/pytorch 2025-07-24T04:23:57.0865898Z GITHUB_WORKFLOW: trunk 2025-07-24T04:23:57.0866115Z GITHUB_JOB: test 2025-07-24T04:23:57.0866324Z GITHUB_RUN_ID: 16487210679 2025-07-24T04:23:57.0866556Z GITHUB_RUN_NUMBER: 137322 2025-07-24T04:23:57.0866777Z GITHUB_RUN_ATTEMPT: 1 2025-07-24T04:23:57.0866992Z JOB_ID: 46615439117 2025-07-24T04:23:57.0867321Z JOB_NAME: linux-jammy-rocm-py3.10 / test (default, 1, 2, linux.rocm.gpu.2) 2025-07-24T04:23:57.0867691Z BRANCH: 2025-07-24T04:23:57.0867913Z SHA1: 27c8ef1839e0747b36769efb192befc9308ff93c 2025-07-24T04:23:57.0868203Z CONTINUE_THROUGH_ERROR: False 2025-07-24T04:23:57.0868454Z VERBOSE_TEST_LOGS: False 2025-07-24T04:23:57.0868684Z TEST_SHOWLOCALS: False 2025-07-24T04:23:57.0868905Z NO_TEST_TIMEOUT: False 2025-07-24T04:23:57.0869116Z NO_TD: False 2025-07-24T04:23:57.0869310Z TEST_CONFIG: default 2025-07-24T04:23:57.0869516Z SHARD_NUMBER: 1 2025-07-24T04:23:57.0869714Z NUM_TEST_SHARDS: 2 2025-07-24T04:23:57.0869922Z REENABLED_ISSUES: 2025-07-24T04:23:57.0870532Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f 2025-07-24T04:23:57.0871199Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2025-07-24T04:23:57.0871475Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2025-07-24T04:23:57.0872198Z TESTS_TO_INCLUDE: test_nn test_torch test_cuda test_ops test_unary_ufuncs test_binary_ufuncs test_autograd inductor/test_torchinductor distributed/test_c10d_common distributed/test_c10d_nccl 2025-07-24T04:23:57.0872904Z DASHBOARD_TAG: 2025-07-24T04:23:57.0873104Z ##[endgroup] 2025-07-24T04:23:57.0923374Z + [[ default == \m\u\l\t\i\g\p\u ]] 2025-07-24T04:23:57.0923695Z + [[ linux-jammy-rocm-py3.10 == *onnx* ]] 2025-07-24T04:23:57.0924000Z + TEST_COMMAND=.ci/pytorch/test.sh 2025-07-24T04:23:57.0939373Z +++ nproc --ignore=2 2025-07-24T04:23:57.0964989Z ++ docker run --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e MAX_JOBS=126 -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e TESTS_TO_INCLUDE -e DASHBOARD_TAG --env-file=/var/home/pytorchci/actions-runner/_work/_temp/github_env_16487210679 --ulimit stack=10485760:83886080 --ulimit core=0 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --shm-size=8g --tty --detach --name= --user jenkins -v /var/home/pytorchci/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f 2025-07-24T04:23:57.2366575Z + container_name=a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:23:57.2367650Z + echo CONTAINER_NAME=a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:23:57.2369303Z + docker exec -t a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 sh -c 'cd .. && cp -R workspace pytorch && cd pytorch && pip install dist/*.whl && .ci/pytorch/test.sh' 2025-07-24T04:24:15.8981427Z Processing ./dist/torch-2.9.0a0+git27c8ef1-cp310-cp310-linux_x86_64.whl 2025-07-24T04:24:16.8166037Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.9.0a0+git27c8ef1) (3.18.0) 2025-07-24T04:24:16.8170629Z Requirement already satisfied: typing-extensions>=4.10.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.9.0a0+git27c8ef1) (4.14.1) 2025-07-24T04:24:16.8174003Z Requirement already satisfied: sympy>=1.13.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.9.0a0+git27c8ef1) (1.13.3) 2025-07-24T04:24:16.8178388Z Requirement already satisfied: networkx>=2.5.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.9.0a0+git27c8ef1) (2.8.8) 2025-07-24T04:24:16.8181464Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.9.0a0+git27c8ef1) (3.1.6) 2025-07-24T04:24:16.8185881Z Requirement already satisfied: fsspec>=0.8.5 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.9.0a0+git27c8ef1) (2025.5.1) 2025-07-24T04:24:16.8552800Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy>=1.13.3->torch==2.9.0a0+git27c8ef1) (1.3.0) 2025-07-24T04:24:16.8606151Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.9.0a0+git27c8ef1) (3.0.2) 2025-07-24T04:24:17.5123655Z Installing collected packages: torch 2025-07-24T04:24:30.6306118Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2025-07-24T04:24:30.6307531Z helion 0.0.10 requires filecheck, which is not installed. 2025-07-24T04:24:30.6308343Z timm 1.0.14 requires torchvision, which is not installed. 2025-07-24T04:24:30.6309090Z Successfully installed torch-2.9.0a0+git27c8ef1 2025-07-24T04:24:30.6951807Z + export TERM=vt100 2025-07-24T04:24:30.6952166Z + TERM=vt100 2025-07-24T04:24:30.6958789Z ++ dirname .ci/pytorch/test.sh 2025-07-24T04:24:30.6976792Z + source .ci/pytorch/common.sh 2025-07-24T04:24:30.6985290Z +++ dirname .ci/pytorch/common.sh 2025-07-24T04:24:30.7000851Z ++ source .ci/pytorch/common_utils.sh 2025-07-24T04:24:30.7003284Z +++ declare -f -t trap_add 2025-07-24T04:24:30.7008151Z ++ set -ex -o pipefail 2025-07-24T04:24:30.7008581Z ++ [[ linux-jammy-rocm-py3.10 == *rocm* ]] 2025-07-24T04:24:30.7009065Z ++ unset HIP_PLATFORM 2025-07-24T04:24:30.7009465Z ++ export PYTORCH_TEST_WITH_ROCM=1 2025-07-24T04:24:30.7009911Z ++ PYTORCH_TEST_WITH_ROCM=1 2025-07-24T04:24:30.7010307Z ++ BUILD_TEST_LIBTORCH=0 2025-07-24T04:24:30.7017316Z ++ dirname .ci/pytorch/test.sh 2025-07-24T04:24:30.7032830Z + source .ci/pytorch/common-build.sh 2025-07-24T04:24:30.7035242Z ++ [[ linux-jammy-rocm-py3.10 != *win-* ]] 2025-07-24T04:24:30.7047546Z ++++ dirname .ci/pytorch/common-build.sh 2025-07-24T04:24:30.7061106Z +++ cd .ci/pytorch 2025-07-24T04:24:30.7061356Z +++ pwd -P 2025-07-24T04:24:30.7064612Z ++ script_dir=/var/lib/jenkins/pytorch/.ci/pytorch 2025-07-24T04:24:30.7065275Z ++ [[ linux-jammy-rocm-py3.10 == *-pch* ]] 2025-07-24T04:24:30.7065746Z ++ which sccache 2025-07-24T04:24:30.7085767Z ++ [[ -z '' ]] 2025-07-24T04:24:30.7086118Z ++ unset SCCACHE_BUCKET 2025-07-24T04:24:30.7086489Z ++ unset SCCACHE_REGION 2025-07-24T04:24:30.7086867Z ++ sccache --stop-server 2025-07-24T04:24:30.7122680Z ++ true 2025-07-24T04:24:30.7123046Z ++ rm -f /var/lib/jenkins/sccache_error.log 2025-07-24T04:24:30.7143680Z ++ trap_add sccache_epilogue EXIT 2025-07-24T04:24:30.7144160Z ++ trap_add_cmd=sccache_epilogue 2025-07-24T04:24:30.7144562Z ++ shift 2025-07-24T04:24:30.7144892Z ++ for trap_add_name in "$@" 2025-07-24T04:24:30.7158513Z ++++ trap -p EXIT 2025-07-24T04:24:30.7162295Z +++ eval 'extract_trap_cmd ' 2025-07-24T04:24:30.7162665Z ++++ extract_trap_cmd 2025-07-24T04:24:30.7162939Z ++++ printf '%s\n' '' 2025-07-24T04:24:30.7163216Z +++ printf '%s\n' sccache_epilogue 2025-07-24T04:24:30.7166671Z ++ trap -- ' 2025-07-24T04:24:30.7167166Z sccache_epilogue' EXIT 2025-07-24T04:24:30.7167443Z ++ [[ -n '' ]] 2025-07-24T04:24:30.7167698Z ++ [[ linux-jammy-rocm-py3.10 == *rocm* ]] 2025-07-24T04:24:30.7168094Z ++ SCCACHE_ERROR_LOG=/var/lib/jenkins/sccache_error.log 2025-07-24T04:24:30.7168461Z ++ SCCACHE_IDLE_TIMEOUT=0 2025-07-24T04:24:30.7168733Z ++ sccache --start-server 2025-07-24T04:24:30.7198644Z sccache: Starting the server... 2025-07-24T04:24:30.7565458Z sccache: Listening on address 127.0.0.1:4226 2025-07-24T04:24:30.7579410Z ++ sccache --zero-stats 2025-07-24T04:24:30.7609841Z Statistics zeroed. 2025-07-24T04:24:30.7614916Z ++ which ccache 2025-07-24T04:24:30.7630864Z + [[ linux-jammy-rocm-py3.10 != *rocm* ]] 2025-07-24T04:24:30.7631157Z + echo 'Environment variables:' 2025-07-24T04:24:30.7631401Z Environment variables: 2025-07-24T04:24:30.7631608Z + env 2025-07-24T04:24:30.7645193Z GITHUB_WORKSPACE=/var/home/pytorchci/actions-runner/_work/pytorch/pytorch 2025-07-24T04:24:30.7645644Z CONTINUE_THROUGH_ERROR=False 2025-07-24T04:24:30.7645930Z BUILD_ENVIRONMENT=linux-jammy-rocm-py3.10 2025-07-24T04:24:30.7646237Z HOSTNAME=gpud4f2.jax.cs.cpe.ice.amd.com 2025-07-24T04:24:30.7646825Z GITHUB_PATH=/var/home/pytorchci/actions-runner/_work/_temp/_runner_file_commands/add_path_bb5f7a7f-8e1a-4721-9501-cab91cca98e7 2025-07-24T04:24:30.7647385Z GITHUB_ACTION=__self 2025-07-24T04:24:30.7647614Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2025-07-24T04:24:30.7647884Z GITHUB_RUN_NUMBER=137322 2025-07-24T04:24:30.7648106Z TEST_CONFIG=default 2025-07-24T04:24:30.7648328Z GITHUB_REPOSITORY_OWNER_ID=21003710 2025-07-24T04:24:30.7648587Z AWS_DEFAULT_REGION=us-east-1 2025-07-24T04:24:30.7648847Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2025-07-24T04:24:30.7649120Z GITHUB_REF_TYPE=tag 2025-07-24T04:24:30.7649540Z *** 2025-07-24T04:24:30.7649736Z GITHUB_REPOSITORY_ID=65600975 2025-07-24T04:24:30.7649991Z GITHUB_ACTIONS=true 2025-07-24T04:24:30.7650241Z SHA1=27c8ef1839e0747b36769efb192befc9308ff93c 2025-07-24T04:24:30.7650587Z GITHUB_SHA=27c8ef1839e0747b36769efb192befc9308ff93c 2025-07-24T04:24:30.7651091Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/trunk.yml@refs/tags/ciflow/trunk/149961 2025-07-24T04:24:30.7651542Z UCC_HOME=/usr 2025-07-24T04:24:30.7651750Z VERBOSE_TEST_LOGS=False 2025-07-24T04:24:30.7651992Z GITHUB_REF=refs/tags/ciflow/trunk/149961 2025-07-24T04:24:30.7652253Z SHARD_NUMBER=1 2025-07-24T04:24:30.7652469Z GITHUB_REF_PROTECTED=false 2025-07-24T04:24:30.7652708Z HOME=/var/lib/jenkins 2025-07-24T04:24:30.7652978Z GITHUB_API_URL=https://api.github.com 2025-07-24T04:24:30.7653289Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2025-07-24T04:24:30.7653574Z LANG=C.UTF-8 2025-07-24T04:24:30.7653839Z UCX_COMMIT=cc312eaa4655c0cc5c2bcd796db938f90563bcf6 2025-07-24T04:24:30.7654181Z PYTORCH_TEST_WITH_ROCM=1 2025-07-24T04:24:30.7654436Z NUM_TEST_SHARDS=2 2025-07-24T04:24:30.7654659Z UCX_HOME=/usr 2025-07-24T04:24:30.7655428Z GITHUB_STATE=/var/home/pytorchci/actions-runner/_work/_temp/_runner_file_commands/save_state_bb5f7a7f-8e1a-4721-9501-cab91cca98e7 2025-07-24T04:24:30.7656220Z JOB_NAME=linux-jammy-rocm-py3.10 / test (default, 1, 2, linux.rocm.gpu.2) 2025-07-24T04:24:30.7656657Z MAGMA_HOME=/opt/rocm/magma 2025-07-24T04:24:30.7657268Z GITHUB_ENV=/var/home/pytorchci/actions-runner/_work/_temp/_runner_file_commands/set_env_bb5f7a7f-8e1a-4721-9501-cab91cca98e7 2025-07-24T04:24:30.7658103Z GITHUB_EVENT_PATH=/var/home/pytorchci/actions-runner/_work/_temp/_github_workflow/event.json 2025-07-24T04:24:30.7658611Z GITHUB_EVENT_NAME=push 2025-07-24T04:24:30.7658843Z DASHBOARD_TAG= 2025-07-24T04:24:30.7659059Z GITHUB_RUN_ID=16487210679 2025-07-24T04:24:30.7659695Z GITHUB_STEP_SUMMARY=/var/home/pytorchci/actions-runner/_work/_temp/_runner_file_commands/step_summary_bb5f7a7f-8e1a-4721-9501-cab91cca98e7 2025-07-24T04:24:30.7660408Z GITHUB_ACTOR=pytorch-bot[bot] 2025-07-24T04:24:30.7660672Z PR_NUMBER= 2025-07-24T04:24:30.7660883Z GITHUB_RUN_ATTEMPT=1 2025-07-24T04:24:30.7661129Z ANACONDA_PYTHON_VERSION=3.10 2025-07-24T04:24:30.7661437Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2025-07-24T04:24:30.7662104Z TERM=vt100 2025-07-24T04:24:30.7662308Z INSTALLED_VISION=yes 2025-07-24T04:24:30.7662527Z BRANCH= 2025-07-24T04:24:30.7662739Z OPENSSL_ROOT_DIR=/opt/openssl 2025-07-24T04:24:30.7663559Z TESTS_TO_INCLUDE=test_nn test_torch test_cuda test_ops test_unary_ufuncs test_binary_ufuncs test_autograd inductor/test_torchinductor distributed/test_c10d_common distributed/test_c10d_nccl 2025-07-24T04:24:30.7664562Z GITHUB_ACTION_PATH=/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-rocm 2025-07-24T04:24:30.7665069Z GITHUB_SERVER_URL=https://github.com 2025-07-24T04:24:30.7665344Z PYTORCH_ROCM_ARCH=gfx90a;gfx942 2025-07-24T04:24:30.7665622Z UCC_COMMIT=0c0fc21559835044ab107199e334f7157d6a0d3d 2025-07-24T04:24:30.7665909Z REENABLED_ISSUES= 2025-07-24T04:24:30.7666103Z SHLVL=1 2025-07-24T04:24:30.7666277Z MAX_JOBS=126 2025-07-24T04:24:30.7666500Z GITHUB_ACTOR_ID=54816060 2025-07-24T04:24:30.7666806Z GITHUB_WORKFLOW_SHA=27c8ef1839e0747b36769efb192befc9308ff93c 2025-07-24T04:24:30.7667156Z GITHUB_REF_NAME=ciflow/trunk/149961 2025-07-24T04:24:30.7667419Z ROCM_PATH=/opt/rocm 2025-07-24T04:24:30.7667629Z GITHUB_JOB=test 2025-07-24T04:24:30.7667844Z NO_TEST_TIMEOUT=False 2025-07-24T04:24:30.7668082Z GITHUB_REPOSITORY=pytorch/pytorch 2025-07-24T04:24:30.7668331Z LC_ALL=C.UTF-8 2025-07-24T04:24:30.7668540Z GITHUB_RETENTION_DAYS=90 2025-07-24T04:24:30.7668779Z OPENSSL_DIR=/opt/openssl 2025-07-24T04:24:30.7669011Z GITHUB_ACTION_REPOSITORY= 2025-07-24T04:24:30.7669818Z PATH=/opt/cache/bin:/opt/rocm/llvm/bin:/opt/rocm/opencl/bin:/opt/rocm/hip/bin:/opt/rocm/hcc/bin:/opt/rocm/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2025-07-24T04:24:30.7670639Z GITHUB_BASE_REF= 2025-07-24T04:24:30.7670836Z CI=true 2025-07-24T04:24:30.7671037Z GITHUB_REPOSITORY_OWNER=pytorch 2025-07-24T04:24:30.7671276Z JOB_ID=46615439117 2025-07-24T04:24:30.7671481Z GITHUB_HEAD_REF= 2025-07-24T04:24:30.7671680Z GITHUB_ACTION_REF= 2025-07-24T04:24:30.7671881Z TEST_SHOWLOCALS=False 2025-07-24T04:24:30.7672100Z GITHUB_WORKFLOW=trunk 2025-07-24T04:24:30.7672328Z DEBIAN_FRONTEND=noninteractive 2025-07-24T04:24:30.7672893Z GITHUB_OUTPUT=/var/home/pytorchci/actions-runner/_work/_temp/_runner_file_commands/set_output_bb5f7a7f-8e1a-4721-9501-cab91cca98e7 2025-07-24T04:24:30.7673465Z NO_TD=False 2025-07-24T04:24:30.7673654Z OLDPWD=/var/lib/jenkins 2025-07-24T04:24:30.7673858Z _=/usr/bin/env 2025-07-24T04:24:30.7674313Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2025-07-24T04:24:30.7794426Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2025-07-24T04:24:30.7796231Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2025-07-24T04:24:30.7797351Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2025-07-24T04:24:30.7809841Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2025-07-24T04:24:30.7810495Z + BUILD_DIR=build 2025-07-24T04:24:30.7810879Z + BUILD_RENAMED_DIR=build_renamed 2025-07-24T04:24:30.7824295Z + BUILD_BIN_DIR=build/bin 2025-07-24T04:24:30.7824650Z + SHARD_NUMBER=1 2025-07-24T04:24:30.7824877Z + NUM_TEST_SHARDS=2 2025-07-24T04:24:30.7825115Z + export TORCH_SERIALIZATION_DEBUG=1 2025-07-24T04:24:30.7825411Z + TORCH_SERIALIZATION_DEBUG=1 2025-07-24T04:24:30.7825660Z + export VALGRIND=ON 2025-07-24T04:24:30.7825876Z + VALGRIND=ON 2025-07-24T04:24:30.7826103Z + [[ linux-jammy-rocm-py3.10 == *clang9* ]] 2025-07-24T04:24:30.7826413Z + [[ linux-jammy-rocm-py3.10 == *xpu* ]] 2025-07-24T04:24:30.7826717Z + [[ linux-jammy-rocm-py3.10 == *s390x* ]] 2025-07-24T04:24:30.7826995Z + [[ 0 == \1 ]] 2025-07-24T04:24:30.7827192Z + [[ False == \1 ]] 2025-07-24T04:24:30.7827418Z + [[ linux-jammy-rocm-py3.10 != *bazel* ]] 2025-07-24T04:24:30.7827700Z ++ realpath build/custom_test_artifacts 2025-07-24T04:24:30.7828130Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/pytorch/build/custom_test_artifacts 2025-07-24T04:24:30.7828991Z + [[ -n test_nn test_torch test_cuda test_ops test_unary_ufuncs test_binary_ufuncs test_autograd inductor/test_torchinductor distributed/test_c10d_common distributed/test_c10d_nccl ]] 2025-07-24T04:24:30.7829964Z + echo 'Setting INCLUDE_CLAUSE' 2025-07-24T04:24:30.7830223Z Setting INCLUDE_CLAUSE 2025-07-24T04:24:30.7830951Z + INCLUDE_CLAUSE='--include test_nn test_torch test_cuda test_ops test_unary_ufuncs test_binary_ufuncs test_autograd inductor/test_torchinductor distributed/test_c10d_common distributed/test_c10d_nccl' 2025-07-24T04:24:30.7831726Z + echo 'Environment variables' 2025-07-24T04:24:30.7831974Z Environment variables 2025-07-24T04:24:30.7832181Z + env 2025-07-24T04:24:30.7839048Z GITHUB_WORKSPACE=/var/home/pytorchci/actions-runner/_work/pytorch/pytorch 2025-07-24T04:24:30.7839468Z CONTINUE_THROUGH_ERROR=False 2025-07-24T04:24:30.7839753Z BUILD_ENVIRONMENT=linux-jammy-rocm-py3.10 2025-07-24T04:24:30.7840076Z HOSTNAME=gpud4f2.jax.cs.cpe.ice.amd.com 2025-07-24T04:24:30.7840723Z GITHUB_PATH=/var/home/pytorchci/actions-runner/_work/_temp/_runner_file_commands/add_path_bb5f7a7f-8e1a-4721-9501-cab91cca98e7 2025-07-24T04:24:30.7841280Z GITHUB_ACTION=__self 2025-07-24T04:24:30.7841513Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2025-07-24T04:24:30.7841772Z GITHUB_RUN_NUMBER=137322 2025-07-24T04:24:30.7841997Z TEST_CONFIG=default 2025-07-24T04:24:30.7842212Z GITHUB_REPOSITORY_OWNER_ID=21003710 2025-07-24T04:24:30.7842587Z AWS_DEFAULT_REGION=us-east-1 2025-07-24T04:24:30.7842858Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2025-07-24T04:24:30.7843131Z GITHUB_REF_TYPE=tag 2025-07-24T04:24:30.7843395Z *** 2025-07-24T04:24:30.7843597Z GITHUB_REPOSITORY_ID=65600975 2025-07-24T04:24:30.7843859Z GITHUB_ACTIONS=true 2025-07-24T04:24:30.7844105Z SHA1=27c8ef1839e0747b36769efb192befc9308ff93c 2025-07-24T04:24:30.7844426Z GITHUB_SHA=27c8ef1839e0747b36769efb192befc9308ff93c 2025-07-24T04:24:30.7844920Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/trunk.yml@refs/tags/ciflow/trunk/149961 2025-07-24T04:24:30.7845360Z UCC_HOME=/usr 2025-07-24T04:24:30.7845570Z TORCH_SERIALIZATION_DEBUG=1 2025-07-24T04:24:30.7845803Z VERBOSE_TEST_LOGS=False 2025-07-24T04:24:30.7846047Z GITHUB_REF=refs/tags/ciflow/trunk/149961 2025-07-24T04:24:30.7846314Z SHARD_NUMBER=1 2025-07-24T04:24:30.7846520Z GITHUB_REF_PROTECTED=false 2025-07-24T04:24:30.7846751Z HOME=/var/lib/jenkins 2025-07-24T04:24:30.7846999Z GITHUB_API_URL=https://api.github.com 2025-07-24T04:24:30.7847292Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2025-07-24T04:24:30.7847543Z LANG=C.UTF-8 2025-07-24T04:24:30.7847787Z UCX_COMMIT=cc312eaa4655c0cc5c2bcd796db938f90563bcf6 2025-07-24T04:24:30.7848098Z PYTORCH_TEST_WITH_ROCM=1 2025-07-24T04:24:30.7848318Z NUM_TEST_SHARDS=2 2025-07-24T04:24:30.7848516Z UCX_HOME=/usr 2025-07-24T04:24:30.7849028Z GITHUB_STATE=/var/home/pytorchci/actions-runner/_work/_temp/_runner_file_commands/save_state_bb5f7a7f-8e1a-4721-9501-cab91cca98e7 2025-07-24T04:24:30.7849893Z JOB_NAME=linux-jammy-rocm-py3.10 / test (default, 1, 2, linux.rocm.gpu.2) 2025-07-24T04:24:30.7850288Z MAGMA_HOME=/opt/rocm/magma 2025-07-24T04:24:30.7850812Z GITHUB_ENV=/var/home/pytorchci/actions-runner/_work/_temp/_runner_file_commands/set_env_bb5f7a7f-8e1a-4721-9501-cab91cca98e7 2025-07-24T04:24:30.7851536Z GITHUB_EVENT_PATH=/var/home/pytorchci/actions-runner/_work/_temp/_github_workflow/event.json 2025-07-24T04:24:30.7851979Z GITHUB_EVENT_NAME=push 2025-07-24T04:24:30.7852194Z DASHBOARD_TAG= 2025-07-24T04:24:30.7852398Z GITHUB_RUN_ID=16487210679 2025-07-24T04:24:30.7852960Z GITHUB_STEP_SUMMARY=/var/home/pytorchci/actions-runner/_work/_temp/_runner_file_commands/step_summary_bb5f7a7f-8e1a-4721-9501-cab91cca98e7 2025-07-24T04:24:30.7853561Z GITHUB_ACTOR=pytorch-bot[bot] 2025-07-24T04:24:30.7853797Z PR_NUMBER= 2025-07-24T04:24:30.7853985Z GITHUB_RUN_ATTEMPT=1 2025-07-24T04:24:30.7854196Z VALGRIND=ON 2025-07-24T04:24:30.7854400Z ANACONDA_PYTHON_VERSION=3.10 2025-07-24T04:24:30.7854698Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2025-07-24T04:24:30.7855003Z TERM=vt100 2025-07-24T04:24:30.7855192Z INSTALLED_VISION=yes 2025-07-24T04:24:30.7855553Z BRANCH= 2025-07-24T04:24:30.7855748Z OPENSSL_ROOT_DIR=/opt/openssl 2025-07-24T04:24:30.7856457Z TESTS_TO_INCLUDE=test_nn test_torch test_cuda test_ops test_unary_ufuncs test_binary_ufuncs test_autograd inductor/test_torchinductor distributed/test_c10d_common distributed/test_c10d_nccl 2025-07-24T04:24:30.7857404Z GITHUB_ACTION_PATH=/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-rocm 2025-07-24T04:24:30.7857910Z GITHUB_SERVER_URL=https://github.com 2025-07-24T04:24:30.7858189Z PYTORCH_ROCM_ARCH=gfx90a;gfx942 2025-07-24T04:24:30.7858466Z UCC_COMMIT=0c0fc21559835044ab107199e334f7157d6a0d3d 2025-07-24T04:24:30.7858756Z REENABLED_ISSUES= 2025-07-24T04:24:30.7858954Z SHLVL=1 2025-07-24T04:24:30.7859131Z MAX_JOBS=126 2025-07-24T04:24:30.7859327Z GITHUB_ACTOR_ID=54816060 2025-07-24T04:24:30.7859628Z GITHUB_WORKFLOW_SHA=27c8ef1839e0747b36769efb192befc9308ff93c 2025-07-24T04:24:30.7859973Z GITHUB_REF_NAME=ciflow/trunk/149961 2025-07-24T04:24:30.7860230Z ROCM_PATH=/opt/rocm 2025-07-24T04:24:30.7860445Z GITHUB_JOB=test 2025-07-24T04:24:30.7860650Z NO_TEST_TIMEOUT=False 2025-07-24T04:24:30.7860880Z GITHUB_REPOSITORY=pytorch/pytorch 2025-07-24T04:24:30.7861136Z LC_ALL=C.UTF-8 2025-07-24T04:24:30.7861339Z GITHUB_RETENTION_DAYS=90 2025-07-24T04:24:30.7861571Z OPENSSL_DIR=/opt/openssl 2025-07-24T04:24:30.7861807Z GITHUB_ACTION_REPOSITORY= 2025-07-24T04:24:30.7862607Z PATH=/opt/cache/bin:/opt/rocm/llvm/bin:/opt/rocm/opencl/bin:/opt/rocm/hip/bin:/opt/rocm/hcc/bin:/opt/rocm/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2025-07-24T04:24:30.7863416Z GITHUB_BASE_REF= 2025-07-24T04:24:30.7863618Z CI=true 2025-07-24T04:24:30.7863814Z GITHUB_REPOSITORY_OWNER=pytorch 2025-07-24T04:24:30.7864056Z JOB_ID=46615439117 2025-07-24T04:24:30.7864255Z GITHUB_HEAD_REF= 2025-07-24T04:24:30.7864487Z GITHUB_ACTION_REF= 2025-07-24T04:24:30.7864685Z TEST_SHOWLOCALS=False 2025-07-24T04:24:30.7864899Z GITHUB_WORKFLOW=trunk 2025-07-24T04:24:30.7865130Z DEBIAN_FRONTEND=noninteractive 2025-07-24T04:24:30.7865688Z GITHUB_OUTPUT=/var/home/pytorchci/actions-runner/_work/_temp/_runner_file_commands/set_output_bb5f7a7f-8e1a-4721-9501-cab91cca98e7 2025-07-24T04:24:30.7866244Z NO_TD=False 2025-07-24T04:24:30.7866438Z OLDPWD=/var/lib/jenkins 2025-07-24T04:24:30.7866654Z _=/usr/bin/env 2025-07-24T04:24:30.7866855Z + echo 'Testing pytorch' 2025-07-24T04:24:30.7867075Z Testing pytorch 2025-07-24T04:24:30.7867274Z + export LANG=C.UTF-8 2025-07-24T04:24:30.7867471Z + LANG=C.UTF-8 2025-07-24T04:24:30.7867661Z + PR_NUMBER= 2025-07-24T04:24:30.7867862Z + [[ default == \d\e\f\a\u\l\t ]] 2025-07-24T04:24:30.7868119Z + export CUDA_VISIBLE_DEVICES=0 2025-07-24T04:24:30.7868363Z + CUDA_VISIBLE_DEVICES=0 2025-07-24T04:24:30.7868591Z + export HIP_VISIBLE_DEVICES=0 2025-07-24T04:24:30.7868839Z + HIP_VISIBLE_DEVICES=0 2025-07-24T04:24:30.7869203Z + [[ default == \d\i\s\t\r\i\b\u\t\e\d ]] 2025-07-24T04:24:30.7869479Z + [[ default == \s\l\o\w ]] 2025-07-24T04:24:30.7869758Z + [[ linux-jammy-rocm-py3.10 == *slow-gradcheck* ]] 2025-07-24T04:24:30.7870075Z + [[ linux-jammy-rocm-py3.10 == *cuda* ]] 2025-07-24T04:24:30.7870349Z + [[ linux-jammy-rocm-py3.10 == *rocm* ]] 2025-07-24T04:24:30.7870653Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2025-07-24T04:24:30.7870946Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2025-07-24T04:24:30.7871219Z + [[ default == *crossref* ]] 2025-07-24T04:24:30.7871476Z + [[ linux-jammy-rocm-py3.10 == *rocm* ]] 2025-07-24T04:24:30.7871741Z + export VALGRIND=OFF 2025-07-24T04:24:30.7871950Z + VALGRIND=OFF 2025-07-24T04:24:30.7872140Z + rocminfo 2025-07-24T04:24:30.7981631Z ROCk module version 6.10.5 is loaded 2025-07-24T04:24:31.0196565Z ===================== 2025-07-24T04:24:31.0197041Z HSA System Attributes 2025-07-24T04:24:31.0197446Z ===================== 2025-07-24T04:24:31.0197841Z Runtime Version: 1.15 2025-07-24T04:24:31.0198257Z Runtime Ext Version: 1.7 2025-07-24T04:24:31.0198692Z System Timestamp Freq.: 1000.000000MHz 2025-07-24T04:24:31.0199794Z Sig. Max Wait Duration: 18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count) 2025-07-24T04:24:31.0200587Z Machine Model: LARGE 2025-07-24T04:24:31.0201231Z System Endianness: LITTLE 2025-07-24T04:24:31.0201810Z Mwaitx: DISABLED 2025-07-24T04:24:31.0202236Z XNACK enabled: NO 2025-07-24T04:24:31.0202909Z DMAbuf Support: YES 2025-07-24T04:24:31.0203316Z VMM Support: YES 2025-07-24T04:24:31.0203581Z 2025-07-24T04:24:31.0203715Z ========== 2025-07-24T04:24:31.0204092Z HSA Agents 2025-07-24T04:24:31.0204456Z ========== 2025-07-24T04:24:31.0204802Z ******* 2025-07-24T04:24:31.0205159Z Agent 1 2025-07-24T04:24:31.0205524Z ******* 2025-07-24T04:24:31.0205984Z Name: AMD EPYC 7713 64-Core Processor 2025-07-24T04:24:31.0206571Z Uuid: CPU-XX 2025-07-24T04:24:31.0207164Z Marketing Name: AMD EPYC 7713 64-Core Processor 2025-07-24T04:24:31.0207824Z Vendor Name: CPU 2025-07-24T04:24:31.0208661Z Feature: None specified 2025-07-24T04:24:31.0209383Z Profile: FULL_PROFILE 2025-07-24T04:24:31.0209858Z Float Round Mode: NEAR 2025-07-24T04:24:31.0210297Z Max Queue Number: 0(0x0) 2025-07-24T04:24:31.0210699Z Queue Min Size: 0(0x0) 2025-07-24T04:24:31.0211118Z Queue Max Size: 0(0x0) 2025-07-24T04:24:31.0211518Z Queue Type: MULTI 2025-07-24T04:24:31.0211902Z Node: 0 2025-07-24T04:24:31.0212279Z Device Type: CPU 2025-07-24T04:24:31.0212653Z Cache Info: 2025-07-24T04:24:31.0212944Z L1: 32768(0x8000) KB 2025-07-24T04:24:31.0213307Z Chip ID: 0(0x0) 2025-07-24T04:24:31.0213698Z ASIC Revision: 0(0x0) 2025-07-24T04:24:31.0214113Z Cacheline Size: 64(0x40) 2025-07-24T04:24:31.0214546Z Max Clock Freq. (MHz): 2000 2025-07-24T04:24:31.0214935Z BDFID: 0 2025-07-24T04:24:31.0215338Z Internal Node ID: 0 2025-07-24T04:24:31.0215749Z Compute Unit: 64 2025-07-24T04:24:31.0216336Z SIMDs per CU: 0 2025-07-24T04:24:31.0216755Z Shader Engines: 0 2025-07-24T04:24:31.0217184Z Shader Arrs. per Eng.: 0 2025-07-24T04:24:31.0217622Z WatchPts on Addr. Ranges:1 2025-07-24T04:24:31.0217998Z Memory Properties: 2025-07-24T04:24:31.0218288Z Features: None 2025-07-24T04:24:31.0218553Z Pool Info: 2025-07-24T04:24:31.0218794Z Pool 1 2025-07-24T04:24:31.0219092Z Segment: GLOBAL; FLAGS: FINE GRAINED 2025-07-24T04:24:31.0219444Z Size: 528249788(0x1f7c73bc) KB 2025-07-24T04:24:31.0219785Z Allocatable: TRUE 2025-07-24T04:24:31.0220148Z Alloc Granule: 4KB 2025-07-24T04:24:31.0220513Z Alloc Recommended Granule:4KB 2025-07-24T04:24:31.0220882Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0221404Z Accessible by all: TRUE 2025-07-24T04:24:31.0221709Z Pool 2 2025-07-24T04:24:31.0221988Z Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED 2025-07-24T04:24:31.0222321Z Size: 528249788(0x1f7c73bc) KB 2025-07-24T04:24:31.0222647Z Allocatable: TRUE 2025-07-24T04:24:31.0222987Z Alloc Granule: 4KB 2025-07-24T04:24:31.0223358Z Alloc Recommended Granule:4KB 2025-07-24T04:24:31.0223715Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0224071Z Accessible by all: TRUE 2025-07-24T04:24:31.0224379Z Pool 3 2025-07-24T04:24:31.0224665Z Segment: GLOBAL; FLAGS: KERNARG, FINE GRAINED 2025-07-24T04:24:31.0224996Z Size: 528249788(0x1f7c73bc) KB 2025-07-24T04:24:31.0225328Z Allocatable: TRUE 2025-07-24T04:24:31.0225740Z Alloc Granule: 4KB 2025-07-24T04:24:31.0226114Z Alloc Recommended Granule:4KB 2025-07-24T04:24:31.0226498Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0226857Z Accessible by all: TRUE 2025-07-24T04:24:31.0227181Z Pool 4 2025-07-24T04:24:31.0227477Z Segment: GLOBAL; FLAGS: COARSE GRAINED 2025-07-24T04:24:31.0227827Z Size: 528249788(0x1f7c73bc) KB 2025-07-24T04:24:31.0228168Z Allocatable: TRUE 2025-07-24T04:24:31.0228519Z Alloc Granule: 4KB 2025-07-24T04:24:31.0228890Z Alloc Recommended Granule:4KB 2025-07-24T04:24:31.0229275Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0229631Z Accessible by all: TRUE 2025-07-24T04:24:31.0229946Z ISA Info: 2025-07-24T04:24:31.0230184Z ******* 2025-07-24T04:24:31.0230408Z Agent 2 2025-07-24T04:24:31.0230623Z ******* 2025-07-24T04:24:31.0230878Z Name: AMD EPYC 7713 64-Core Processor 2025-07-24T04:24:31.0231214Z Uuid: CPU-XX 2025-07-24T04:24:31.0231568Z Marketing Name: AMD EPYC 7713 64-Core Processor 2025-07-24T04:24:31.0231935Z Vendor Name: CPU 2025-07-24T04:24:31.0232277Z Feature: None specified 2025-07-24T04:24:31.0232757Z Profile: FULL_PROFILE 2025-07-24T04:24:31.0233117Z Float Round Mode: NEAR 2025-07-24T04:24:31.0233470Z Max Queue Number: 0(0x0) 2025-07-24T04:24:31.0233826Z Queue Min Size: 0(0x0) 2025-07-24T04:24:31.0234159Z Queue Max Size: 0(0x0) 2025-07-24T04:24:31.0234490Z Queue Type: MULTI 2025-07-24T04:24:31.0234798Z Node: 1 2025-07-24T04:24:31.0235102Z Device Type: CPU 2025-07-24T04:24:31.0235399Z Cache Info: 2025-07-24T04:24:31.0235640Z L1: 32768(0x8000) KB 2025-07-24T04:24:31.0235962Z Chip ID: 0(0x0) 2025-07-24T04:24:31.0236291Z ASIC Revision: 0(0x0) 2025-07-24T04:24:31.0236640Z Cacheline Size: 64(0x40) 2025-07-24T04:24:31.0237118Z Max Clock Freq. (MHz): 2000 2025-07-24T04:24:31.0237438Z BDFID: 0 2025-07-24T04:24:31.0237766Z Internal Node ID: 1 2025-07-24T04:24:31.0238102Z Compute Unit: 64 2025-07-24T04:24:31.0238429Z SIMDs per CU: 0 2025-07-24T04:24:31.0238758Z Shader Engines: 0 2025-07-24T04:24:31.0239109Z Shader Arrs. per Eng.: 0 2025-07-24T04:24:31.0239462Z WatchPts on Addr. Ranges:1 2025-07-24T04:24:31.0239779Z Memory Properties: 2025-07-24T04:24:31.0240014Z Features: None 2025-07-24T04:24:31.0240259Z Pool Info: 2025-07-24T04:24:31.0240490Z Pool 1 2025-07-24T04:24:31.0240772Z Segment: GLOBAL; FLAGS: FINE GRAINED 2025-07-24T04:24:31.0241115Z Size: 528402436(0x1f7ec804) KB 2025-07-24T04:24:31.0241460Z Allocatable: TRUE 2025-07-24T04:24:31.0241807Z Alloc Granule: 4KB 2025-07-24T04:24:31.0242176Z Alloc Recommended Granule:4KB 2025-07-24T04:24:31.0242628Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0242990Z Accessible by all: TRUE 2025-07-24T04:24:31.0243300Z Pool 2 2025-07-24T04:24:31.0243584Z Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED 2025-07-24T04:24:31.0243915Z Size: 528402436(0x1f7ec804) KB 2025-07-24T04:24:31.0244240Z Allocatable: TRUE 2025-07-24T04:24:31.0244587Z Alloc Granule: 4KB 2025-07-24T04:24:31.0244954Z Alloc Recommended Granule:4KB 2025-07-24T04:24:31.0245319Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0245678Z Accessible by all: TRUE 2025-07-24T04:24:31.0245983Z Pool 3 2025-07-24T04:24:31.0246262Z Segment: GLOBAL; FLAGS: KERNARG, FINE GRAINED 2025-07-24T04:24:31.0246589Z Size: 528402436(0x1f7ec804) KB 2025-07-24T04:24:31.0246914Z Allocatable: TRUE 2025-07-24T04:24:31.0247258Z Alloc Granule: 4KB 2025-07-24T04:24:31.0247617Z Alloc Recommended Granule:4KB 2025-07-24T04:24:31.0248147Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0248501Z Accessible by all: TRUE 2025-07-24T04:24:31.0248810Z Pool 4 2025-07-24T04:24:31.0249080Z Segment: GLOBAL; FLAGS: COARSE GRAINED 2025-07-24T04:24:31.0249408Z Size: 528402436(0x1f7ec804) KB 2025-07-24T04:24:31.0249727Z Allocatable: TRUE 2025-07-24T04:24:31.0250073Z Alloc Granule: 4KB 2025-07-24T04:24:31.0250433Z Alloc Recommended Granule:4KB 2025-07-24T04:24:31.0250793Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0251142Z Accessible by all: TRUE 2025-07-24T04:24:31.0251444Z ISA Info: 2025-07-24T04:24:31.0251663Z ******* 2025-07-24T04:24:31.0251882Z Agent 3 2025-07-24T04:24:31.0252093Z ******* 2025-07-24T04:24:31.0252667Z Name: gfx90a 2025-07-24T04:24:31.0252981Z Uuid: GPU-0ca137a0ba8e063d 2025-07-24T04:24:31.0253338Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:24:31.0253697Z Vendor Name: AMD 2025-07-24T04:24:31.0254037Z Feature: KERNEL_DISPATCH 2025-07-24T04:24:31.0254379Z Profile: BASE_PROFILE 2025-07-24T04:24:31.0254738Z Float Round Mode: NEAR 2025-07-24T04:24:31.0255103Z Max Queue Number: 128(0x80) 2025-07-24T04:24:31.0255446Z Queue Min Size: 64(0x40) 2025-07-24T04:24:31.0255794Z Queue Max Size: 131072(0x20000) 2025-07-24T04:24:31.0256135Z Queue Type: MULTI 2025-07-24T04:24:31.0256460Z Node: 2 2025-07-24T04:24:31.0256782Z Device Type: GPU 2025-07-24T04:24:31.0257084Z Cache Info: 2025-07-24T04:24:31.0257331Z L1: 16(0x10) KB 2025-07-24T04:24:31.0257625Z L2: 8192(0x2000) KB 2025-07-24T04:24:31.0257927Z Chip ID: 29708(0x740c) 2025-07-24T04:24:31.0258255Z ASIC Revision: 1(0x1) 2025-07-24T04:24:31.0258597Z Cacheline Size: 128(0x80) 2025-07-24T04:24:31.0258949Z Max Clock Freq. (MHz): 1700 2025-07-24T04:24:31.0259278Z BDFID: 12800 2025-07-24T04:24:31.0259610Z Internal Node ID: 2 2025-07-24T04:24:31.0259949Z Compute Unit: 104 2025-07-24T04:24:31.0260280Z SIMDs per CU: 4 2025-07-24T04:24:31.0260615Z Shader Engines: 8 2025-07-24T04:24:31.0260969Z Shader Arrs. per Eng.: 1 2025-07-24T04:24:31.0261325Z WatchPts on Addr. Ranges:4 2025-07-24T04:24:31.0261686Z Coherent Host Access: FALSE 2025-07-24T04:24:31.0262005Z Memory Properties: 2025-07-24T04:24:31.0262267Z Features: KERNEL_DISPATCH 2025-07-24T04:24:31.0262591Z Fast F16 Operation: TRUE 2025-07-24T04:24:31.0262942Z Wavefront Size: 64(0x40) 2025-07-24T04:24:31.0263290Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:24:31.0263776Z Workgroup Max Size per Dimension: 2025-07-24T04:24:31.0264056Z x 1024(0x400) 2025-07-24T04:24:31.0264347Z y 1024(0x400) 2025-07-24T04:24:31.0264617Z z 1024(0x400) 2025-07-24T04:24:31.0264927Z Max Waves Per CU: 32(0x20) 2025-07-24T04:24:31.0265273Z Max Work-item Per CU: 2048(0x800) 2025-07-24T04:24:31.0265615Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:24:31.0265921Z Grid Max Size per Dimension: 2025-07-24T04:24:31.0266176Z x 4294967295(0xffffffff) 2025-07-24T04:24:31.0266461Z y 4294967295(0xffffffff) 2025-07-24T04:24:31.0266742Z z 4294967295(0xffffffff) 2025-07-24T04:24:31.0267073Z Max fbarriers/Workgrp: 32 2025-07-24T04:24:31.0267470Z Packet Processor uCode:: 92 2025-07-24T04:24:31.0267964Z SDMA engine uCode:: 9 2025-07-24T04:24:31.0268310Z IOMMU Support:: None 2025-07-24T04:24:31.0268638Z Pool Info: 2025-07-24T04:24:31.0268868Z Pool 1 2025-07-24T04:24:31.0269163Z Segment: GLOBAL; FLAGS: COARSE GRAINED 2025-07-24T04:24:31.0269509Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0269844Z Allocatable: TRUE 2025-07-24T04:24:31.0270189Z Alloc Granule: 4KB 2025-07-24T04:24:31.0270550Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0270915Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0271277Z Accessible by all: FALSE 2025-07-24T04:24:31.0271586Z Pool 2 2025-07-24T04:24:31.0271874Z Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED 2025-07-24T04:24:31.0272207Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0272535Z Allocatable: TRUE 2025-07-24T04:24:31.0272879Z Alloc Granule: 4KB 2025-07-24T04:24:31.0273245Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0273600Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0273952Z Accessible by all: FALSE 2025-07-24T04:24:31.0274257Z Pool 3 2025-07-24T04:24:31.0274671Z Segment: GLOBAL; FLAGS: FINE GRAINED 2025-07-24T04:24:31.0274999Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0275328Z Allocatable: TRUE 2025-07-24T04:24:31.0275673Z Alloc Granule: 4KB 2025-07-24T04:24:31.0276035Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0276396Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0276747Z Accessible by all: FALSE 2025-07-24T04:24:31.0277054Z Pool 4 2025-07-24T04:24:31.0277316Z Segment: GROUP 2025-07-24T04:24:31.0277629Z Size: 64(0x40) KB 2025-07-24T04:24:31.0277945Z Allocatable: FALSE 2025-07-24T04:24:31.0278281Z Alloc Granule: 0KB 2025-07-24T04:24:31.0278765Z Alloc Recommended Granule:0KB 2025-07-24T04:24:31.0279127Z Alloc Alignment: 0KB 2025-07-24T04:24:31.0279477Z Accessible by all: FALSE 2025-07-24T04:24:31.0279780Z ISA Info: 2025-07-24T04:24:31.0279998Z ISA 1 2025-07-24T04:24:31.0280283Z Name: amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack- 2025-07-24T04:24:31.0280658Z Machine Models: HSA_MACHINE_MODEL_LARGE 2025-07-24T04:24:31.0281016Z Profiles: HSA_PROFILE_BASE 2025-07-24T04:24:31.0281369Z Default Rounding Mode: NEAR 2025-07-24T04:24:31.0281732Z Default Rounding Mode: NEAR 2025-07-24T04:24:31.0282068Z Fast f16: TRUE 2025-07-24T04:24:31.0282472Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:24:31.0282802Z Workgroup Max Size per Dimension: 2025-07-24T04:24:31.0283078Z x 1024(0x400) 2025-07-24T04:24:31.0283528Z y 1024(0x400) 2025-07-24T04:24:31.0283811Z z 1024(0x400) 2025-07-24T04:24:31.0284121Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:24:31.0284425Z Grid Max Size per Dimension: 2025-07-24T04:24:31.0284687Z x 4294967295(0xffffffff) 2025-07-24T04:24:31.0284974Z y 4294967295(0xffffffff) 2025-07-24T04:24:31.0285264Z z 4294967295(0xffffffff) 2025-07-24T04:24:31.0285583Z FBarrier Max Size: 32 2025-07-24T04:24:31.0285881Z ******* 2025-07-24T04:24:31.0286096Z Agent 4 2025-07-24T04:24:31.0286306Z ******* 2025-07-24T04:24:31.0286548Z Name: gfx90a 2025-07-24T04:24:31.0286866Z Uuid: GPU-5ac94fbd079c5a52 2025-07-24T04:24:31.0287207Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:24:31.0287554Z Vendor Name: AMD 2025-07-24T04:24:31.0287886Z Feature: KERNEL_DISPATCH 2025-07-24T04:24:31.0288214Z Profile: BASE_PROFILE 2025-07-24T04:24:31.0288563Z Float Round Mode: NEAR 2025-07-24T04:24:31.0288907Z Max Queue Number: 128(0x80) 2025-07-24T04:24:31.0289244Z Queue Min Size: 64(0x40) 2025-07-24T04:24:31.0289568Z Queue Max Size: 131072(0x20000) 2025-07-24T04:24:31.0289897Z Queue Type: MULTI 2025-07-24T04:24:31.0290204Z Node: 3 2025-07-24T04:24:31.0290517Z Device Type: GPU 2025-07-24T04:24:31.0290808Z Cache Info: 2025-07-24T04:24:31.0291048Z L1: 16(0x10) KB 2025-07-24T04:24:31.0291339Z L2: 8192(0x2000) KB 2025-07-24T04:24:31.0291632Z Chip ID: 29708(0x740c) 2025-07-24T04:24:31.0291950Z ASIC Revision: 1(0x1) 2025-07-24T04:24:31.0292285Z Cacheline Size: 128(0x80) 2025-07-24T04:24:31.0292626Z Max Clock Freq. (MHz): 1700 2025-07-24T04:24:31.0292947Z BDFID: 13568 2025-07-24T04:24:31.0293264Z Internal Node ID: 3 2025-07-24T04:24:31.0293760Z Compute Unit: 104 2025-07-24T04:24:31.0294093Z SIMDs per CU: 4 2025-07-24T04:24:31.0294425Z Shader Engines: 8 2025-07-24T04:24:31.0294764Z Shader Arrs. per Eng.: 1 2025-07-24T04:24:31.0295115Z WatchPts on Addr. Ranges:4 2025-07-24T04:24:31.0295472Z Coherent Host Access: FALSE 2025-07-24T04:24:31.0295778Z Memory Properties: 2025-07-24T04:24:31.0296029Z Features: KERNEL_DISPATCH 2025-07-24T04:24:31.0296344Z Fast F16 Operation: TRUE 2025-07-24T04:24:31.0296690Z Wavefront Size: 64(0x40) 2025-07-24T04:24:31.0297039Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:24:31.0297355Z Workgroup Max Size per Dimension: 2025-07-24T04:24:31.0297636Z x 1024(0x400) 2025-07-24T04:24:31.0297915Z y 1024(0x400) 2025-07-24T04:24:31.0298332Z z 1024(0x400) 2025-07-24T04:24:31.0298638Z Max Waves Per CU: 32(0x20) 2025-07-24T04:24:31.0298982Z Max Work-item Per CU: 2048(0x800) 2025-07-24T04:24:31.0299321Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:24:31.0299619Z Grid Max Size per Dimension: 2025-07-24T04:24:31.0299863Z x 4294967295(0xffffffff) 2025-07-24T04:24:31.0300149Z y 4294967295(0xffffffff) 2025-07-24T04:24:31.0300440Z z 4294967295(0xffffffff) 2025-07-24T04:24:31.0300772Z Max fbarriers/Workgrp: 32 2025-07-24T04:24:31.0301141Z Packet Processor uCode:: 92 2025-07-24T04:24:31.0301494Z SDMA engine uCode:: 9 2025-07-24T04:24:31.0301845Z IOMMU Support:: None 2025-07-24T04:24:31.0302146Z Pool Info: 2025-07-24T04:24:31.0302380Z Pool 1 2025-07-24T04:24:31.0302668Z Segment: GLOBAL; FLAGS: COARSE GRAINED 2025-07-24T04:24:31.0303001Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0303327Z Allocatable: TRUE 2025-07-24T04:24:31.0303668Z Alloc Granule: 4KB 2025-07-24T04:24:31.0304029Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0304391Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0304754Z Accessible by all: FALSE 2025-07-24T04:24:31.0305065Z Pool 2 2025-07-24T04:24:31.0305343Z Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED 2025-07-24T04:24:31.0305677Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0306001Z Allocatable: TRUE 2025-07-24T04:24:31.0306346Z Alloc Granule: 4KB 2025-07-24T04:24:31.0306702Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0307064Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0307413Z Accessible by all: FALSE 2025-07-24T04:24:31.0307713Z Pool 3 2025-07-24T04:24:31.0307985Z Segment: GLOBAL; FLAGS: FINE GRAINED 2025-07-24T04:24:31.0308312Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0308770Z Allocatable: TRUE 2025-07-24T04:24:31.0309115Z Alloc Granule: 4KB 2025-07-24T04:24:31.0309474Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0309832Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0310180Z Accessible by all: FALSE 2025-07-24T04:24:31.0310479Z Pool 4 2025-07-24T04:24:31.0310741Z Segment: GROUP 2025-07-24T04:24:31.0311054Z Size: 64(0x40) KB 2025-07-24T04:24:31.0311375Z Allocatable: FALSE 2025-07-24T04:24:31.0311717Z Alloc Granule: 0KB 2025-07-24T04:24:31.0312068Z Alloc Recommended Granule:0KB 2025-07-24T04:24:31.0312431Z Alloc Alignment: 0KB 2025-07-24T04:24:31.0312779Z Accessible by all: FALSE 2025-07-24T04:24:31.0313234Z ISA Info: 2025-07-24T04:24:31.0313448Z ISA 1 2025-07-24T04:24:31.0313735Z Name: amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack- 2025-07-24T04:24:31.0314115Z Machine Models: HSA_MACHINE_MODEL_LARGE 2025-07-24T04:24:31.0314466Z Profiles: HSA_PROFILE_BASE 2025-07-24T04:24:31.0314818Z Default Rounding Mode: NEAR 2025-07-24T04:24:31.0315181Z Default Rounding Mode: NEAR 2025-07-24T04:24:31.0315515Z Fast f16: TRUE 2025-07-24T04:24:31.0315847Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:24:31.0316189Z Workgroup Max Size per Dimension: 2025-07-24T04:24:31.0316482Z x 1024(0x400) 2025-07-24T04:24:31.0316782Z y 1024(0x400) 2025-07-24T04:24:31.0317071Z z 1024(0x400) 2025-07-24T04:24:31.0317393Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:24:31.0317704Z Grid Max Size per Dimension: 2025-07-24T04:24:31.0317965Z x 4294967295(0xffffffff) 2025-07-24T04:24:31.0318262Z y 4294967295(0xffffffff) 2025-07-24T04:24:31.0318552Z z 4294967295(0xffffffff) 2025-07-24T04:24:31.0318878Z FBarrier Max Size: 32 2025-07-24T04:24:31.0319180Z ******* 2025-07-24T04:24:31.0319399Z Agent 5 2025-07-24T04:24:31.0319607Z ******* 2025-07-24T04:24:31.0319857Z Name: gfx90a 2025-07-24T04:24:31.0320180Z Uuid: GPU-5a57d8d7f376d86a 2025-07-24T04:24:31.0320531Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:24:31.0320884Z Vendor Name: AMD 2025-07-24T04:24:31.0321218Z Feature: KERNEL_DISPATCH 2025-07-24T04:24:31.0321546Z Profile: BASE_PROFILE 2025-07-24T04:24:31.0321888Z Float Round Mode: NEAR 2025-07-24T04:24:31.0322232Z Max Queue Number: 128(0x80) 2025-07-24T04:24:31.0322643Z Queue Min Size: 64(0x40) 2025-07-24T04:24:31.0322976Z Queue Max Size: 131072(0x20000) 2025-07-24T04:24:31.0323311Z Queue Type: MULTI 2025-07-24T04:24:31.0323780Z Node: 4 2025-07-24T04:24:31.0324107Z Device Type: GPU 2025-07-24T04:24:31.0324409Z Cache Info: 2025-07-24T04:24:31.0324660Z L1: 16(0x10) KB 2025-07-24T04:24:31.0324954Z L2: 8192(0x2000) KB 2025-07-24T04:24:31.0325254Z Chip ID: 29708(0x740c) 2025-07-24T04:24:31.0325581Z ASIC Revision: 1(0x1) 2025-07-24T04:24:31.0325927Z Cacheline Size: 128(0x80) 2025-07-24T04:24:31.0326268Z Max Clock Freq. (MHz): 1700 2025-07-24T04:24:31.0326590Z BDFID: 4352 2025-07-24T04:24:31.0326924Z Internal Node ID: 4 2025-07-24T04:24:31.0327266Z Compute Unit: 104 2025-07-24T04:24:31.0327599Z SIMDs per CU: 4 2025-07-24T04:24:31.0327937Z Shader Engines: 8 2025-07-24T04:24:31.0328449Z Shader Arrs. per Eng.: 1 2025-07-24T04:24:31.0328819Z WatchPts on Addr. Ranges:4 2025-07-24T04:24:31.0329182Z Coherent Host Access: FALSE 2025-07-24T04:24:31.0329502Z Memory Properties: 2025-07-24T04:24:31.0329756Z Features: KERNEL_DISPATCH 2025-07-24T04:24:31.0330076Z Fast F16 Operation: TRUE 2025-07-24T04:24:31.0330427Z Wavefront Size: 64(0x40) 2025-07-24T04:24:31.0330773Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:24:31.0331102Z Workgroup Max Size per Dimension: 2025-07-24T04:24:31.0331379Z x 1024(0x400) 2025-07-24T04:24:31.0331674Z y 1024(0x400) 2025-07-24T04:24:31.0331964Z z 1024(0x400) 2025-07-24T04:24:31.0332276Z Max Waves Per CU: 32(0x20) 2025-07-24T04:24:31.0332626Z Max Work-item Per CU: 2048(0x800) 2025-07-24T04:24:31.0332974Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:24:31.0333287Z Grid Max Size per Dimension: 2025-07-24T04:24:31.0333539Z x 4294967295(0xffffffff) 2025-07-24T04:24:31.0333828Z y 4294967295(0xffffffff) 2025-07-24T04:24:31.0334116Z z 4294967295(0xffffffff) 2025-07-24T04:24:31.0334450Z Max fbarriers/Workgrp: 32 2025-07-24T04:24:31.0334827Z Packet Processor uCode:: 92 2025-07-24T04:24:31.0335186Z SDMA engine uCode:: 9 2025-07-24T04:24:31.0335537Z IOMMU Support:: None 2025-07-24T04:24:31.0335844Z Pool Info: 2025-07-24T04:24:31.0336071Z Pool 1 2025-07-24T04:24:31.0336356Z Segment: GLOBAL; FLAGS: COARSE GRAINED 2025-07-24T04:24:31.0336695Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0337027Z Allocatable: TRUE 2025-07-24T04:24:31.0337380Z Alloc Granule: 4KB 2025-07-24T04:24:31.0337745Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0338118Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0338485Z Accessible by all: FALSE 2025-07-24T04:24:31.0338795Z Pool 2 2025-07-24T04:24:31.0339219Z Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED 2025-07-24T04:24:31.0339552Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0339883Z Allocatable: TRUE 2025-07-24T04:24:31.0340228Z Alloc Granule: 4KB 2025-07-24T04:24:31.0340599Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0340965Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0341320Z Accessible by all: FALSE 2025-07-24T04:24:31.0341628Z Pool 3 2025-07-24T04:24:31.0341913Z Segment: GLOBAL; FLAGS: FINE GRAINED 2025-07-24T04:24:31.0342242Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0342569Z Allocatable: TRUE 2025-07-24T04:24:31.0342918Z Alloc Granule: 4KB 2025-07-24T04:24:31.0343278Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0343771Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0344123Z Accessible by all: FALSE 2025-07-24T04:24:31.0344425Z Pool 4 2025-07-24T04:24:31.0344695Z Segment: GROUP 2025-07-24T04:24:31.0345014Z Size: 64(0x40) KB 2025-07-24T04:24:31.0345341Z Allocatable: FALSE 2025-07-24T04:24:31.0345692Z Alloc Granule: 0KB 2025-07-24T04:24:31.0346053Z Alloc Recommended Granule:0KB 2025-07-24T04:24:31.0346414Z Alloc Alignment: 0KB 2025-07-24T04:24:31.0346767Z Accessible by all: FALSE 2025-07-24T04:24:31.0347085Z ISA Info: 2025-07-24T04:24:31.0347323Z ISA 1 2025-07-24T04:24:31.0347619Z Name: amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack- 2025-07-24T04:24:31.0348009Z Machine Models: HSA_MACHINE_MODEL_LARGE 2025-07-24T04:24:31.0348383Z Profiles: HSA_PROFILE_BASE 2025-07-24T04:24:31.0348744Z Default Rounding Mode: NEAR 2025-07-24T04:24:31.0349105Z Default Rounding Mode: NEAR 2025-07-24T04:24:31.0349447Z Fast f16: TRUE 2025-07-24T04:24:31.0349788Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:24:31.0350122Z Workgroup Max Size per Dimension: 2025-07-24T04:24:31.0350413Z x 1024(0x400) 2025-07-24T04:24:31.0350712Z y 1024(0x400) 2025-07-24T04:24:31.0351003Z z 1024(0x400) 2025-07-24T04:24:31.0351321Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:24:31.0351632Z Grid Max Size per Dimension: 2025-07-24T04:24:31.0351897Z x 4294967295(0xffffffff) 2025-07-24T04:24:31.0352195Z y 4294967295(0xffffffff) 2025-07-24T04:24:31.0352485Z z 4294967295(0xffffffff) 2025-07-24T04:24:31.0352812Z FBarrier Max Size: 32 2025-07-24T04:24:31.0353108Z ******* 2025-07-24T04:24:31.0353328Z Agent 6 2025-07-24T04:24:31.0353540Z ******* 2025-07-24T04:24:31.0353789Z Name: gfx90a 2025-07-24T04:24:31.0354251Z Uuid: GPU-869e5dfc4dcadcc3 2025-07-24T04:24:31.0354612Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:24:31.0354969Z Vendor Name: AMD 2025-07-24T04:24:31.0355309Z Feature: KERNEL_DISPATCH 2025-07-24T04:24:31.0355647Z Profile: BASE_PROFILE 2025-07-24T04:24:31.0355991Z Float Round Mode: NEAR 2025-07-24T04:24:31.0356341Z Max Queue Number: 128(0x80) 2025-07-24T04:24:31.0356681Z Queue Min Size: 64(0x40) 2025-07-24T04:24:31.0357016Z Queue Max Size: 131072(0x20000) 2025-07-24T04:24:31.0357346Z Queue Type: MULTI 2025-07-24T04:24:31.0357660Z Node: 5 2025-07-24T04:24:31.0357988Z Device Type: GPU 2025-07-24T04:24:31.0358294Z Cache Info: 2025-07-24T04:24:31.0358681Z L1: 16(0x10) KB 2025-07-24T04:24:31.0358978Z L2: 8192(0x2000) KB 2025-07-24T04:24:31.0359282Z Chip ID: 29708(0x740c) 2025-07-24T04:24:31.0359612Z ASIC Revision: 1(0x1) 2025-07-24T04:24:31.0359966Z Cacheline Size: 128(0x80) 2025-07-24T04:24:31.0360317Z Max Clock Freq. (MHz): 1700 2025-07-24T04:24:31.0360645Z BDFID: 5120 2025-07-24T04:24:31.0360973Z Internal Node ID: 5 2025-07-24T04:24:31.0361313Z Compute Unit: 104 2025-07-24T04:24:31.0361657Z SIMDs per CU: 4 2025-07-24T04:24:31.0361993Z Shader Engines: 8 2025-07-24T04:24:31.0362404Z Shader Arrs. per Eng.: 1 2025-07-24T04:24:31.0362772Z WatchPts on Addr. Ranges:4 2025-07-24T04:24:31.0363132Z Coherent Host Access: FALSE 2025-07-24T04:24:31.0363450Z Memory Properties: 2025-07-24T04:24:31.0363708Z Features: KERNEL_DISPATCH 2025-07-24T04:24:31.0364025Z Fast F16 Operation: TRUE 2025-07-24T04:24:31.0364374Z Wavefront Size: 64(0x40) 2025-07-24T04:24:31.0364724Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:24:31.0365044Z Workgroup Max Size per Dimension: 2025-07-24T04:24:31.0365315Z x 1024(0x400) 2025-07-24T04:24:31.0365601Z y 1024(0x400) 2025-07-24T04:24:31.0365880Z z 1024(0x400) 2025-07-24T04:24:31.0366188Z Max Waves Per CU: 32(0x20) 2025-07-24T04:24:31.0366533Z Max Work-item Per CU: 2048(0x800) 2025-07-24T04:24:31.0366877Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:24:31.0367183Z Grid Max Size per Dimension: 2025-07-24T04:24:31.0367440Z x 4294967295(0xffffffff) 2025-07-24T04:24:31.0367727Z y 4294967295(0xffffffff) 2025-07-24T04:24:31.0368012Z z 4294967295(0xffffffff) 2025-07-24T04:24:31.0368344Z Max fbarriers/Workgrp: 32 2025-07-24T04:24:31.0368727Z Packet Processor uCode:: 92 2025-07-24T04:24:31.0369087Z SDMA engine uCode:: 9 2025-07-24T04:24:31.0369598Z IOMMU Support:: None 2025-07-24T04:24:31.0369908Z Pool Info: 2025-07-24T04:24:31.0370148Z Pool 1 2025-07-24T04:24:31.0370431Z Segment: GLOBAL; FLAGS: COARSE GRAINED 2025-07-24T04:24:31.0370777Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0371121Z Allocatable: TRUE 2025-07-24T04:24:31.0371468Z Alloc Granule: 4KB 2025-07-24T04:24:31.0371833Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0372206Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0372562Z Accessible by all: FALSE 2025-07-24T04:24:31.0372868Z Pool 2 2025-07-24T04:24:31.0373159Z Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED 2025-07-24T04:24:31.0373498Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0374018Z Allocatable: TRUE 2025-07-24T04:24:31.0374362Z Alloc Granule: 4KB 2025-07-24T04:24:31.0374740Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0375105Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0375457Z Accessible by all: FALSE 2025-07-24T04:24:31.0375763Z Pool 3 2025-07-24T04:24:31.0376042Z Segment: GLOBAL; FLAGS: FINE GRAINED 2025-07-24T04:24:31.0376361Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0376684Z Allocatable: TRUE 2025-07-24T04:24:31.0377026Z Alloc Granule: 4KB 2025-07-24T04:24:31.0377395Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0377759Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0378120Z Accessible by all: FALSE 2025-07-24T04:24:31.0378438Z Pool 4 2025-07-24T04:24:31.0378702Z Segment: GROUP 2025-07-24T04:24:31.0379014Z Size: 64(0x40) KB 2025-07-24T04:24:31.0379338Z Allocatable: FALSE 2025-07-24T04:24:31.0379687Z Alloc Granule: 0KB 2025-07-24T04:24:31.0380041Z Alloc Recommended Granule:0KB 2025-07-24T04:24:31.0380401Z Alloc Alignment: 0KB 2025-07-24T04:24:31.0380753Z Accessible by all: FALSE 2025-07-24T04:24:31.0381061Z ISA Info: 2025-07-24T04:24:31.0381281Z ISA 1 2025-07-24T04:24:31.0381571Z Name: amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack- 2025-07-24T04:24:31.0381947Z Machine Models: HSA_MACHINE_MODEL_LARGE 2025-07-24T04:24:31.0382316Z Profiles: HSA_PROFILE_BASE 2025-07-24T04:24:31.0382670Z Default Rounding Mode: NEAR 2025-07-24T04:24:31.0383033Z Default Rounding Mode: NEAR 2025-07-24T04:24:31.0383372Z Fast f16: TRUE 2025-07-24T04:24:31.0383714Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:24:31.0384138Z Workgroup Max Size per Dimension: 2025-07-24T04:24:31.0384731Z x 1024(0x400) 2025-07-24T04:24:31.0398181Z y 1024(0x400) 2025-07-24T04:24:31.0398547Z z 1024(0x400) 2025-07-24T04:24:31.0398894Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:24:31.0399212Z Grid Max Size per Dimension: 2025-07-24T04:24:31.0399492Z x 4294967295(0xffffffff) 2025-07-24T04:24:31.0399794Z y 4294967295(0xffffffff) 2025-07-24T04:24:31.0400088Z z 4294967295(0xffffffff) 2025-07-24T04:24:31.0400413Z FBarrier Max Size: 32 2025-07-24T04:24:31.0400723Z ******* 2025-07-24T04:24:31.0400948Z Agent 7 2025-07-24T04:24:31.0401161Z ******* 2025-07-24T04:24:31.0401408Z Name: gfx90a 2025-07-24T04:24:31.0401726Z Uuid: GPU-2d13401035fea6d5 2025-07-24T04:24:31.0402073Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:24:31.0402501Z Vendor Name: AMD 2025-07-24T04:24:31.0403015Z Feature: KERNEL_DISPATCH 2025-07-24T04:24:31.0403349Z Profile: BASE_PROFILE 2025-07-24T04:24:31.0403688Z Float Round Mode: NEAR 2025-07-24T04:24:31.0404027Z Max Queue Number: 128(0x80) 2025-07-24T04:24:31.0404358Z Queue Min Size: 64(0x40) 2025-07-24T04:24:31.0404685Z Queue Max Size: 131072(0x20000) 2025-07-24T04:24:31.0405009Z Queue Type: MULTI 2025-07-24T04:24:31.0405328Z Node: 6 2025-07-24T04:24:31.0405649Z Device Type: GPU 2025-07-24T04:24:31.0405960Z Cache Info: 2025-07-24T04:24:31.0406210Z L1: 16(0x10) KB 2025-07-24T04:24:31.0406501Z L2: 8192(0x2000) KB 2025-07-24T04:24:31.0406814Z Chip ID: 29708(0x740c) 2025-07-24T04:24:31.0407142Z ASIC Revision: 1(0x1) 2025-07-24T04:24:31.0407487Z Cacheline Size: 128(0x80) 2025-07-24T04:24:31.0407828Z Max Clock Freq. (MHz): 1700 2025-07-24T04:24:31.0408145Z BDFID: 44544 2025-07-24T04:24:31.0408461Z Internal Node ID: 6 2025-07-24T04:24:31.0408801Z Compute Unit: 104 2025-07-24T04:24:31.0409125Z SIMDs per CU: 4 2025-07-24T04:24:31.0409457Z Shader Engines: 8 2025-07-24T04:24:31.0409797Z Shader Arrs. per Eng.: 1 2025-07-24T04:24:31.0410151Z WatchPts on Addr. Ranges:4 2025-07-24T04:24:31.0410506Z Coherent Host Access: FALSE 2025-07-24T04:24:31.0410821Z Memory Properties: 2025-07-24T04:24:31.0411070Z Features: KERNEL_DISPATCH 2025-07-24T04:24:31.0411390Z Fast F16 Operation: TRUE 2025-07-24T04:24:31.0411741Z Wavefront Size: 64(0x40) 2025-07-24T04:24:31.0412086Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:24:31.0412405Z Workgroup Max Size per Dimension: 2025-07-24T04:24:31.0412678Z x 1024(0x400) 2025-07-24T04:24:31.0412957Z y 1024(0x400) 2025-07-24T04:24:31.0413422Z z 1024(0x400) 2025-07-24T04:24:31.0413734Z Max Waves Per CU: 32(0x20) 2025-07-24T04:24:31.0414080Z Max Work-item Per CU: 2048(0x800) 2025-07-24T04:24:31.0414420Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:24:31.0414728Z Grid Max Size per Dimension: 2025-07-24T04:24:31.0414977Z x 4294967295(0xffffffff) 2025-07-24T04:24:31.0415257Z y 4294967295(0xffffffff) 2025-07-24T04:24:31.0415542Z z 4294967295(0xffffffff) 2025-07-24T04:24:31.0415867Z Max fbarriers/Workgrp: 32 2025-07-24T04:24:31.0416242Z Packet Processor uCode:: 92 2025-07-24T04:24:31.0416596Z SDMA engine uCode:: 9 2025-07-24T04:24:31.0416942Z IOMMU Support:: None 2025-07-24T04:24:31.0417245Z Pool Info: 2025-07-24T04:24:31.0417474Z Pool 1 2025-07-24T04:24:31.0417896Z Segment: GLOBAL; FLAGS: COARSE GRAINED 2025-07-24T04:24:31.0418232Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0418566Z Allocatable: TRUE 2025-07-24T04:24:31.0418908Z Alloc Granule: 4KB 2025-07-24T04:24:31.0419263Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0419636Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0419992Z Accessible by all: FALSE 2025-07-24T04:24:31.0420297Z Pool 2 2025-07-24T04:24:31.0420578Z Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED 2025-07-24T04:24:31.0420921Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0421257Z Allocatable: TRUE 2025-07-24T04:24:31.0421606Z Alloc Granule: 4KB 2025-07-24T04:24:31.0421961Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0422321Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0422667Z Accessible by all: FALSE 2025-07-24T04:24:31.0422967Z Pool 3 2025-07-24T04:24:31.0423250Z Segment: GLOBAL; FLAGS: FINE GRAINED 2025-07-24T04:24:31.0423571Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0423892Z Allocatable: TRUE 2025-07-24T04:24:31.0424229Z Alloc Granule: 4KB 2025-07-24T04:24:31.0424593Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0424953Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0425309Z Accessible by all: FALSE 2025-07-24T04:24:31.0425612Z Pool 4 2025-07-24T04:24:31.0425875Z Segment: GROUP 2025-07-24T04:24:31.0426182Z Size: 64(0x40) KB 2025-07-24T04:24:31.0426502Z Allocatable: FALSE 2025-07-24T04:24:31.0426844Z Alloc Granule: 0KB 2025-07-24T04:24:31.0427204Z Alloc Recommended Granule:0KB 2025-07-24T04:24:31.0427559Z Alloc Alignment: 0KB 2025-07-24T04:24:31.0427907Z Accessible by all: FALSE 2025-07-24T04:24:31.0428214Z ISA Info: 2025-07-24T04:24:31.0428571Z ISA 1 2025-07-24T04:24:31.0428869Z Name: amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack- 2025-07-24T04:24:31.0429252Z Machine Models: HSA_MACHINE_MODEL_LARGE 2025-07-24T04:24:31.0429610Z Profiles: HSA_PROFILE_BASE 2025-07-24T04:24:31.0429962Z Default Rounding Mode: NEAR 2025-07-24T04:24:31.0430323Z Default Rounding Mode: NEAR 2025-07-24T04:24:31.0430660Z Fast f16: TRUE 2025-07-24T04:24:31.0430995Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:24:31.0431321Z Workgroup Max Size per Dimension: 2025-07-24T04:24:31.0431606Z x 1024(0x400) 2025-07-24T04:24:31.0431895Z y 1024(0x400) 2025-07-24T04:24:31.0432176Z z 1024(0x400) 2025-07-24T04:24:31.0432488Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:24:31.0432924Z Grid Max Size per Dimension: 2025-07-24T04:24:31.0433189Z x 4294967295(0xffffffff) 2025-07-24T04:24:31.0433470Z y 4294967295(0xffffffff) 2025-07-24T04:24:31.0433752Z z 4294967295(0xffffffff) 2025-07-24T04:24:31.0434072Z FBarrier Max Size: 32 2025-07-24T04:24:31.0434368Z ******* 2025-07-24T04:24:31.0434587Z Agent 8 2025-07-24T04:24:31.0434796Z ******* 2025-07-24T04:24:31.0435039Z Name: gfx90a 2025-07-24T04:24:31.0435352Z Uuid: GPU-d15980d687168477 2025-07-24T04:24:31.0435693Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:24:31.0436040Z Vendor Name: AMD 2025-07-24T04:24:31.0436376Z Feature: KERNEL_DISPATCH 2025-07-24T04:24:31.0436705Z Profile: BASE_PROFILE 2025-07-24T04:24:31.0437045Z Float Round Mode: NEAR 2025-07-24T04:24:31.0437380Z Max Queue Number: 128(0x80) 2025-07-24T04:24:31.0437715Z Queue Min Size: 64(0x40) 2025-07-24T04:24:31.0438046Z Queue Max Size: 131072(0x20000) 2025-07-24T04:24:31.0438374Z Queue Type: MULTI 2025-07-24T04:24:31.0438686Z Node: 7 2025-07-24T04:24:31.0438996Z Device Type: GPU 2025-07-24T04:24:31.0439289Z Cache Info: 2025-07-24T04:24:31.0439539Z L1: 16(0x10) KB 2025-07-24T04:24:31.0439835Z L2: 8192(0x2000) KB 2025-07-24T04:24:31.0440134Z Chip ID: 29708(0x740c) 2025-07-24T04:24:31.0440458Z ASIC Revision: 1(0x1) 2025-07-24T04:24:31.0440792Z Cacheline Size: 128(0x80) 2025-07-24T04:24:31.0441135Z Max Clock Freq. (MHz): 1700 2025-07-24T04:24:31.0441448Z BDFID: 45824 2025-07-24T04:24:31.0441772Z Internal Node ID: 7 2025-07-24T04:24:31.0442120Z Compute Unit: 104 2025-07-24T04:24:31.0442531Z SIMDs per CU: 4 2025-07-24T04:24:31.0442860Z Shader Engines: 8 2025-07-24T04:24:31.0443368Z Shader Arrs. per Eng.: 1 2025-07-24T04:24:31.0443729Z WatchPts on Addr. Ranges:4 2025-07-24T04:24:31.0444092Z Coherent Host Access: FALSE 2025-07-24T04:24:31.0444406Z Memory Properties: 2025-07-24T04:24:31.0444661Z Features: KERNEL_DISPATCH 2025-07-24T04:24:31.0444990Z Fast F16 Operation: TRUE 2025-07-24T04:24:31.0445354Z Wavefront Size: 64(0x40) 2025-07-24T04:24:31.0445708Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:24:31.0446025Z Workgroup Max Size per Dimension: 2025-07-24T04:24:31.0446295Z x 1024(0x400) 2025-07-24T04:24:31.0446575Z y 1024(0x400) 2025-07-24T04:24:31.0446855Z z 1024(0x400) 2025-07-24T04:24:31.0447162Z Max Waves Per CU: 32(0x20) 2025-07-24T04:24:31.0447503Z Max Work-item Per CU: 2048(0x800) 2025-07-24T04:24:31.0447998Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:24:31.0448298Z Grid Max Size per Dimension: 2025-07-24T04:24:31.0448554Z x 4294967295(0xffffffff) 2025-07-24T04:24:31.0448834Z y 4294967295(0xffffffff) 2025-07-24T04:24:31.0449118Z z 4294967295(0xffffffff) 2025-07-24T04:24:31.0449452Z Max fbarriers/Workgrp: 32 2025-07-24T04:24:31.0449822Z Packet Processor uCode:: 92 2025-07-24T04:24:31.0450176Z SDMA engine uCode:: 9 2025-07-24T04:24:31.0450512Z IOMMU Support:: None 2025-07-24T04:24:31.0450827Z Pool Info: 2025-07-24T04:24:31.0451066Z Pool 1 2025-07-24T04:24:31.0451353Z Segment: GLOBAL; FLAGS: COARSE GRAINED 2025-07-24T04:24:31.0451698Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0452026Z Allocatable: TRUE 2025-07-24T04:24:31.0452370Z Alloc Granule: 4KB 2025-07-24T04:24:31.0452733Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0453093Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0453441Z Accessible by all: FALSE 2025-07-24T04:24:31.0453745Z Pool 2 2025-07-24T04:24:31.0454020Z Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED 2025-07-24T04:24:31.0454355Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0454680Z Allocatable: TRUE 2025-07-24T04:24:31.0455014Z Alloc Granule: 4KB 2025-07-24T04:24:31.0455368Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0455729Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0456074Z Accessible by all: FALSE 2025-07-24T04:24:31.0456374Z Pool 3 2025-07-24T04:24:31.0456642Z Segment: GLOBAL; FLAGS: FINE GRAINED 2025-07-24T04:24:31.0456963Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0457279Z Allocatable: TRUE 2025-07-24T04:24:31.0457616Z Alloc Granule: 4KB 2025-07-24T04:24:31.0457969Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0458463Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0458825Z Accessible by all: FALSE 2025-07-24T04:24:31.0459126Z Pool 4 2025-07-24T04:24:31.0459380Z Segment: GROUP 2025-07-24T04:24:31.0459688Z Size: 64(0x40) KB 2025-07-24T04:24:31.0460007Z Allocatable: FALSE 2025-07-24T04:24:31.0460339Z Alloc Granule: 0KB 2025-07-24T04:24:31.0460691Z Alloc Recommended Granule:0KB 2025-07-24T04:24:31.0461049Z Alloc Alignment: 0KB 2025-07-24T04:24:31.0461395Z Accessible by all: FALSE 2025-07-24T04:24:31.0461694Z ISA Info: 2025-07-24T04:24:31.0461918Z ISA 1 2025-07-24T04:24:31.0462204Z Name: amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack- 2025-07-24T04:24:31.0462574Z Machine Models: HSA_MACHINE_MODEL_LARGE 2025-07-24T04:24:31.0463064Z Profiles: HSA_PROFILE_BASE 2025-07-24T04:24:31.0463416Z Default Rounding Mode: NEAR 2025-07-24T04:24:31.0463774Z Default Rounding Mode: NEAR 2025-07-24T04:24:31.0464103Z Fast f16: TRUE 2025-07-24T04:24:31.0464436Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:24:31.0464756Z Workgroup Max Size per Dimension: 2025-07-24T04:24:31.0465039Z x 1024(0x400) 2025-07-24T04:24:31.0465323Z y 1024(0x400) 2025-07-24T04:24:31.0465600Z z 1024(0x400) 2025-07-24T04:24:31.0465911Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:24:31.0466218Z Grid Max Size per Dimension: 2025-07-24T04:24:31.0466482Z x 4294967295(0xffffffff) 2025-07-24T04:24:31.0466765Z y 4294967295(0xffffffff) 2025-07-24T04:24:31.0467051Z z 4294967295(0xffffffff) 2025-07-24T04:24:31.0467368Z FBarrier Max Size: 32 2025-07-24T04:24:31.0467663Z ******* 2025-07-24T04:24:31.0467873Z Agent 9 2025-07-24T04:24:31.0468085Z ******* 2025-07-24T04:24:31.0468324Z Name: gfx90a 2025-07-24T04:24:31.0468633Z Uuid: GPU-61b229e107271af2 2025-07-24T04:24:31.0468972Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:24:31.0469320Z Vendor Name: AMD 2025-07-24T04:24:31.0469649Z Feature: KERNEL_DISPATCH 2025-07-24T04:24:31.0469987Z Profile: BASE_PROFILE 2025-07-24T04:24:31.0470323Z Float Round Mode: NEAR 2025-07-24T04:24:31.0470664Z Max Queue Number: 128(0x80) 2025-07-24T04:24:31.0470998Z Queue Min Size: 64(0x40) 2025-07-24T04:24:31.0471329Z Queue Max Size: 131072(0x20000) 2025-07-24T04:24:31.0471653Z Queue Type: MULTI 2025-07-24T04:24:31.0471957Z Node: 8 2025-07-24T04:24:31.0472262Z Device Type: GPU 2025-07-24T04:24:31.0472554Z Cache Info: 2025-07-24T04:24:31.0472793Z L1: 16(0x10) KB 2025-07-24T04:24:31.0473430Z L2: 8192(0x2000) KB 2025-07-24T04:24:31.0473733Z Chip ID: 29708(0x740c) 2025-07-24T04:24:31.0474052Z ASIC Revision: 1(0x1) 2025-07-24T04:24:31.0474384Z Cacheline Size: 128(0x80) 2025-07-24T04:24:31.0474719Z Max Clock Freq. (MHz): 1700 2025-07-24T04:24:31.0475032Z BDFID: 36352 2025-07-24T04:24:31.0475356Z Internal Node ID: 8 2025-07-24T04:24:31.0475687Z Compute Unit: 104 2025-07-24T04:24:31.0476008Z SIMDs per CU: 4 2025-07-24T04:24:31.0476334Z Shader Engines: 8 2025-07-24T04:24:31.0476670Z Shader Arrs. per Eng.: 1 2025-07-24T04:24:31.0477024Z WatchPts on Addr. Ranges:4 2025-07-24T04:24:31.0477379Z Coherent Host Access: FALSE 2025-07-24T04:24:31.0477826Z Memory Properties: 2025-07-24T04:24:31.0478072Z Features: KERNEL_DISPATCH 2025-07-24T04:24:31.0478382Z Fast F16 Operation: TRUE 2025-07-24T04:24:31.0478722Z Wavefront Size: 64(0x40) 2025-07-24T04:24:31.0479067Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:24:31.0479384Z Workgroup Max Size per Dimension: 2025-07-24T04:24:31.0479649Z x 1024(0x400) 2025-07-24T04:24:31.0479933Z y 1024(0x400) 2025-07-24T04:24:31.0480208Z z 1024(0x400) 2025-07-24T04:24:31.0480509Z Max Waves Per CU: 32(0x20) 2025-07-24T04:24:31.0480849Z Max Work-item Per CU: 2048(0x800) 2025-07-24T04:24:31.0481190Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:24:31.0481494Z Grid Max Size per Dimension: 2025-07-24T04:24:31.0481749Z x 4294967295(0xffffffff) 2025-07-24T04:24:31.0482032Z y 4294967295(0xffffffff) 2025-07-24T04:24:31.0482314Z z 4294967295(0xffffffff) 2025-07-24T04:24:31.0482706Z Max fbarriers/Workgrp: 32 2025-07-24T04:24:31.0483075Z Packet Processor uCode:: 92 2025-07-24T04:24:31.0483426Z SDMA engine uCode:: 9 2025-07-24T04:24:31.0483773Z IOMMU Support:: None 2025-07-24T04:24:31.0484070Z Pool Info: 2025-07-24T04:24:31.0484291Z Pool 1 2025-07-24T04:24:31.0484573Z Segment: GLOBAL; FLAGS: COARSE GRAINED 2025-07-24T04:24:31.0484905Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0485232Z Allocatable: TRUE 2025-07-24T04:24:31.0485571Z Alloc Granule: 4KB 2025-07-24T04:24:31.0485927Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0486292Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0486640Z Accessible by all: FALSE 2025-07-24T04:24:31.0486940Z Pool 2 2025-07-24T04:24:31.0487218Z Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED 2025-07-24T04:24:31.0487542Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0487864Z Allocatable: TRUE 2025-07-24T04:24:31.0488367Z Alloc Granule: 4KB 2025-07-24T04:24:31.0488723Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0489081Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0489427Z Accessible by all: FALSE 2025-07-24T04:24:31.0489724Z Pool 3 2025-07-24T04:24:31.0489988Z Segment: GLOBAL; FLAGS: FINE GRAINED 2025-07-24T04:24:31.0490309Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0490629Z Allocatable: TRUE 2025-07-24T04:24:31.0490969Z Alloc Granule: 4KB 2025-07-24T04:24:31.0491319Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0491676Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0492025Z Accessible by all: FALSE 2025-07-24T04:24:31.0492323Z Pool 4 2025-07-24T04:24:31.0492740Z Segment: GROUP 2025-07-24T04:24:31.0493050Z Size: 64(0x40) KB 2025-07-24T04:24:31.0493370Z Allocatable: FALSE 2025-07-24T04:24:31.0493709Z Alloc Granule: 0KB 2025-07-24T04:24:31.0494061Z Alloc Recommended Granule:0KB 2025-07-24T04:24:31.0494410Z Alloc Alignment: 0KB 2025-07-24T04:24:31.0494756Z Accessible by all: FALSE 2025-07-24T04:24:31.0495062Z ISA Info: 2025-07-24T04:24:31.0495283Z ISA 1 2025-07-24T04:24:31.0495569Z Name: amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack- 2025-07-24T04:24:31.0495942Z Machine Models: HSA_MACHINE_MODEL_LARGE 2025-07-24T04:24:31.0496303Z Profiles: HSA_PROFILE_BASE 2025-07-24T04:24:31.0496661Z Default Rounding Mode: NEAR 2025-07-24T04:24:31.0497025Z Default Rounding Mode: NEAR 2025-07-24T04:24:31.0497362Z Fast f16: TRUE 2025-07-24T04:24:31.0497695Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:24:31.0498016Z Workgroup Max Size per Dimension: 2025-07-24T04:24:31.0498298Z x 1024(0x400) 2025-07-24T04:24:31.0498597Z y 1024(0x400) 2025-07-24T04:24:31.0498870Z z 1024(0x400) 2025-07-24T04:24:31.0499177Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:24:31.0499489Z Grid Max Size per Dimension: 2025-07-24T04:24:31.0499756Z x 4294967295(0xffffffff) 2025-07-24T04:24:31.0500041Z y 4294967295(0xffffffff) 2025-07-24T04:24:31.0500321Z z 4294967295(0xffffffff) 2025-07-24T04:24:31.0500640Z FBarrier Max Size: 32 2025-07-24T04:24:31.0500940Z ******* 2025-07-24T04:24:31.0501154Z Agent 10 2025-07-24T04:24:31.0501366Z ******* 2025-07-24T04:24:31.0501608Z Name: gfx90a 2025-07-24T04:24:31.0501919Z Uuid: GPU-f5ee81c23f707083 2025-07-24T04:24:31.0502250Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:24:31.0502593Z Vendor Name: AMD 2025-07-24T04:24:31.0503055Z Feature: KERNEL_DISPATCH 2025-07-24T04:24:31.0503390Z Profile: BASE_PROFILE 2025-07-24T04:24:31.0503728Z Float Round Mode: NEAR 2025-07-24T04:24:31.0504064Z Max Queue Number: 128(0x80) 2025-07-24T04:24:31.0504394Z Queue Min Size: 64(0x40) 2025-07-24T04:24:31.0504719Z Queue Max Size: 131072(0x20000) 2025-07-24T04:24:31.0505045Z Queue Type: MULTI 2025-07-24T04:24:31.0505350Z Node: 9 2025-07-24T04:24:31.0505660Z Device Type: GPU 2025-07-24T04:24:31.0505953Z Cache Info: 2025-07-24T04:24:31.0506190Z L1: 16(0x10) KB 2025-07-24T04:24:31.0506477Z L2: 8192(0x2000) KB 2025-07-24T04:24:31.0506768Z Chip ID: 29708(0x740c) 2025-07-24T04:24:31.0507213Z ASIC Revision: 1(0x1) 2025-07-24T04:24:31.0507548Z Cacheline Size: 128(0x80) 2025-07-24T04:24:31.0507887Z Max Clock Freq. (MHz): 1700 2025-07-24T04:24:31.0508204Z BDFID: 37632 2025-07-24T04:24:31.0508525Z Internal Node ID: 9 2025-07-24T04:24:31.0508858Z Compute Unit: 104 2025-07-24T04:24:31.0509181Z SIMDs per CU: 4 2025-07-24T04:24:31.0509509Z Shader Engines: 8 2025-07-24T04:24:31.0509853Z Shader Arrs. per Eng.: 1 2025-07-24T04:24:31.0510206Z WatchPts on Addr. Ranges:4 2025-07-24T04:24:31.0510568Z Coherent Host Access: FALSE 2025-07-24T04:24:31.0510884Z Memory Properties: 2025-07-24T04:24:31.0511130Z Features: KERNEL_DISPATCH 2025-07-24T04:24:31.0511442Z Fast F16 Operation: TRUE 2025-07-24T04:24:31.0511788Z Wavefront Size: 64(0x40) 2025-07-24T04:24:31.0512133Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:24:31.0512450Z Workgroup Max Size per Dimension: 2025-07-24T04:24:31.0512712Z x 1024(0x400) 2025-07-24T04:24:31.0512994Z y 1024(0x400) 2025-07-24T04:24:31.0513265Z z 1024(0x400) 2025-07-24T04:24:31.0513570Z Max Waves Per CU: 32(0x20) 2025-07-24T04:24:31.0513910Z Max Work-item Per CU: 2048(0x800) 2025-07-24T04:24:31.0514252Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:24:31.0514557Z Grid Max Size per Dimension: 2025-07-24T04:24:31.0514809Z x 4294967295(0xffffffff) 2025-07-24T04:24:31.0515087Z y 4294967295(0xffffffff) 2025-07-24T04:24:31.0515366Z z 4294967295(0xffffffff) 2025-07-24T04:24:31.0515692Z Max fbarriers/Workgrp: 32 2025-07-24T04:24:31.0516062Z Packet Processor uCode:: 92 2025-07-24T04:24:31.0516417Z SDMA engine uCode:: 9 2025-07-24T04:24:31.0516755Z IOMMU Support:: None 2025-07-24T04:24:31.0517052Z Pool Info: 2025-07-24T04:24:31.0517274Z Pool 1 2025-07-24T04:24:31.0517555Z Segment: GLOBAL; FLAGS: COARSE GRAINED 2025-07-24T04:24:31.0518022Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0518356Z Allocatable: TRUE 2025-07-24T04:24:31.0518700Z Alloc Granule: 4KB 2025-07-24T04:24:31.0519066Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0519427Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0519775Z Accessible by all: FALSE 2025-07-24T04:24:31.0520071Z Pool 2 2025-07-24T04:24:31.0520346Z Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED 2025-07-24T04:24:31.0520672Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0520991Z Allocatable: TRUE 2025-07-24T04:24:31.0521326Z Alloc Granule: 4KB 2025-07-24T04:24:31.0521685Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0522041Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0522600Z Accessible by all: FALSE 2025-07-24T04:24:31.0522902Z Pool 3 2025-07-24T04:24:31.0523170Z Segment: GLOBAL; FLAGS: FINE GRAINED 2025-07-24T04:24:31.0523491Z Size: 67092480(0x3ffc000) KB 2025-07-24T04:24:31.0523810Z Allocatable: TRUE 2025-07-24T04:24:31.0524146Z Alloc Granule: 4KB 2025-07-24T04:24:31.0524490Z Alloc Recommended Granule:2048KB 2025-07-24T04:24:31.0524847Z Alloc Alignment: 4KB 2025-07-24T04:24:31.0525192Z Accessible by all: FALSE 2025-07-24T04:24:31.0525497Z Pool 4 2025-07-24T04:24:31.0525755Z Segment: GROUP 2025-07-24T04:24:31.0526066Z Size: 64(0x40) KB 2025-07-24T04:24:31.0526387Z Allocatable: FALSE 2025-07-24T04:24:31.0526724Z Alloc Granule: 0KB 2025-07-24T04:24:31.0527079Z Alloc Recommended Granule:0KB 2025-07-24T04:24:31.0527437Z Alloc Alignment: 0KB 2025-07-24T04:24:31.0527783Z Accessible by all: FALSE 2025-07-24T04:24:31.0528083Z ISA Info: 2025-07-24T04:24:31.0528302Z ISA 1 2025-07-24T04:24:31.0528581Z Name: amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack- 2025-07-24T04:24:31.0528956Z Machine Models: HSA_MACHINE_MODEL_LARGE 2025-07-24T04:24:31.0529311Z Profiles: HSA_PROFILE_BASE 2025-07-24T04:24:31.0529664Z Default Rounding Mode: NEAR 2025-07-24T04:24:31.0530018Z Default Rounding Mode: NEAR 2025-07-24T04:24:31.0530347Z Fast f16: TRUE 2025-07-24T04:24:31.0530681Z Workgroup Max Size: 1024(0x400) 2025-07-24T04:24:31.0531009Z Workgroup Max Size per Dimension: 2025-07-24T04:24:31.0531292Z x 1024(0x400) 2025-07-24T04:24:31.0531578Z y 1024(0x400) 2025-07-24T04:24:31.0531855Z z 1024(0x400) 2025-07-24T04:24:31.0532160Z Grid Max Size: 4294967295(0xffffffff) 2025-07-24T04:24:31.0532464Z Grid Max Size per Dimension: 2025-07-24T04:24:31.0532882Z x 4294967295(0xffffffff) 2025-07-24T04:24:31.0533167Z y 4294967295(0xffffffff) 2025-07-24T04:24:31.0533454Z z 4294967295(0xffffffff) 2025-07-24T04:24:31.0533768Z FBarrier Max Size: 32 2025-07-24T04:24:31.0534064Z *** Done *** 2025-07-24T04:24:31.0534300Z + rocminfo 2025-07-24T04:24:31.0534506Z + grep -E 'Name:.*\sgfx|Marketing' 2025-07-24T04:24:31.2715769Z Marketing Name: AMD EPYC 7713 64-Core Processor 2025-07-24T04:24:31.2716488Z Marketing Name: AMD EPYC 7713 64-Core Processor 2025-07-24T04:24:31.2717075Z Name: gfx90a 2025-07-24T04:24:31.2717643Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:24:31.2718205Z Name: gfx90a 2025-07-24T04:24:31.2718770Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:24:31.2719326Z Name: gfx90a 2025-07-24T04:24:31.2720228Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:24:31.2720800Z Name: gfx90a 2025-07-24T04:24:31.2721348Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:24:31.2721891Z Name: gfx90a 2025-07-24T04:24:31.2722663Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:24:31.2723335Z Name: gfx90a 2025-07-24T04:24:31.2724001Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:24:31.2724656Z Name: gfx90a 2025-07-24T04:24:31.2725318Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:24:31.2725987Z Name: gfx90a 2025-07-24T04:24:31.2726567Z Marketing Name: AMD Instinct MI250X/MI250 2025-07-24T04:24:31.2905956Z + MAYBE_ROCM=rocm/ 2025-07-24T04:24:31.2907424Z + [[ linux-jammy-rocm-py3.10 == *xpu* ]] 2025-07-24T04:24:31.2907955Z + [[ linux-jammy-rocm-py3.10 != *-bazel-* ]] 2025-07-24T04:24:31.2908354Z + pip_install ninja==1.10.2 2025-07-24T04:24:31.2908779Z + pip_install_pkg='python3 -m pip install --progress-bar off' 2025-07-24T04:24:31.2909317Z + python3 -m pip install --progress-bar off ninja==1.10.2 2025-07-24T04:24:31.7076511Z Collecting ninja==1.10.2 2025-07-24T04:24:31.7254846Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2025-07-24T04:24:31.7330583Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2025-07-24T04:24:32.3087348Z Installing collected packages: ninja 2025-07-24T04:24:32.3088405Z Attempting uninstall: ninja 2025-07-24T04:24:32.3095609Z Found existing installation: ninja 1.11.1.3 2025-07-24T04:24:32.3119599Z Uninstalling ninja-1.11.1.3: 2025-07-24T04:24:32.3214785Z Successfully uninstalled ninja-1.11.1.3 2025-07-24T04:24:32.3586980Z Successfully installed ninja-1.10.2 2025-07-24T04:24:32.4357374Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/rocm/llvm/bin:/opt/rocm/opencl/bin:/opt/rocm/hip/bin:/opt/rocm/hcc/bin:/opt/rocm/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2025-07-24T04:24:32.4360528Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/rocm/llvm/bin:/opt/rocm/opencl/bin:/opt/rocm/hip/bin:/opt/rocm/hcc/bin:/opt/rocm/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2025-07-24T04:24:32.4362227Z + [[ linux-jammy-rocm-py3.10 == *aarch64* ]] 2025-07-24T04:24:32.4363124Z + [[ linux-jammy-rocm-py3.10 == *asan* ]] 2025-07-24T04:24:32.4363631Z + [[ linux-jammy-rocm-py3.10 == *-debug* ]] 2025-07-24T04:24:32.4364594Z + [[ linux-jammy-rocm-py3.10 != *-bazel-* ]] 2025-07-24T04:24:32.4365344Z + echo 'We are not in debug mode: linux-jammy-rocm-py3.10. Expect the assertion to pass' 2025-07-24T04:24:32.4366268Z We are not in debug mode: linux-jammy-rocm-py3.10. Expect the assertion to pass 2025-07-24T04:24:32.4367029Z + cd test 2025-07-24T04:24:32.4368476Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2025-07-24T04:24:34.1968356Z + [[ default == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2025-07-24T04:24:34.1969122Z + [[ default == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2025-07-24T04:24:34.1969889Z + [[ default == \l\e\g\a\c\y\_\n\v\i\d\i\a\_\d\r\i\v\e\r ]] 2025-07-24T04:24:34.1972569Z + DYNAMO_BENCHMARK_FLAGS=() 2025-07-24T04:24:34.1973101Z + [[ default == *pr_time_benchmarks* ]] 2025-07-24T04:24:34.1973540Z + [[ default == *dynamo_eager* ]] 2025-07-24T04:24:34.1973920Z + [[ default == *aot_eager* ]] 2025-07-24T04:24:34.1974285Z + [[ default == *aot_inductor* ]] 2025-07-24T04:24:34.1974721Z + [[ default == *max_autotune_inductor* ]] 2025-07-24T04:24:34.1975134Z + [[ default == *inductor* ]] 2025-07-24T04:24:34.1975939Z + [[ default == *dynamic* ]] 2025-07-24T04:24:34.1976282Z + [[ default == *cpu* ]] 2025-07-24T04:24:34.1976662Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2025-07-24T04:24:34.1993582Z + [[ linux-jammy-rocm-py3.10 == *libtorch* ]] 2025-07-24T04:24:34.1993911Z + [[ linux-jammy-rocm-py3.10 == *-bazel-* ]] 2025-07-24T04:24:34.2000690Z + cd test 2025-07-24T04:24:34.2001840Z + python -c 'import torch; print(torch.__config__.show())' 2025-07-24T04:24:35.7062629Z PyTorch built with: 2025-07-24T04:24:35.7063058Z - GCC 11.4 2025-07-24T04:24:35.7063436Z - C++ Version: 201703 2025-07-24T04:24:35.7064270Z - Intel(R) oneAPI Math Kernel Library Version 2024.2-Product Build 20240605 for Intel(R) 64 architecture applications 2025-07-24T04:24:35.7065328Z - Intel(R) MKL-DNN v3.7.1 (Git Hash 8d263e693366ef8db40acc569cc7d8edf644556d) 2025-07-24T04:24:35.7066010Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2025-07-24T04:24:35.7066632Z - LAPACK is enabled (usually provided by MKL) 2025-07-24T04:24:35.7067212Z - NNPACK is enabled 2025-07-24T04:24:35.7067705Z - CPU capability usage: AVX2 2025-07-24T04:24:35.7068232Z - HIP Runtime 6.4.43484 2025-07-24T04:24:35.7068675Z - MIOpen 3.4.0 2025-07-24T04:24:35.7069118Z - Magma 2.7.2 2025-07-24T04:24:35.7073671Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, COMMIT_SHA=27c8ef1839e0747b36769efb192befc9308ff93c, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -DC10_NODEPRECATED -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=range-loop-construct -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -faligned-new -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, TORCH_VERSION=2.9.0, USE_CUDA=OFF, USE_CUDNN=OFF, USE_CUSPARSELT=OFF, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=ON, USE_ROCM_KERNEL_ASSERT=OFF, USE_XCCL=OFF, USE_XPU=OFF, 2025-07-24T04:24:35.7077558Z 2025-07-24T04:24:36.0229121Z + cd test 2025-07-24T04:24:36.0229834Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2025-07-24T04:24:37.2648414Z ATen/Parallel: 2025-07-24T04:24:37.2648878Z at::get_num_threads() : 128 2025-07-24T04:24:37.2649355Z at::get_num_interop_threads() : 128 2025-07-24T04:24:37.2649834Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2025-07-24T04:24:37.2650284Z omp_get_max_threads() : 128 2025-07-24T04:24:37.2651727Z Intel(R) oneAPI Math Kernel Library Version 2024.2-Product Build 20240605 for Intel(R) 64 architecture applications 2025-07-24T04:24:37.2652628Z mkl_get_max_threads() : 128 2025-07-24T04:24:37.2653218Z Intel(R) MKL-DNN v3.7.1 (Git Hash 8d263e693366ef8db40acc569cc7d8edf644556d) 2025-07-24T04:24:37.2653863Z std::thread::hardware_concurrency() : 128 2025-07-24T04:24:37.2654331Z Environment variables: 2025-07-24T04:24:37.2654714Z OMP_NUM_THREADS : [not set] 2025-07-24T04:24:37.2655118Z MKL_NUM_THREADS : [not set] 2025-07-24T04:24:37.2655531Z ATen parallel backend: OpenMP 2025-07-24T04:24:37.2655857Z 2025-07-24T04:24:37.7733779Z + [[ default == *numpy_2* ]] 2025-07-24T04:24:37.7734325Z + [[ linux-jammy-rocm-py3.10 == *aarch64* ]] 2025-07-24T04:24:37.7734830Z + [[ default == *backward* ]] 2025-07-24T04:24:37.7735243Z + [[ default == *xla* ]] 2025-07-24T04:24:37.7735709Z + [[ default == *executorch* ]] 2025-07-24T04:24:37.7736233Z + [[ default == \j\i\t\_\l\e\g\a\c\y ]] 2025-07-24T04:24:37.7736845Z + [[ linux-jammy-rocm-py3.10 == *libtorch* ]] 2025-07-24T04:24:37.7737476Z + [[ default == distributed ]] 2025-07-24T04:24:37.7738034Z + [[ default == *operator_benchmark* ]] 2025-07-24T04:24:37.7738616Z + [[ default == *inductor_distributed* ]] 2025-07-24T04:24:37.7739643Z + [[ default == *inductor-halide* ]] 2025-07-24T04:24:37.7740041Z + [[ default == *inductor-triton-cpu* ]] 2025-07-24T04:24:37.7740427Z + [[ default == *inductor-micro-benchmark* ]] 2025-07-24T04:24:37.7740800Z + [[ default == *huggingface* ]] 2025-07-24T04:24:37.7741109Z + [[ default == *timm* ]] 2025-07-24T04:24:37.7741404Z + [[ default == cachebench ]] 2025-07-24T04:24:37.7741718Z + [[ default == verify_cachebench ]] 2025-07-24T04:24:37.7742060Z + [[ default == *torchbench* ]] 2025-07-24T04:24:37.7742400Z + [[ default == *inductor_cpp_wrapper* ]] 2025-07-24T04:24:37.7742754Z + [[ default == *inductor* ]] 2025-07-24T04:24:37.7743054Z + [[ default == *einops* ]] 2025-07-24T04:24:37.7743361Z + [[ default == *dynamo_wrapped* ]] 2025-07-24T04:24:37.7743722Z + [[ linux-jammy-rocm-py3.10 == *rocm* ]] 2025-07-24T04:24:37.7744692Z + [[ -n test_nn test_torch test_cuda test_ops test_unary_ufuncs test_binary_ufuncs test_autograd inductor/test_torchinductor distributed/test_c10d_common distributed/test_c10d_nccl ]] 2025-07-24T04:24:37.7745652Z + install_torchvision 2025-07-24T04:24:37.7745910Z + local orig_preload 2025-07-24T04:24:37.7746129Z + local commit 2025-07-24T04:24:37.7746352Z ++ get_pinned_commit vision 2025-07-24T04:24:37.7746605Z ++ cat .github/ci_commit_pins/vision.txt 2025-07-24T04:24:37.7762892Z + commit=966da7e46f65d6d49df3e31214470a4fe5cc8e66 2025-07-24T04:24:37.7763220Z + orig_preload= 2025-07-24T04:24:37.7763436Z + '[' -n '' ']' 2025-07-24T04:24:37.7763670Z + [[ linux-jammy-rocm-py3.10 == *cuda* ]] 2025-07-24T04:24:37.7764286Z + pip_build_and_install git+https://github.com/pytorch/vision.git@966da7e46f65d6d49df3e31214470a4fe5cc8e66 dist/vision 2025-07-24T04:24:37.7765077Z + local build_target=git+https://github.com/pytorch/vision.git@966da7e46f65d6d49df3e31214470a4fe5cc8e66 2025-07-24T04:24:37.7765618Z + local wheel_dir=dist/vision 2025-07-24T04:24:37.7766021Z + local found_whl=0 2025-07-24T04:24:37.7766404Z + for file in "${wheel_dir}"/*.whl 2025-07-24T04:24:37.7766871Z + [[ -f dist/vision/*.whl ]] 2025-07-24T04:24:37.7767258Z + '[' 0 == 0 ']' 2025-07-24T04:24:37.7768338Z + python3 -m pip wheel --no-build-isolation --no-deps --no-use-pep517 -w dist/vision git+https://github.com/pytorch/vision.git@966da7e46f65d6d49df3e31214470a4fe5cc8e66 2025-07-24T04:24:38.1254919Z Collecting git+https://github.com/pytorch/vision.git@966da7e46f65d6d49df3e31214470a4fe5cc8e66 2025-07-24T04:24:38.1260688Z Cloning https://github.com/pytorch/vision.git (to revision 966da7e46f65d6d49df3e31214470a4fe5cc8e66) to /tmp/pip-req-build-ctak1qre 2025-07-24T04:24:38.1299516Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-ctak1qre 2025-07-24T04:24:40.4200329Z Running command git rev-parse -q --verify 'sha^966da7e46f65d6d49df3e31214470a4fe5cc8e66' 2025-07-24T04:24:40.4240328Z Running command git fetch -q https://github.com/pytorch/vision.git 966da7e46f65d6d49df3e31214470a4fe5cc8e66 2025-07-24T04:24:40.8818326Z Running command git checkout -q 966da7e46f65d6d49df3e31214470a4fe5cc8e66 2025-07-24T04:24:41.4621599Z Resolved https://github.com/pytorch/vision.git to commit 966da7e46f65d6d49df3e31214470a4fe5cc8e66 2025-07-24T04:24:44.2286724Z Preparing metadata (setup.py) ... [?25l- \ | / - done 2025-07-24T04:24:44.2326667Z [?25hBuilding wheels for collected packages: torchvision 2025-07-24T04:24:44.2412891Z  DEPRECATION: Building 'torchvision' using the legacy setup.py bdist_wheel mechanism, which will be removed in a future version. pip 25.3 will enforce this behaviour change. A possible replacement is to use the standardized build interface by setting the `--use-pep517` option, (possibly combined with `--no-build-isolation`), or adding a `pyproject.toml` file to the source tree of 'torchvision'. Discussion can be found at https://github.com/pypa/pip/issues/6334 2025-07-24T04:25:42.7117723Z  Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - done 2025-07-24T04:25:42.7148792Z [?25h Created wheel for torchvision: filename=torchvision-0.22.0a0+966da7e-cp310-cp310-linux_x86_64.whl size=1570185 sha256=5dde5593d815e7a9f50d725f4d1754dfb9bbfc813bf847e9bb1b2494ac8579eb 2025-07-24T04:25:42.7152301Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/9c/9d/3e/42fa2d5ac6ba44a90363f8fff0fa9e712e24d4f977637c81cb 2025-07-24T04:25:42.7192997Z Successfully built torchvision 2025-07-24T04:25:42.8377270Z + for file in "${wheel_dir}"/*.whl 2025-07-24T04:25:42.8378060Z + pip_install_whl dist/vision/torchvision-0.22.0a0+966da7e-cp310-cp310-linux_x86_64.whl 2025-07-24T04:25:42.8379106Z + args=('dist/vision/torchvision-0.22.0a0+966da7e-cp310-cp310-linux_x86_64.whl') 2025-07-24T04:25:42.8379782Z + local args 2025-07-24T04:25:42.8380407Z + [[ dist/vision/torchvision-0.22.0a0+966da7e-cp310-cp310-linux_x86_64.whl == *\ * ]] 2025-07-24T04:25:42.8381133Z + for path in "${args[@]}" 2025-07-24T04:25:42.8381849Z + echo 'Installing dist/vision/torchvision-0.22.0a0+966da7e-cp310-cp310-linux_x86_64.whl' 2025-07-24T04:25:42.8382848Z Installing dist/vision/torchvision-0.22.0a0+966da7e-cp310-cp310-linux_x86_64.whl 2025-07-24T04:25:42.8383977Z + python3 -mpip install --no-index --no-deps dist/vision/torchvision-0.22.0a0+966da7e-cp310-cp310-linux_x86_64.whl 2025-07-24T04:25:43.2014823Z Processing ./dist/vision/torchvision-0.22.0a0+966da7e-cp310-cp310-linux_x86_64.whl 2025-07-24T04:25:43.2111890Z Installing collected packages: torchvision 2025-07-24T04:25:43.6484160Z Successfully installed torchvision-0.22.0a0+966da7e 2025-07-24T04:25:43.6971821Z + '[' -n '' ']' 2025-07-24T04:25:43.6972221Z + test_python_shard 1 2025-07-24T04:25:43.6972788Z + [[ -z 2 ]] 2025-07-24T04:25:43.6975131Z + python test/run_test.py --exclude-jit-executor --exclude-distributed-tests --include test_nn test_torch test_cuda test_ops test_unary_ufuncs test_binary_ufuncs test_autograd inductor/test_torchinductor distributed/test_c10d_common distributed/test_c10d_nccl --shard 1 2 --verbose --upload-artifacts-while-running 2025-07-24T04:25:46.6484188Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. 2025-07-24T04:25:46.6486500Z import pkg_resources 2025-07-24T04:25:48.7792246Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/pytorch/test/.pytorch-disabled-tests.json 2025-07-24T04:25:48.9557863Z Ignoring disabled issues: [''] 2025-07-24T04:25:48.9817049Z Found test times from artifacts 2025-07-24T04:25:49.0848374Z Found test times from artifacts 2025-07-24T04:25:49.0887824Z Running 25% of tests based on TD 2025-07-24T04:25:49.0893799Z Running parallel tests on 8 processes 2025-07-24T04:25:49.0894481Z Name: tests to run (est. time: 12.09min) 2025-07-24T04:25:49.0894890Z Serial tests (1): 2025-07-24T04:25:49.0895172Z test_torch 1/1 2025-07-24T04:25:49.0895453Z Parallel tests (1): 2025-07-24T04:25:49.0895734Z inductor/test_torchinductor 2/2 2025-07-24T04:25:49.0896027Z Name: excluded (est. time: 35.92min) 2025-07-24T04:25:49.0896283Z Serial tests (1): 2025-07-24T04:25:49.0896482Z test_nn 1/1 2025-07-24T04:25:49.0896693Z Parallel tests (6): 2025-07-24T04:25:49.0896910Z test_ops 2/10 2025-07-24T04:25:49.0897109Z test_ops 5/10 2025-07-24T04:25:49.0897307Z test_ops 6/10 2025-07-24T04:25:49.0897496Z test_ops 9/10 2025-07-24T04:25:49.0897694Z test_ops 10/10 2025-07-24T04:25:49.0897896Z test_cuda 1/1 2025-07-24T04:25:49.0898233Z Running test_torch 1/1 ... [2025-07-24 04:25:49.089714] 2025-07-24T04:25:49.0898566Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2025-07-24T04:25:49.0902366Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_torch.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-07-24 04:25:49.090078] 2025-07-24T04:28:25.8770505Z 2025-07-24T04:28:25.8771271Z test_torch 1/1 was successful, full logs can be found in artifacts with path test/test-reports/test_torch_1.1_c9908f82c76e4368_.log 2025-07-24T04:28:25.9048897Z Running 1068 items in this shard: test/test_torch.py::TestBasicVitalSigns::test_basic_vitals, test/test_torch.py::TestBasicVitalSigns::test_basic_vitals_read_write, test/test_torch.py::TestBasicVitalSigns::test_dataloader_vitals, test/test_torch.py::TestTorch::test_RNGState, test/test_torch.py::TestTorch::test_RNGStateAliasing, test/test_torch.py::TestTorch::test_RNG_after_pickle, test/test_torch.py::TestTorch::test_Size, test/test_torch.py::TestTorch::test_Size_concat_non_tuple_sequence, test/test_torch.py::TestTorch::test_Size_concat_wildcard, test/test_torch.py::TestTorch::test_Size_iter, test/test_torch.py::TestTorch::test_Size_scalar, test/test_torch.py::TestTorch::test_add_meta_scalar, test/test_torch.py::TestTorch::test_allow_tensor_metadata_change, test/test_torch.py::TestTorch::test_apply, test/test_torch.py::TestTorch::test_as_subclass, test/test_torch.py::TestTorch::test_assert_async, test/test_torch.py::TestTorch::test_backward_hooks_traverse, test/test_torch.py::TestTorch::test_batch_norm_cpu_inference, test/test_torch.py::TestTorch::test_bf16_supported_on_cpu, test/test_torch.py::TestTorch::test_bmm_multithreaded, test/test_torch.py::TestTorch::test_boxMullerState, test/test_torch.py::TestTorch::test_cat_neg_dim, test/test_torch.py::TestTorch::test_check, test/test_torch.py::TestTorch::test_chunk_neg_dim, test/test_torch.py::TestTorch::test_conj_neg_tolist, test/test_torch.py::TestTorch::test_conj_physical_meta_stride, test/test_torch.py::TestTorch::test_contains, test/test_torch.py::TestTorch::test_copy_broadcast, test/test_torch.py::TestTorch::test_copy_dtypes, test/test_torch.py::TestTorch::test_copy_float16, test/test_torch.py::TestTorch::test_copy_many_to_one, test/test_torch.py::TestTorch::test_copy_transpose, test/test_torch.py::TestTorch::test_cuda_not_built, test/test_torch.py::TestTorch::test_cummax_neg_dim, test/test_torch.py::TestTorch::test_cummin_neg_dim, test/test_torch.py::TestTorch::test_cumprod_neg_dim, test/test_torch.py::TestTorch::test_cumsum_neg_dim, test/test_torch.py::TestTorch::test_cxx_flags, test/test_torch.py::TestTorch::test_data_ptr_of_empty_tensor_with_storage, test/test_torch.py::TestTorch::test_data_ptr_of_empty_view_with_storage, test/test_torch.py::TestTorch::test_deepcopy_gradient, test/test_torch.py::TestTorch::test_deepcopy_parameter, test/test_torch.py::TestTorch::test_deterministic_fill_uninitialized_memory, test/test_torch.py::TestTorch::test_deterministic_flag, test/test_torch.py::TestTorch::test_device, test/test_torch.py::TestTorch::test_dim_order, test/test_torch.py::TestTorch::test_dir, test/test_torch.py::TestTorch::test_doc, test/test_torch.py::TestTorch::test_doc_template, test/test_torch.py::TestTorch::test_dot_data_use, test/test_torch.py::TestTorch::test_dtype_is_signed, test/test_torch.py::TestTorch::test_element_size, test/test_torch.py::TestTorch::test_empty_meta, test/test_torch.py::TestTorch::test_empty_storage_view, test/test_torch.py::TestTorch::test_equal, test/test_torch.py::TestTorch::test_error_msg_type_translation, test/test_torch.py::TestTorch::test_fill_diagonal, test/test_torch.py::TestTorch::test_format_scalar_meta, test/test_torch.py::TestTorch::test_from_buffer, test/test_torch.py::TestTorch::test_from_file, test/test_torch.py::TestTorch::test_gather_neg_dim, test/test_torch.py::TestTorch::test_generator_cpu, test/test_torch.py::TestTorch::test_get_cpu_capability, test/test_torch.py::TestTorch::test_has_internal_overlap, test/test_torch.py::TestTorch::test_has_storage, test/test_torch.py::TestTorch::test_index_add, test/test_torch.py::TestTorch::test_index_add_all_dtypes, test/test_torch.py::TestTorch::test_index_add_cornercase, test/test_torch.py::TestTorch::test_index_add_correctness, test/test_torch.py::TestTorch::test_index_add_neg_dim, test/test_torch.py::TestTorch::test_index_copy_neg_dim, test/test_torch.py::TestTorch::test_index_fill_neg_dim, test/test_torch.py::TestTorch::test_index_select_neg_dim, test/test_torch.py::TestTorch::test_invalid_arg_error_handling, test/test_torch.py::TestTorch::test_invalid_generator_raises, test/test_torch.py::TestTorch::test_is_nonzero, test/test_torch.py::TestTorch::test_is_same_size, test/test_torch.py::TestTorch::test_iter, test/test_torch.py::TestTorch::test_kthvalue_neg_dim, test/test_torch.py::TestTorch::test_linspace_logspace, test/test_torch.py::TestTorch::test_logcumsumexp_neg_dim, test/test_torch.py::TestTorch::test_manual_seed, test/test_torch.py::TestTorch::test_map, test/test_torch.py::TestTorch::test_map2, test/test_torch.py::TestTorch::test_max_neg_dim, test/test_torch.py::TestTorch::test_mean_neg_dim, test/test_torch.py::TestTorch::test_median_neg_dim, test/test_torch.py::TestTorch::test_memory_format, test/test_torch.py::TestTorch::test_memory_format_contiguous_returns_same_tensor_if_already_satisfies, test/test_torch.py::TestTorch::test_memory_format_empty, test/test_torch.py::TestTorch::test_min_neg_dim, test/test_torch.py::TestTorch::test_mode_neg_dim, test/test_torch.py::TestTorch::test_multinomial_invalid_probs, test/test_torch.py::TestTorch::test_nanmedian_neg_dim, test/test_torch.py::TestTorch::test_narrow_neg_dim, test/test_torch.py::TestTorch::test_nbytes, test/test_torch.py::TestTorch::test_ndim, test/test_torch.py::TestTorch::test_new, test/test_torch.py::TestTorch::test_newaxis_numpy_comparison, test/test_torch.py::TestTorch::test_newindex, test/test_torch.py::TestTorch::test_no_cuda_monkeypatch, test/test_torch.py::TestTorch::test_norm_neg_dim, test/test_torch.py::TestTorch::test_normal_shape, test/test_torch.py::TestTorch::test_numel, test/test_torch.py::TestTorch::test_parallel_info, test/test_torch.py::TestTorch::test_parsing_double, test/test_torch.py::TestTorch::test_parsing_int64, test/test_torch.py::TestTorch::test_parsing_intlist, test/test_torch.py::TestTorch::test_permute, test/test_torch.py::TestTorch::test_pickle, test/test_torch.py::TestTorch::test_pickle_dtype, test/test_torch.py::TestTorch::test_pickle_function, test/test_torch.py::TestTorch::test_pickle_generator, test/test_torch.py::TestTorch::test_pickle_parameter, test/test_torch.py::TestTorch::test_pickle_parameter_no_requires_grad, test/test_torch.py::TestTorch::test_pickle_size, test/test_torch.py::TestTorch::test_pin_memory, test/test_torch.py::TestTorch::test_print, test/test_torch.py::TestTorch::test_prod_neg_dim, test/test_torch.py::TestTorch::test_pyobj_preserved, test/test_torch.py::TestTorch::test_qengine, test/test_torch.py::TestTorch::test_renorm_neg_dim, test/test_torch.py::TestTorch::test_resizable, test/test_torch.py::TestTorch::test_reversed, test/test_torch.py::TestTorch::test_scatter_neg_dim, test/test_torch.py::TestTorch::test_select_neg_dim, test/test_torch.py::TestTorch::test_set_flush_denormal, test/test_torch.py::TestTorch::test_setting_real_imag_to_a_number, test/test_torch.py::TestTorch::test_show_config, test/test_torch.py::TestTorch::test_size_neg_dim, test/test_torch.py::TestTorch::test_size_stride, test/test_torch.py::TestTorch::test_sizeof, test/test_torch.py::TestTorch::test_slice, test/test_torch.py::TestTorch::test_slow_test, test/test_torch.py::TestTorch::test_sobolengine_bounds, test/test_torch.py::TestTorch::test_sobolengine_bounds_scrambled, test/test_torch.py::TestTorch::test_sobolengine_continuing, test/test_torch.py::TestTorch::test_sobolengine_continuing_scrambled, test/test_torch.py::TestTorch::test_sobolengine_default_dtype, test/test_torch.py::TestTorch::test_sobolengine_distribution, test/test_torch.py::TestTorch::test_sobolengine_distribution_scrambled, test/test_torch.py::TestTorch::test_sobolengine_draw, test/test_torch.py::TestTorch::test_sobolengine_draw_base2, test/test_torch.py::TestTorch::test_sobolengine_draw_base2_scrambled, test/test_torch.py::TestTorch::test_sobolengine_draw_scrambled, test/test_torch.py::TestTorch::test_sobolengine_fast_forward, test/test_torch.py::TestTorch::test_sobolengine_fast_forward_scrambled, test/test_torch.py::TestTorch::test_sobolengine_first_point, test/test_torch.py::TestTorch::test_sobolengine_high_dim, test/test_torch.py::TestTorch::test_sobolengine_raise, test/test_torch.py::TestTorch::test_sobolengine_reset, test/test_torch.py::TestTorch::test_sobolengine_reset_scrambled, test/test_torch.py::TestTorch::test_sort_neg_dim, test/test_torch.py::TestTorch::test_split_neg_dim, test/test_torch.py::TestTorch::test_split_with_sizes_copy_out, test/test_torch.py::TestTorch::test_squeeze_neg_dim, test/test_torch.py::TestTorch::test_std_neg_dim, test/test_torch.py::TestTorch::test_storage_base_init, test/test_torch.py::TestTorch::test_storage_base_new, test/test_torch.py::TestTorch::test_storage_byteswap, test/test_torch.py::TestTorch::test_storage_casts, test/test_torch.py::TestTorch::test_storage_cycle_via_dict, test/test_torch.py::TestTorch::test_storage_cycle_via_slots, test/test_torch.py::TestTorch::test_storage_dead_weak_ref, test/test_torch.py::TestTorch::test_storage_dealloc, test/test_torch.py::TestTorch::test_storage_dealloc_resurrected, test/test_torch.py::TestTorch::test_storage_dealloc_subclass_resurrected, test/test_torch.py::TestTorch::test_storage_dealloc_subclass_zombie, test/test_torch.py::TestTorch::test_storage_dict_dealloc, test/test_torch.py::TestTorch::test_storage_error, test/test_torch.py::TestTorch::test_storage_error_no_attribute, test/test_torch.py::TestTorch::test_storage_finalizer_dealloc, test/test_torch.py::TestTorch::test_storage_fix_weakref_no_leak, test/test_torch.py::TestTorch::test_storage_from_tensor_dealloc, test/test_torch.py::TestTorch::test_storage_from_tensor_dealloc_resurrected, test/test_torch.py::TestTorch::test_storage_from_tensor_dealloc_zombie, test/test_torch.py::TestTorch::test_storage_preserve_nonhermetic_in_hermetic_context, test/test_torch.py::TestTorch::test_storage_resurrected_weak_ref, test/test_torch.py::TestTorch::test_storage_slot_dealloc, test/test_torch.py::TestTorch::test_storage_weakref_dealloc, test/test_torch.py::TestTorch::test_structseq_repr, test/test_torch.py::TestTorch::test_subclass_preserved, test/test_torch.py::TestTorch::test_subclass_tensors, test/test_torch.py::TestTorch::test_sum_neg_dim, test/test_torch.py::TestTorch::test_swap_basic, test/test_torch.py::TestTorch::test_swap_fail_slots, test/test_torch.py::TestTorch::test_t_not_2d_error, test/test_torch.py::TestTorch::test_tensor_base_init, test/test_torch.py::TestTorch::test_tensor_base_new, test/test_torch.py::TestTorch::test_tensor_ctor_scalar, test/test_torch.py::TestTorch::test_tensor_cycle_via_dict, test/test_torch.py::TestTorch::test_tensor_cycle_via_slots, test/test_torch.py::TestTorch::test_tensor_dead_weak_ref, test/test_torch.py::TestTorch::test_tensor_dict_dealloc, test/test_torch.py::TestTorch::test_tensor_finalizer_dealloc, test/test_torch.py::TestTorch::test_tensor_fix_weakref_no_leak, test/test_torch.py::TestTorch::test_tensor_item_no_warning, test/test_torch.py::TestTorch::test_tensor_ressurecting_clear, test/test_torch.py::TestTorch::test_tensor_resurrected_weak_ref, test/test_torch.py::TestTorch::test_tensor_set, test/test_torch.py::TestTorch::test_tensor_set_errors, test/test_torch.py::TestTorch::test_tensor_slot_dealloc, test/test_torch.py::TestTorch::test_tensor_weakref_dealloc, test/test_torch.py::TestTorch::test_tensor_where_scalar, test/test_torch.py::TestTorch::test_tensor_with_grad_to_scalar_warning, test/test_torch.py::TestTorch::test_tensoriterator_output_setup, test/test_torch.py::TestTorch::test_terminate_handler_on_crash, test/test_torch.py::TestTorch::test_to, test/test_torch.py::TestTorch::test_to_with_tensor, test/test_torch.py::TestTorch::test_topk_neg_dim, test/test_torch.py::TestTorch::test_torch_from_file, test/test_torch.py::TestTorch::test_transpose_neg_dim, test/test_torch.py::TestTorch::test_type, test/test_torch.py::TestTorch::test_type_alias, test/test_torch.py::TestTorch::test_type_conversion_via_dtype_name, test/test_torch.py::TestTorch::test_typed_storage_deprecation_warning, test/test_torch.py::TestTorch::test_typed_storage_internal_no_warning, test/test_torch.py::TestTorch::test_unbind_neg_dim, test/test_torch.py::TestTorch::test_unflatten, test/test_torch.py::TestTorch::test_unfold_neg_dim, test/test_torch.py::TestTorch::test_unsqueeze_neg_dim, test/test_torch.py::TestTorch::test_upsample_nearest1d_meta, test/test_torch.py::TestTorch::test_upsample_nearest2d_meta, test/test_torch.py::TestTorch::test_var_neg_dim, test/test_torch.py::TestTorch::test_warn_types, test/test_torch.py::TestTorch::test_wildcard_import, test/test_torch.py::TestVitalSignsCudaCUDA::test_cuda_vitals_gpu_only_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test__local_scalar_dense_with_empty_tensor_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_cuda_errors_with_cpu_scalars_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_use_cpu_scalar_False_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_use_cpu_scalar_False_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_use_cpu_scalar_False_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_use_cpu_scalar_False_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_use_cpu_scalar_False_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_use_cpu_scalar_False_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_use_cpu_scalar_False_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_use_cpu_scalar_False_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_use_cpu_scalar_False_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_use_cpu_scalar_False_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_use_cpu_scalar_True_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_use_cpu_scalar_True_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_use_cpu_scalar_True_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_use_cpu_scalar_True_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_use_cpu_scalar_True_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_use_cpu_scalar_True_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_use_cpu_scalar_True_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_use_cpu_scalar_True_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_use_cpu_scalar_True_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_use_cpu_scalar_True_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_assertRaisesRegex_ignore_msg_non_native_device_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_edge_cases_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_edge_cases_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_edge_cases_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_p_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_p_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_p_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bfloat16_neg_abs_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bool_tensor_value_change_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_add_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_addcdiv_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_addcmul_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_atan2_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_copy_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_dist_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_div_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_eq_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_fmod_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_ge_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_gt_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_le_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_lerp_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_lt_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_map2_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_map_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_masked_fill_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_masked_scatter_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_masked_select_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_max_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_min_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_mul_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_ne_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_pow_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_remainder_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_sub_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_uint16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_uint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_uint64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cauchy_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cauchy_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cauchy_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cauchy_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cauchy_kstest_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cauchy_no_inf_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cauchy_no_inf_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_cuda_backward_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_euclidean_large_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_grad_p_lt_1_no_nan_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_large_batch_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_large_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_non_contiguous_batch_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_non_contiguous_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_norm_batch_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_norm_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_same_inputs_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_check_tensor_all_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_check_tensor_internal_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_clone_all_dtypes_and_devices_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_clone_not_memory_dense_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_clone_zero_stride_dim_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_complex_half_experimental_warning_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_constants_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_conv_transposed_backward_agnostic_to_memory_format_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_conv_transposed_large_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_complex32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy_all_dtypes_and_devices_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy_math_view_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy_mem_overlap_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy_transpose_math_view_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy_transpose_math_view_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy_transpose_math_view_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_corrcoef_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_corrcoef_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_corrcoef_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cov_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cpp_warnings_have_python_context_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cublas_config_nondeterministic_alert_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cummax_cummin_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cummax_discontiguous_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cummin_discontiguous_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cumprod_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cumsum_64bit_indexing_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cumsum_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deepcopy_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deepcopy_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deepcopy_scalar_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deepcopy_scalar_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_cumsum_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_complex32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_uint16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_uint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_uint64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_interpolate_bilinear_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_replication_pad2d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_uint16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_uint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_uint64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_device_guard_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_dim_function_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_discontiguous_out_cumsum_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_dist_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_dtypetensor_warnings_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_errors_index_copy_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_expected_failure_xla_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_kstest_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_kstest_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_kstest_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_kstest_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_no_zero_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_no_zero_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gather_backward_deterministic_path_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gather_backward_one_dim_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_kstest_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scale_will_not_overflow_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaler_deprecated_warning_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaler_pass_itself_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_accumulation_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach0_fused0_AdamW_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach0_fused0_Adam_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach0_fused0_SGD_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach2_fused_True_AdamW_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach2_fused_True_Adam_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach2_fused_True_SGD_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach_True_fused1_AdamW_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach_True_fused1_Adam_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach_True_fused1_SGD_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_clipping_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_clipping_separate_unscale_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_multiple_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_penalty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_state_dict_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_unscale_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_unscale_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_unscale_sparse_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_update_scale_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_all_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_all_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_all_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_extreme_cases_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_extreme_cases_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_extreme_cases_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_spacing_list_length_error_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_spacing_list_length_error_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_spacing_list_length_error_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_type_promotion_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_hook_remove_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_add_deterministic_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_add_large_inputs_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_add_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_deterministic_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_put_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_put_non_accumulate_deterministic_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_float8_e4m3fn, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_float8_e4m3fnuz, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_float8_e5m2, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_float8_e5m2fnuz, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_int64_upsample3d_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_invalid_shapes_grid_sampler_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_is_set_to_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_is_signed_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_complex32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_float8_e4m3fn, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_float8_e4m3fnuz, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_float8_e5m2, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_float8_e5m2fnuz, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_uint16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_uint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_uint64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_large_cumprod_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_large_cumsum_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_log_normal_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_log_normal_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_log_normal_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_log_normal_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_logcumsumexp_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lognormal_kstest_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_bool_tensor_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_bfloat16_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_bfloat16_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_bool_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_bool_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_complex128_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_complex128_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_complex64_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_complex64_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_float16_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_float16_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_float32_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_float32_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_float64_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_float64_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int16_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int16_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int32_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int32_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int64_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int64_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int8_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int8_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_uint8_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_uint8_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_bool_tensor_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_inplace_noncontiguous_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_large_tensor_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_discontiguous_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_clone_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_consistency_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_cpu_and_cuda_ops_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_empty_like_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_factory_like_functions_preserve_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_operators_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_preserved_after_permute_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_propagation_rules_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_to_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_type_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_type_shortcuts_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_module_share_memory_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_cpu_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_cpu_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_cpu_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_deterministic_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_deterministic_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_deterministic_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_device_constrain_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_empty_w_replacement_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_empty_wo_replacement_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_gpu_device_constrain_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_rng_state_advance_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_narrow_copy_non_contiguous_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_narrow_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_no_nondeterministic_alert_interpolate_bilinear_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_no_nondeterministic_alert_interpolate_trilinear_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_AdaptiveAvgPool2d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_AdaptiveAvgPool3d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_AdaptiveMaxPool2d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_AvgPool3d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_CTCLoss_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_EmbeddingBag_max_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_FractionalMaxPool2d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_FractionalMaxPool3d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxPool3d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool1d_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool1d_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool1d_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool2d_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool2d_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool2d_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool3d_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool3d_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool3d_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_NLLLoss_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_ReflectionPad1d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_ReflectionPad3d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_ReplicationPad1d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_ReplicationPad2d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_ReplicationPad3d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_bincount_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_grid_sample_2d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_grid_sample_3d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_histc_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_histc_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_interpolate_bicubic_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_interpolate_bilinear_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_interpolate_linear_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_interpolate_trilinear_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_kthvalue_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_median_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_put_accumulate_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_put_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_resize_quantized_cuda_qint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_resize_quantized_cuda_qint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_resize_quantized_cuda_quint2x4, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_resize_quantized_cuda_quint4x2, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_resize_quantized_cuda_quint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_normal_kstest_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_normal_kstest_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_normal_kstest_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_normal_kstest_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nullary_op_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_pairwise_distance_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_params_invalidated_with_grads_invalidated_and_graph_partition_AdamW_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_params_invalidated_with_grads_invalidated_and_graph_partition_Adam_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_params_invalidated_with_grads_invalidated_and_graph_partition_SGD_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_params_invalidated_with_grads_invalidated_between_unscale_and_step_AdamW_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_params_invalidated_with_grads_invalidated_between_unscale_and_step_Adam_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_params_invalidated_with_grads_invalidated_between_unscale_and_step_SGD_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_pdist_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_pdist_norm_large_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_pickle_gradscaler_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_pin_memory_from_constructor_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_reduced_type_float_copy_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_reduced_type_float_copy_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_repeat_interleave_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scalar_check_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_add_bool_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_add_non_unique_index_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_add_one_dim_deterministic_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_add_to_large_input_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_bool_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_multiply_unsupported_dtypes_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_multiply_unsupported_dtypes_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_to_large_input_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_zero_size_index_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_serialization_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_default_tensor_type_warnings_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_shift_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_skip_xla_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_all_devices_non_blocking_False_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_all_devices_non_blocking_True_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_uint16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_uint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_uint64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_qint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_qint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_quint4x2, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_quint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_use_count_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_strides_propagation_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_sync_warning_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_uint16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_uint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_uint64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_set_errors_multigpu_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_shape_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_type_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_ternary_op_mem_overlap_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_uniform_kstest_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_uniform_kstest_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_uniform_kstest_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_uniform_kstest_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_untyped_storage_meta_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_warn_always_caught_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_where_scalar_handcrafted_values_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_advancedindex_mixed_cpu_devices_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_advancedindex_mixed_devices_error_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_clamp_cuda_float32, test/test_torch.py::TestDevicePrecisionCUDA::test_clamp_cuda_float64, test/test_torch.py::TestDevicePrecisionCUDA::test_clamp_cuda_int64, test/test_torch.py::TestDevicePrecisionCUDA::test_copy_broadcast_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_copy_noncontig_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_cuda_device_idx_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_device_serialization_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_float16, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_float32, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_float64, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_int16, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_int32, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_int64, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_int8, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_uint8, test/test_torch.py::TestDevicePrecisionCUDA::test_index_add_bfloat16_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_multidevice_serialization_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_type_conversions_same_device_cuda 2025-07-24T04:28:25.9309881Z 2025-07-24T04:28:25.9310118Z GITHUB_RUN_ID, GITHUB_RUN_ATTEMPT, or ARTIFACTS_FILE_SUFFIX not set, not uploading 2025-07-24T04:28:25.9310555Z Uploading artifacts took 0.00 seconds 2025-07-24T04:28:25.9310939Z Running inductor/test_torchinductor 2/2 ... [2025-07-24 04:28:25.878471] 2025-07-24T04:28:25.9311332Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2025-07-24T04:28:25.9312273Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_torchinductor.py', '-m', 'serial', '--shard-id=2', '--num-shards=2', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-07-24 04:28:25.878822] 2025-07-24T04:28:35.3070734Z 2025-07-24T04:28:35.3072243Z inductor/test_torchinductor 2/2 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_torchinductor_2.2_b32212974daf9b4d_.log 2025-07-24T04:28:35.3073721Z Running 0 items in this shard: 2025-07-24T04:28:35.3074066Z 2025-07-24T04:28:38.9312088Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. 2025-07-24T04:28:38.9314286Z import pkg_resources 2025-07-24T04:28:38.9535599Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. 2025-07-24T04:28:38.9537134Z import pkg_resources 2025-07-24T04:28:38.9668447Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. 2025-07-24T04:28:38.9670452Z import pkg_resources 2025-07-24T04:28:38.9680995Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. 2025-07-24T04:28:38.9682545Z import pkg_resources 2025-07-24T04:28:38.9719294Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. 2025-07-24T04:28:38.9721360Z import pkg_resources 2025-07-24T04:28:38.9822277Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. 2025-07-24T04:28:38.9824331Z import pkg_resources 2025-07-24T04:28:38.9858899Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. 2025-07-24T04:28:38.9860553Z import pkg_resources 2025-07-24T04:28:38.9907610Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. 2025-07-24T04:28:38.9910037Z import pkg_resources 2025-07-24T04:28:39.5502338Z Running inductor/test_torchinductor 2/2 ... [2025-07-24 04:28:39.550004] 2025-07-24T04:28:39.5502855Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2025-07-24T04:28:39.5505976Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_torchinductor.py', '-m', 'not serial', '--shard-id=2', '--num-shards=2', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-07-24 04:28:39.550379] 2025-07-24T04:37:32.5754258Z 2025-07-24T04:37:32.5755527Z inductor/test_torchinductor 2/2 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_torchinductor_2.2_cb602b025e6e559d_.log 2025-07-24T04:37:32.5888948Z Running 500 items in this shard: test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast1_broadcast1, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast1_broadcast3, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast1_dense, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast1_double, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast1_int, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast2_broadcast1, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast2_broadcast3, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast2_dense, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast2_strided, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast3_broadcast1, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast3_broadcast2, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast3_broadcast3, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast3_double, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast3_strided, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast3_transposed, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_dense_broadcast2, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_dense_int, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_dense_transposed, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_double_broadcast1, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_double_broadcast3, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_double_dense, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_int_broadcast1, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_int_broadcast2, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_int_dense, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_int_double, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_int_int, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_strided_broadcast2, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_strided_dense, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_strided_double, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_strided_int, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_transposed_broadcast1, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_transposed_broadcast3, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_transposed_dense, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_transposed_strided, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_transposed_transposed, test/inductor/test_torchinductor.py::GPUTests::test__dyn_quant_matmul_4bit_cuda, test/inductor/test_torchinductor.py::GPUTests::test__unsafe_masked_index_cuda, test/inductor/test_torchinductor.py::GPUTests::test_adaptive_avg_pool2d2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_adaptive_avg_pool2d_low_prec_cuda, test/inductor/test_torchinductor.py::GPUTests::test_adaptive_avg_pool_errors_with_long_cuda, test/inductor/test_torchinductor.py::GPUTests::test_adaptive_max_pool2d2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_add_complex3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_add_complex_cuda, test/inductor/test_torchinductor.py::GPUTests::test_add_const_int_cuda, test/inductor/test_torchinductor.py::GPUTests::test_adding_tensor_offsets_cuda, test/inductor/test_torchinductor.py::GPUTests::test_addmv_cuda, test/inductor/test_torchinductor.py::GPUTests::test_aliased_buffer_reuse_cuda, test/inductor/test_torchinductor.py::GPUTests::test_aoti_eager_cache_hit_cuda, test/inductor/test_torchinductor.py::GPUTests::test_aoti_eager_dtype_device_layout_cuda, test/inductor/test_torchinductor.py::GPUTests::test_aoti_eager_override_registration_cuda, test/inductor/test_torchinductor.py::GPUTests::test_aoti_eager_support_out_cuda, test/inductor/test_torchinductor.py::GPUTests::test_aoti_eager_support_str_cuda, test/inductor/test_torchinductor.py::GPUTests::test_arange3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_arange4_cuda, test/inductor/test_torchinductor.py::GPUTests::test_arange5_cuda, test/inductor/test_torchinductor.py::GPUTests::test_argmax_argmin3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_argmax_argmin_with_duplicates_cuda, test/inductor/test_torchinductor.py::GPUTests::test_argmax_argmin_with_nan_cuda, test/inductor/test_torchinductor.py::GPUTests::test_argmax_min_int32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_argmax_to_float_cuda, test/inductor/test_torchinductor.py::GPUTests::test_as_strided_scatter_cuda, test/inductor/test_torchinductor.py::GPUTests::test_assert_alignment_op_name_pass_cuda, test/inductor/test_torchinductor.py::GPUTests::test_avg_pool2d1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_avg_pool2d2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_avg_pool2d3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_avg_pool2d5_cuda, test/inductor/test_torchinductor.py::GPUTests::test_avg_pool2d_backward2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_avg_pool2d_backward3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_avg_pool2d_backward4_cuda, test/inductor/test_torchinductor.py::GPUTests::test_avg_pool2d_backward_cuda, test/inductor/test_torchinductor.py::GPUTests::test_avg_pool3d_backward2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_avg_pool3d_backward3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_avg_pool3d_backward_cuda, test/inductor/test_torchinductor.py::GPUTests::test_avg_pool_errors_with_uint_cuda, test/inductor/test_torchinductor.py::GPUTests::test_baddbmm_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bernoulli1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bernoulli2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bfloat16_to_int16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bitwise3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bitwise_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bmm1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bool_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_broadcast_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_int16_int16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_int16_int32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_int16_int64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_int16_int8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_int16_uint8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_int32_int64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_int32_int8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_int8_int16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_int8_int32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_int8_int64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_uint8_int16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_uint8_uint8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_nd_tiling_False_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_nd_tiling_True_cuda, test/inductor/test_torchinductor.py::GPUTests::test_buffer_batch_norm_cuda, test/inductor/test_torchinductor.py::GPUTests::test_buffer_copied_in_graph_cuda, test/inductor/test_torchinductor.py::GPUTests::test_buffer_copied_in_graph_with_different_shapes_cuda, test/inductor/test_torchinductor.py::GPUTests::test_buffer_use_after_remove_cuda, test/inductor/test_torchinductor.py::GPUTests::test_builtins_round_cuda, test/inductor/test_torchinductor.py::GPUTests::test_builtins_round_float_ndigits_pos_cuda, test/inductor/test_torchinductor.py::GPUTests::test_builtins_round_float_ndigits_zero_cuda, test/inductor/test_torchinductor.py::GPUTests::test_builtins_round_int_ndigits_zero_cuda, test/inductor/test_torchinductor.py::GPUTests::test_cat_extern_kernel_cuda, test/inductor/test_torchinductor.py::GPUTests::test_cat_inplace_cuda, test/inductor/test_torchinductor.py::GPUTests::test_cat_negative_dim_cuda, test/inductor/test_torchinductor.py::GPUTests::test_cat_of_loops_and_extern_kernel_cuda, test/inductor/test_torchinductor.py::GPUTests::test_cat_uint8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_cat_unbacked_2d_cuda, test/inductor/test_torchinductor.py::GPUTests::test_cat_unbacked_legacy_empty_cuda, test/inductor/test_torchinductor.py::GPUTests::test_cat_upcasting_cuda, test/inductor/test_torchinductor.py::GPUTests::test_check_stack_no_cycles_cuda, test/inductor/test_torchinductor.py::GPUTests::test_clamp_cuda, test/inductor/test_torchinductor.py::GPUTests::test_config_option_dont_assume_alignment_cuda, test/inductor/test_torchinductor.py::GPUTests::test_consecutive_split_cumsum_cuda, test/inductor/test_torchinductor.py::GPUTests::test_constant_pad_1d_cuda, test/inductor/test_torchinductor.py::GPUTests::test_constant_pad_2d_cuda, test/inductor/test_torchinductor.py::GPUTests::test_constant_pad_float64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_conv3d_channels_last_use_block_ptr_True_cuda, test/inductor/test_torchinductor.py::GPUTests::test_conv_backward_cuda, test/inductor/test_torchinductor.py::GPUTests::test_conv_bn_fuse_cuda, test/inductor/test_torchinductor.py::GPUTests::test_conv_functional_bn_fuse_cuda, test/inductor/test_torchinductor.py::GPUTests::test_conv_inference_heuristics_cuda, test/inductor/test_torchinductor.py::GPUTests::test_conv_with_as_strided_cuda, test/inductor/test_torchinductor.py::GPUTests::test_convolution2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_convolution4_cuda, test/inductor/test_torchinductor.py::GPUTests::test_convolution5_cuda, test/inductor/test_torchinductor.py::GPUTests::test_cos_cuda, test/inductor/test_torchinductor.py::GPUTests::test_cumsum_cuda, test/inductor/test_torchinductor.py::GPUTests::test_cumsum_no_mask_cuda, test/inductor/test_torchinductor.py::GPUTests::test_cumsum_zero_dim_cuda, test/inductor/test_torchinductor.py::GPUTests::test_custom_op_1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_custom_op_2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_custom_op_default_layout_constraint_cuda, test/inductor/test_torchinductor.py::GPUTests::test_custom_op_fixed_layout_channels_last_cuda, test/inductor/test_torchinductor.py::GPUTests::test_custom_op_fixed_layout_sequential_cuda, test/inductor/test_torchinductor.py::GPUTests::test_custom_scan_op_compiled_cuda, test/inductor/test_torchinductor.py::GPUTests::test_custom_scan_op_cuda, test/inductor/test_torchinductor.py::GPUTests::test_custom_scan_op_multi_input_cuda, test/inductor/test_torchinductor.py::GPUTests::test_custom_scan_would_split_cuda, test/inductor/test_torchinductor.py::GPUTests::test_deterministic_codegen_cuda, test/inductor/test_torchinductor.py::GPUTests::test_diagonal_copy_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dist_bf16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dist_cuda, test/inductor/test_torchinductor.py::GPUTests::test_div5_cuda, test/inductor/test_torchinductor.py::GPUTests::test_div6_cuda, test/inductor/test_torchinductor.py::GPUTests::test_div8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_div9_cuda, test/inductor/test_torchinductor.py::GPUTests::test_div_prim_cuda, test/inductor/test_torchinductor.py::GPUTests::test_div_softmax_symfloat_cuda, test/inductor/test_torchinductor.py::GPUTests::test_div_zero_dim_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dropout2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dropout_deterministic_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dropout_trivial_0_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtype_sympy_expr_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_bfloat16_bfloat16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_bfloat16_float16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_bfloat16_uint8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float16_float16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float16_float64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float16_int32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float16_int64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float16_uint8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float32_float16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float32_float32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float32_int32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float32_int8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float32_uint8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float64_float16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float64_int16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int16_float16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int16_float32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int16_int32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int16_int64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int16_int8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int32_float16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int32_float32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int32_float64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int32_int16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int32_int32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int32_int64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int32_int8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int64_float32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int64_float64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int64_int32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int64_int64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int64_int8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int64_uint8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int8_bfloat16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int8_float32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int8_float64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int8_int16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int8_int32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int8_int64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_uint8_bfloat16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_uint8_float16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_uint8_float32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_uint8_float64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_uint8_int32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_embedding_cuda, test/inductor/test_torchinductor.py::GPUTests::test_empty1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_empty2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_erfc_cuda, test/inductor/test_torchinductor.py::GPUTests::test_erfinv_cuda, test/inductor/test_torchinductor.py::GPUTests::test_exact_stride_cuda, test/inductor/test_torchinductor.py::GPUTests::test_expm1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_fallback_mutable_op_list_cuda, test/inductor/test_torchinductor.py::GPUTests::test_fallback_mutable_op_no_mutated_tensors_cuda, test/inductor/test_torchinductor.py::GPUTests::test_fft_real_input_cuda, test/inductor/test_torchinductor.py::GPUTests::test_fill2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_flip_cat_cuda, test/inductor/test_torchinductor.py::GPUTests::test_flip_cuda, test/inductor/test_torchinductor.py::GPUTests::test_float_index_expression_cuda, test/inductor/test_torchinductor.py::GPUTests::test_float_repr_dynamic_shapes_cuda, test/inductor/test_torchinductor.py::GPUTests::test_fmin_fmax_cuda, test/inductor/test_torchinductor.py::GPUTests::test_forced_buffer_realize_cuda, test/inductor/test_torchinductor.py::GPUTests::test_fractional_max_pool2d1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_fractional_max_pool2d2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_fractional_max_pool2d3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_fractional_max_pool2d4_cuda, test/inductor/test_torchinductor.py::GPUTests::test_fractional_max_pool2d5_cuda, test/inductor/test_torchinductor.py::GPUTests::test_fuse_tiled_cuda, test/inductor/test_torchinductor.py::GPUTests::test_gather2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_gather_scatter_cuda, test/inductor/test_torchinductor.py::GPUTests::test_getitem_cuda, test/inductor/test_torchinductor.py::GPUTests::test_graph_partition_arange1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_graph_partition_constant_tensor2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_graph_partition_misaligned_input_cuda, test/inductor/test_torchinductor.py::GPUTests::test_graph_partition_no_inputs_cuda, test/inductor/test_torchinductor.py::GPUTests::test_graph_partition_unbacked_symint_as_output_cuda, test/inductor/test_torchinductor.py::GPUTests::test_hardsigmoid_cuda, test/inductor/test_torchinductor.py::GPUTests::test_hardswish_cuda, test/inductor/test_torchinductor.py::GPUTests::test_horizonal_fusion2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index_dynamic_shapes_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index_propagation_abs_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index_propagation_device_assert_masked_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index_propagation_nested_indirect_indexing_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index_put1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index_put2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index_put4_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index_put_deterministic_fallback_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index_put_index_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index_put_reinplace_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index_remainder_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index_select_cuda, test/inductor/test_torchinductor.py::GPUTests::test_inductor_multiple_specializations_cuda, test/inductor/test_torchinductor.py::GPUTests::test_inf_cuda, test/inductor/test_torchinductor.py::GPUTests::test_inner_fn_str_and_stride_cuda, test/inductor/test_torchinductor.py::GPUTests::test_inplace_activations_cuda, test/inductor/test_torchinductor.py::GPUTests::test_inplace_add_cuda, test/inductor/test_torchinductor.py::GPUTests::test_input_mutation1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_input_mutation3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_input_mutation4_cuda, test/inductor/test_torchinductor.py::GPUTests::test_input_mutation5_cuda, test/inductor/test_torchinductor.py::GPUTests::test_insignificant_strides_cuda, test/inductor/test_torchinductor.py::GPUTests::test_isinf_cuda, test/inductor/test_torchinductor.py::GPUTests::test_issue102546_cuda, test/inductor/test_torchinductor.py::GPUTests::test_kernel_names_cuda, test/inductor/test_torchinductor.py::GPUTests::test_kwargs_cuda, test/inductor/test_torchinductor.py::GPUTests::test_l1_loss_cuda, test/inductor/test_torchinductor.py::GPUTests::test_large_broadcast_reduction_cuda, test/inductor/test_torchinductor.py::GPUTests::test_large_grid_use_block_ptr_True_cuda, test/inductor/test_torchinductor.py::GPUTests::test_large_pointwise_cuda, test/inductor/test_torchinductor.py::GPUTests::test_large_strided_reduction_cuda, test/inductor/test_torchinductor.py::GPUTests::test_large_tensor_reduction_cuda, test/inductor/test_torchinductor.py::GPUTests::test_leaky_relu_cuda, test/inductor/test_torchinductor.py::GPUTests::test_lgamma_cuda, test/inductor/test_torchinductor.py::GPUTests::test_like_rands3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_linear1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_linear2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_linear_mixed_dtype_cuda, test/inductor/test_torchinductor.py::GPUTests::test_linspace1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_linspace2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_linspace4_cuda, test/inductor/test_torchinductor.py::GPUTests::test_log_fp64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_log_softmax_cuda, test/inductor/test_torchinductor.py::GPUTests::test_logaddexp_cuda, test/inductor/test_torchinductor.py::GPUTests::test_logcumsumexp_cuda, test/inductor/test_torchinductor.py::GPUTests::test_logcumsumexp_zero_dim_cuda, test/inductor/test_torchinductor.py::GPUTests::test_low_memory_max_pool_dilation_2_dim_2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_low_memory_max_pool_dilation_2_dim_3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_masked_scatter_cuda, test/inductor/test_torchinductor.py::GPUTests::test_matmul_layer_norm_cuda, test/inductor/test_torchinductor.py::GPUTests::test_max_min_cuda, test/inductor/test_torchinductor.py::GPUTests::test_max_pool2d3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_max_pool2d5_cuda, test/inductor/test_torchinductor.py::GPUTests::test_max_pool2d_with_indices_backward2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_max_pool2d_with_indices_backward3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_max_pool2d_with_indices_backward4_cuda, test/inductor/test_torchinductor.py::GPUTests::test_max_pool2d_with_indices_backward6_cuda, test/inductor/test_torchinductor.py::GPUTests::test_min_max_reduction_nan_cuda, test/inductor/test_torchinductor.py::GPUTests::test_mix_device_index_cuda, test/inductor/test_torchinductor.py::GPUTests::test_mixed_mm2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_mixed_mm3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_mixed_mm_cuda, test/inductor/test_torchinductor.py::GPUTests::test_mm_mixed_dtype_cuda, test/inductor/test_torchinductor.py::GPUTests::test_mm_views_cuda, test/inductor/test_torchinductor.py::GPUTests::test_mul_index_expr_cuda, test/inductor/test_torchinductor.py::GPUTests::test_mul_softmax_symfloat_cuda, test/inductor/test_torchinductor.py::GPUTests::test_multi_gpu_device_cuda, test/inductor/test_torchinductor.py::GPUTests::test_multi_gpu_recompile_on_index_cuda, test/inductor/test_torchinductor.py::GPUTests::test_multilayer_var_cuda, test/inductor/test_torchinductor.py::GPUTests::test_multilayer_var_lowp_cuda, test/inductor/test_torchinductor.py::GPUTests::test_mutable_custom_op_fixed_layout2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_mutable_custom_op_fixed_layout_cuda, test/inductor/test_torchinductor.py::GPUTests::test_nan_to_num_cuda, test/inductor/test_torchinductor.py::GPUTests::test_neg_index_cuda, test/inductor/test_torchinductor.py::GPUTests::test_neg_max_uint8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_nll_loss_backward_cuda, test/inductor/test_torchinductor.py::GPUTests::test_nll_loss_forward_cuda, test/inductor/test_torchinductor.py::GPUTests::test_no_mega_fusion_during_lowering_cuda, test/inductor/test_torchinductor.py::GPUTests::test_no_op_reduction_cuda, test/inductor/test_torchinductor.py::GPUTests::test_no_specization_over_symbolic_value_cuda, test/inductor/test_torchinductor.py::GPUTests::test_nonzero_unbacked_refinement_cuda, test/inductor/test_torchinductor.py::GPUTests::test_norm_constant_overflow_cuda, test/inductor/test_torchinductor.py::GPUTests::test_output_strides_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pattern_matcher_multi_user_cuda, test/inductor/test_torchinductor.py::GPUTests::test_permute1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_permute2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_bessel_j0_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_bessel_j1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_chebyshev_polynomial_u_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_chebyshev_polynomial_v_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_chebyshev_polynomial_w_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_digamma_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_entr_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_erfc_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_expit_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_expm1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_gammainc_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_gammaincc_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_hermite_polynomial_he_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_i0_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_i0e_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_i1e_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_logit_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_modified_bessel_k0_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_ndtr_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_polygamma_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_scaled_modified_bessel_k1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_shifted_chebyshev_polynomial_t_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_shifted_chebyshev_polynomial_u_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_sinc_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_xlog1py_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_xlogy_cuda, test/inductor/test_torchinductor.py::GPUTests::test_polar_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pow1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pow2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pow3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pow_int_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pow_symfloat_cuda, test/inductor/test_torchinductor.py::GPUTests::test_prepare_softmax_with_fast_math_cuda, test/inductor/test_torchinductor.py::GPUTests::test_profiler_mark_wrapper_call_cuda, test/inductor/test_torchinductor.py::GPUTests::test_randint_cuda, test/inductor/test_torchinductor.py::GPUTests::test_randint_distribution_cuda, test/inductor/test_torchinductor.py::GPUTests::test_randint_int64_mod_cuda, test/inductor/test_torchinductor.py::GPUTests::test_randn_generator_cuda, test/inductor/test_torchinductor.py::GPUTests::test_randn_like_empty_cuda, test/inductor/test_torchinductor.py::GPUTests::test_reduction1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_reduction2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_reduction3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_reduction5_cuda, test/inductor/test_torchinductor.py::GPUTests::test_reduction_config_limit_cuda, test/inductor/test_torchinductor.py::GPUTests::test_reflection_pad2d_backward_cuda, test/inductor/test_torchinductor.py::GPUTests::test_reinterpret_dtypeview_cuda, test/inductor/test_torchinductor.py::GPUTests::test_relu_cuda, test/inductor/test_torchinductor.py::GPUTests::test_remove_no_ops_cuda, test/inductor/test_torchinductor.py::GPUTests::test_remove_noop_slice_cuda, test/inductor/test_torchinductor.py::GPUTests::test_remove_noop_slice_scatter_cuda, test/inductor/test_torchinductor.py::GPUTests::test_remove_noop_view_default_cuda, test/inductor/test_torchinductor.py::GPUTests::test_remove_noop_view_dtype_cuda, test/inductor/test_torchinductor.py::GPUTests::test_repeat_interleave_2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_replication_pad_errors_with_bool_cuda, test/inductor/test_torchinductor.py::GPUTests::test_require_stride_expanded_cuda, test/inductor/test_torchinductor.py::GPUTests::test_resize_as_cuda, test/inductor/test_torchinductor.py::GPUTests::test_resize_cuda, test/inductor/test_torchinductor.py::GPUTests::test_reuse_buffers_with_aliasing_cuda, test/inductor/test_torchinductor.py::GPUTests::test_roll_cuda, test/inductor/test_torchinductor.py::GPUTests::test_rsqrt_cuda, test/inductor/test_torchinductor.py::GPUTests::test_scalar_output_cuda, test/inductor/test_torchinductor.py::GPUTests::test_scaled_dot_product_attention_cuda, test/inductor/test_torchinductor.py::GPUTests::test_scatter1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_scatter2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_scatter3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_scatter4_cuda, test/inductor/test_torchinductor.py::GPUTests::test_scatter5_cuda, test/inductor/test_torchinductor.py::GPUTests::test_scatter_add1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_scatter_add3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_scatter_reduce1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sdpa_prefer_nd_tiling_True_use_block_ptr_False_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sdpa_unaligned_mask_freezing_cuda, test/inductor/test_torchinductor.py::GPUTests::test_searchsorted_cuda, test/inductor/test_torchinductor.py::GPUTests::test_select_scatter_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sgn_extremal_cuda, test/inductor/test_torchinductor.py::GPUTests::test_shape_padding_cuda, test/inductor/test_torchinductor.py::GPUTests::test_should_pad_bench_for_bmm_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sigmoid_cuda, test/inductor/test_torchinductor.py::GPUTests::test_signbit_cuda, test/inductor/test_torchinductor.py::GPUTests::test_silu_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sin_cuda, test/inductor/test_torchinductor.py::GPUTests::test_single_elem_cuda, test/inductor/test_torchinductor.py::GPUTests::test_slice1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_slice3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_slice4_cuda, test/inductor/test_torchinductor.py::GPUTests::test_slice_mutation1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_slice_mutation2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_slice_mutation3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_slice_scatter2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_slice_scatter4_cuda, test/inductor/test_torchinductor.py::GPUTests::test_slice_scatter5_cuda, test/inductor/test_torchinductor.py::GPUTests::test_slice_scatter_reinplace_cuda, test/inductor/test_torchinductor.py::GPUTests::test_softmax_backward_data_cuda, test/inductor/test_torchinductor.py::GPUTests::test_softmax_one_kernel_loop_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sort_bool_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sort_transpose_cuda, test/inductor/test_torchinductor.py::GPUTests::test_special_polygamma_cuda, test/inductor/test_torchinductor.py::GPUTests::test_split_cuda, test/inductor/test_torchinductor.py::GPUTests::test_split_cumprod_low_prec_cuda, test/inductor/test_torchinductor.py::GPUTests::test_split_cumsum_cuda, test/inductor/test_torchinductor.py::GPUTests::test_split_cumsum_index_cuda, test/inductor/test_torchinductor.py::GPUTests::test_split_cumsum_low_prec_cuda, test/inductor/test_torchinductor.py::GPUTests::test_split_reduction_dynamic_shape_cuda, test/inductor/test_torchinductor.py::GPUTests::test_split_with_integer_cuda, test/inductor/test_torchinductor.py::GPUTests::test_squeeze2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_squeeze_varargs_cuda, test/inductor/test_torchinductor.py::GPUTests::test_std_cuda, test/inductor/test_torchinductor.py::GPUTests::test_stride_preservation_with_stride_modifying_fx_pass_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sum1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sum2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sum3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sum5_cuda, test/inductor/test_torchinductor.py::GPUTests::test_tensor2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_tensor_index_slice_cuda, test/inductor/test_torchinductor.py::GPUTests::test_tmp_not_defined_issue1_use_block_ptr_True_cuda, test/inductor/test_torchinductor.py::GPUTests::test_tmp_not_defined_issue2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_to_device_constant_cuda, test/inductor/test_torchinductor.py::GPUTests::test_to_device_cuda, test/inductor/test_torchinductor.py::GPUTests::test_to_dtype_cuda, test/inductor/test_torchinductor.py::GPUTests::test_to_memory_format_cuda, test/inductor/test_torchinductor.py::GPUTests::test_transposed_propagates_cuda, test/inductor/test_torchinductor.py::GPUTests::test_triu_cuda, test/inductor/test_torchinductor.py::GPUTests::test_uint4x2_mixed_mm_cuda, test/inductor/test_torchinductor.py::GPUTests::test_unbacked_floordiv_simplify_cuda, test/inductor/test_torchinductor.py::GPUTests::test_unbacked_floordiv_simplify_errors_cuda, test/inductor/test_torchinductor.py::GPUTests::test_unbind_cuda, test/inductor/test_torchinductor.py::GPUTests::test_unfold_zero_dimension_tensor_cuda, test/inductor/test_torchinductor.py::GPUTests::test_unspec_inputs_bfloat16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_unspec_inputs_float32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_unspec_inputs_float64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_unspec_inputs_int32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_unspec_inputs_uint8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_upsample_bilinear2d_a_cuda, test/inductor/test_torchinductor.py::GPUTests::test_upsample_bilinear2d_b_cuda, test/inductor/test_torchinductor.py::GPUTests::test_upsample_nearest1d_cuda, test/inductor/test_torchinductor.py::GPUTests::test_upsample_nearest2d_cuda, test/inductor/test_torchinductor.py::GPUTests::test_upsample_nearest3d_cuda, test/inductor/test_torchinductor.py::GPUTests::test_var_correction_cuda, test/inductor/test_torchinductor.py::GPUTests::test_var_mean_tile_reduction_False_cuda, test/inductor/test_torchinductor.py::GPUTests::test_var_mean_tile_reduction_True_cuda, test/inductor/test_torchinductor.py::GPUTests::test_vectorized_ops_masked_cuda, test/inductor/test_torchinductor.py::GPUTests::test_vectorized_ops_masked_var_novec_cuda, test/inductor/test_torchinductor.py::GPUTests::test_view_as_complex_cuda, test/inductor/test_torchinductor.py::GPUTests::test_view_detach_cuda, test/inductor/test_torchinductor.py::GPUTests::test_view_on_aliased_cuda, test/inductor/test_torchinductor.py::GPUTests::test_view_uint8_through_differing_bitwidths_cuda, test/inductor/test_torchinductor.py::GPUTests::test_views4_cuda, test/inductor/test_torchinductor.py::GPUTests::test_views5_cuda, test/inductor/test_torchinductor.py::GPUTests::test_views6_cuda, test/inductor/test_torchinductor.py::GPUTests::test_views7_cuda, test/inductor/test_torchinductor.py::GPUTests::test_weight_norm_bwd_cuda, test/inductor/test_torchinductor.py::GPUTests::test_xblock_divides_xnumel_cuda, test/inductor/test_torchinductor.py::GPUTests::test_zero_dim_reductions_cuda, test/inductor/test_torchinductor.py::GPUTests::test_zero_element_mutation_cuda, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_cant_optimize_compute, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_ctr_not_moved_to_cuda_when_used_in_index_put, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_donated_buffer_inplace, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_evict_last_non_coalesced_loads_block_ptr, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_graph_partition, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_graph_partition_condition_op, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_graph_partition_dynamic_scalar_inputs, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_graph_partition_fused_scheduler_node, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_graph_partition_item, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_graph_partition_symint, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_graph_partition_unbacked_symint, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_grouped_mm, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_has_constant_mask_block_multiple_False_ynumel_exceed_ygrid_size_False, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_has_constant_mask_block_multiple_True_ynumel_exceed_ygrid_size_False, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_has_constant_mask_block_multiple_True_ynumel_exceed_ygrid_size_True, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_inductor_detach_view_backend_inductor, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_inductor_sequence_nr, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_not_materialize_pointwise_reduction, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_numpy_on_gpu, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_optimize_compute, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_optimize_indexing_assert, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_optimize_indexing_dtype_with_constraint, test/inductor/test_torchinductor.py::RNNTest::test_rnn_compile_safe, test/inductor/test_torchinductor.py::NanCheckerTest::test_nan_checker_pass 2025-07-24T04:37:32.6014922Z 2025-07-24T04:37:33.4451683Z Running test batch 'tests to run' cost 704.36 seconds 2025-07-24T04:37:34.2969378Z 2025-07-24T04:37:34.2969646Z real 11m50.600s 2025-07-24T04:37:34.2970047Z user 14m1.009s 2025-07-24T04:37:34.2970397Z sys 10m25.912s 2025-07-24T04:37:34.2970807Z + assert_git_not_dirty 2025-07-24T04:37:34.2974215Z + [[ linux-jammy-rocm-py3.10 != *rocm* ]] 2025-07-24T04:37:34.2974861Z + test_aten 2025-07-24T04:37:34.2975293Z + echo 'Running ATen tests with pytorch lib' 2025-07-24T04:37:34.2975819Z Running ATen tests with pytorch lib 2025-07-24T04:37:34.2976261Z + [[ -n '' ]] 2025-07-24T04:37:34.2976645Z + echo 'Running test with the build folder' 2025-07-24T04:37:34.2977206Z Running test with the build folder 2025-07-24T04:37:34.2977652Z + TEST_BASE_DIR=build/bin 2025-07-24T04:37:34.2978846Z + ln -sf /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libc10.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libc10_hip.so build/bin 2025-07-24T04:37:34.2997440Z + ln -sf /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libcaffe2_nvrtc.so build/bin 2025-07-24T04:37:34.3021517Z + ln -sf '/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libmkldnn*' build/bin 2025-07-24T04:37:34.3045752Z + ln -sf '/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libnccl*' build/bin 2025-07-24T04:37:34.3068144Z + ln -sf /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorch.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorch_global_deps.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorch_hip.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorch_python.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorchbind_test.so build/bin 2025-07-24T04:37:34.3085745Z + ls build/bin 2025-07-24T04:37:34.3107110Z BackoffTest 2025-07-24T04:37:34.3107470Z CMakeFiles 2025-07-24T04:37:34.3107813Z CTestTestfile.cmake 2025-07-24T04:37:34.3108209Z CppSignature_test 2025-07-24T04:37:34.3108565Z Dict_test 2025-07-24T04:37:34.3108933Z Dimname_test 2025-07-24T04:37:34.3109255Z FileStoreTest 2025-07-24T04:37:34.3109615Z HashStoreTest 2025-07-24T04:37:34.3109941Z IListRef_test 2025-07-24T04:37:34.3110274Z KernelFunction_test 2025-07-24T04:37:34.3110632Z List_test 2025-07-24T04:37:34.3110945Z MaybeOwned_test 2025-07-24T04:37:34.3111293Z NamedTensor_test 2025-07-24T04:37:34.3111641Z ProcessGroupGlooTest 2025-07-24T04:37:34.3111890Z StorageUtils_test 2025-07-24T04:37:34.3112093Z TCPStoreTest 2025-07-24T04:37:34.3112295Z aot_model_compiler_test 2025-07-24T04:37:34.3112613Z apply_utils_test 2025-07-24T04:37:34.3112837Z atest 2025-07-24T04:37:34.3113030Z backend_fallback_test 2025-07-24T04:37:34.3113264Z basic 2025-07-24T04:37:34.3113484Z broadcast_test 2025-07-24T04:37:34.3113695Z c10_ArrayRef_test 2025-07-24T04:37:34.3113902Z c10_Bitset_test 2025-07-24T04:37:34.3114132Z c10_CompileTimeFunctionPointer_test 2025-07-24T04:37:34.3114411Z c10_ConstexprCrc_test 2025-07-24T04:37:34.3114642Z c10_DeadlockDetection_test 2025-07-24T04:37:34.3114883Z c10_DeviceGuard_test 2025-07-24T04:37:34.3115099Z c10_Device_test 2025-07-24T04:37:34.3115310Z c10_DispatchKeySet_test 2025-07-24T04:37:34.3115536Z c10_Enumerate_test 2025-07-24T04:37:34.3115734Z c10_Half_test 2025-07-24T04:37:34.3115996Z c10_InlineDeviceGuard_test 2025-07-24T04:37:34.3116245Z c10_InlineStreamGuard_test 2025-07-24T04:37:34.3116486Z c10_IntrusiveList_test 2025-07-24T04:37:34.3116706Z c10_LeftRight_test 2025-07-24T04:37:34.3116933Z c10_Metaprogramming_test 2025-07-24T04:37:34.3117167Z c10_NetworkFlow_test 2025-07-24T04:37:34.3117686Z c10_Scalar_test 2025-07-24T04:37:34.3117906Z c10_Semaphore_test 2025-07-24T04:37:34.3118130Z c10_SizesAndStrides_test 2025-07-24T04:37:34.3118378Z c10_StreamGuard_test 2025-07-24T04:37:34.3118595Z c10_SymInt_test 2025-07-24T04:37:34.3118809Z c10_Synchronized_test 2025-07-24T04:37:34.3119027Z c10_ThreadLocal_test 2025-07-24T04:37:34.3119239Z c10_TypeIndex_test 2025-07-24T04:37:34.3119446Z c10_TypeList_test 2025-07-24T04:37:34.3119654Z c10_TypeTraits_test 2025-07-24T04:37:34.3119863Z c10_accumulate_test 2025-07-24T04:37:34.3120069Z c10_bfloat16_test 2025-07-24T04:37:34.3120262Z c10_bit_cast_test 2025-07-24T04:37:34.3120469Z c10_complex_math_test 2025-07-24T04:37:34.3120677Z c10_complex_test 2025-07-24T04:37:34.3120876Z c10_cow_test 2025-07-24T04:37:34.3121067Z c10_error_test 2025-07-24T04:37:34.3121267Z c10_exception_test 2025-07-24T04:37:34.3121465Z c10_flags_test 2025-07-24T04:37:34.3121663Z c10_generic_math_test 2025-07-24T04:37:34.3121911Z c10_hip_HIPAssertionsTest_1_var_test 2025-07-24T04:37:34.3122210Z c10_hip_HIPAssertionsTest_catches_stream 2025-07-24T04:37:34.3122639Z c10_hip_HIPAssertionsTest_catches_thread_and_block_and_device 2025-07-24T04:37:34.3123228Z c10_hip_HIPAssertionsTest_from_2_processes 2025-07-24T04:37:34.3123599Z c10_hip_HIPAssertionsTest_multiple_writes_from_blocks_and_threads 2025-07-24T04:37:34.3124022Z c10_hip_HIPAssertionsTest_multiple_writes_from_multiple_blocks 2025-07-24T04:37:34.3124422Z c10_hip_HIPAssertionsTest_multiple_writes_from_same_block 2025-07-24T04:37:34.3124731Z c10_hip_HIPTest 2025-07-24T04:37:34.3124946Z c10_intrusive_ptr_benchmark 2025-07-24T04:37:34.3125193Z c10_intrusive_ptr_test 2025-07-24T04:37:34.3125412Z c10_irange_test 2025-07-24T04:37:34.3125615Z c10_lazy_test 2025-07-24T04:37:34.3125816Z c10_logging_test 2025-07-24T04:37:34.3126012Z c10_optional_test 2025-07-24T04:37:34.3126230Z c10_ordered_preserving_dict_test 2025-07-24T04:37:34.3126482Z c10_registry_test 2025-07-24T04:37:34.3126689Z c10_small_vector_test 2025-07-24T04:37:34.3126912Z c10_ssize_test 2025-07-24T04:37:34.3127117Z c10_string_util_test 2025-07-24T04:37:34.3127339Z c10_string_view_test 2025-07-24T04:37:34.3127545Z c10_tempfile_test 2025-07-24T04:37:34.3127741Z c10_typeid_test 2025-07-24T04:37:34.3127952Z cmake_install.cmake 2025-07-24T04:37:34.3128158Z cpu_allocator_test 2025-07-24T04:37:34.3128368Z cpu_generator_test 2025-07-24T04:37:34.3128592Z cpu_profiling_allocator_test 2025-07-24T04:37:34.3128840Z cpu_rng_test 2025-07-24T04:37:34.3129035Z dlconvertor_test 2025-07-24T04:37:34.3129241Z example_allreduce 2025-07-24T04:37:34.3129457Z extension_backend_test 2025-07-24T04:37:34.3129679Z half_test 2025-07-24T04:37:34.3129873Z hip_apply_test 2025-07-24T04:37:34.3130073Z hip_complex_math_test 2025-07-24T04:37:34.3130292Z hip_complex_test 2025-07-24T04:37:34.3130502Z hip_distributions_test 2025-07-24T04:37:34.3130727Z hip_dlconvertor_test 2025-07-24T04:37:34.3130940Z hip_generator_test 2025-07-24T04:37:34.3131145Z hip_half_test 2025-07-24T04:37:34.3131351Z hip_integer_divider_test 2025-07-24T04:37:34.3131583Z hip_optional_test 2025-07-24T04:37:34.3131807Z hip_packedtensoraccessor_test 2025-07-24T04:37:34.3132060Z hip_vectorized_test 2025-07-24T04:37:34.3132263Z inline_container_test 2025-07-24T04:37:34.3132476Z ivalue_test 2025-07-24T04:37:34.3132683Z kernel_function_legacy_test 2025-07-24T04:37:34.3132925Z kernel_function_test 2025-07-24T04:37:34.3133146Z kernel_lambda_legacy_test 2025-07-24T04:37:34.3133391Z kernel_lambda_test 2025-07-24T04:37:34.3133615Z kernel_stackbased_test 2025-07-24T04:37:34.3133839Z lazy_tensor_test 2025-07-24T04:37:34.3134048Z legacy_vmap_test 2025-07-24T04:37:34.3134256Z libc10.so 2025-07-24T04:37:34.3134453Z libc10_hip.so 2025-07-24T04:37:34.3134663Z libcaffe2_nvrtc.so 2025-07-24T04:37:34.3134875Z 'libmkldnn*' 2025-07-24T04:37:34.3135062Z 'libnccl*' 2025-07-24T04:37:34.3135254Z libtorch.so 2025-07-24T04:37:34.3135454Z libtorch_cpu.so 2025-07-24T04:37:34.3135668Z libtorch_global_deps.so 2025-07-24T04:37:34.3136090Z libtorch_hip.so 2025-07-24T04:37:34.3136307Z libtorch_python.so 2025-07-24T04:37:34.3136520Z libtorchbind_test.so 2025-07-24T04:37:34.3136765Z make_boxed_from_unboxed_functor_test 2025-07-24T04:37:34.3137031Z math_kernel_test 2025-07-24T04:37:34.3137231Z memory_format_test 2025-07-24T04:37:34.3137445Z memory_overlapping_test 2025-07-24T04:37:34.3137677Z mobile_memory_cleanup 2025-07-24T04:37:34.3137893Z native_test 2025-07-24T04:37:34.3138088Z op_allowlist_test 2025-07-24T04:37:34.3138298Z op_registration_test 2025-07-24T04:37:34.3138513Z operator_name_test 2025-07-24T04:37:34.3138720Z operators_test 2025-07-24T04:37:34.3138940Z packedtensoraccessor_test 2025-07-24T04:37:34.3139181Z parallel_benchmark 2025-07-24T04:37:34.3139376Z pow_test 2025-07-24T04:37:34.3139565Z protoc 2025-07-24T04:37:34.3139756Z protoc-3.13.0.0 2025-07-24T04:37:34.3139962Z quantized_test 2025-07-24T04:37:34.3140166Z reduce_ops_test 2025-07-24T04:37:34.3140385Z reportMemoryUsage_test 2025-07-24T04:37:34.3140619Z scalar_tensor_test 2025-07-24T04:37:34.3140822Z scalar_test 2025-07-24T04:37:34.3141019Z static_runtime_bench 2025-07-24T04:37:34.3141376Z static_runtime_test 2025-07-24T04:37:34.3141605Z stride_properties_test 2025-07-24T04:37:34.3141828Z tensor_iterator_test 2025-07-24T04:37:34.3142037Z test_api 2025-07-24T04:37:34.3142225Z test_cpp_rpc 2025-07-24T04:37:34.3142429Z test_dist_autograd 2025-07-24T04:37:34.3142634Z test_jit 2025-07-24T04:37:34.3142818Z test_lazy 2025-07-24T04:37:34.3143005Z test_mobile_nnc 2025-07-24T04:37:34.3143196Z test_nativert 2025-07-24T04:37:34.3143398Z test_parallel 2025-07-24T04:37:34.3143601Z test_tensorexpr 2025-07-24T04:37:34.3143808Z thread_init_test 2025-07-24T04:37:34.3144019Z torch_shm_manager 2025-07-24T04:37:34.3144236Z tutorial_tensorexpr 2025-07-24T04:37:34.3144454Z type_ptr_test 2025-07-24T04:37:34.3144653Z type_test 2025-07-24T04:37:34.3144847Z undefined_tensor_test 2025-07-24T04:37:34.3145076Z vec_test_all_types_AVX2 2025-07-24T04:37:34.3145306Z vec_test_all_types_AVX512 2025-07-24T04:37:34.3145545Z vec_test_all_types_DEFAULT 2025-07-24T04:37:34.3145782Z verify_api_visibility 2025-07-24T04:37:34.3145991Z weakref_test 2025-07-24T04:37:34.3146184Z wrapdim_test 2025-07-24T04:37:34.3146375Z xla_tensor_test 2025-07-24T04:37:34.3146601Z + aten/tools/run_tests.sh build/bin 2025-07-24T04:37:34.3146863Z + set -e 2025-07-24T04:37:34.3147068Z ++ dirname aten/tools/run_tests.sh 2025-07-24T04:37:34.3157468Z + VALGRIND_SUP=/var/lib/jenkins/pytorch/aten/tools/valgrind.sup 2025-07-24T04:37:34.3157852Z + export CPP_TESTS_DIR=build/bin 2025-07-24T04:37:34.3158109Z + CPP_TESTS_DIR=build/bin 2025-07-24T04:37:34.3158342Z + VALGRIND=OFF 2025-07-24T04:37:34.3159779Z + python test/run_test.py --cpp --verbose -i cpp/basic cpp/atest cpp/scalar_test cpp/broadcast_test cpp/wrapdim_test cpp/apply_utils_test cpp/dlconvertor_test cpp/native_test cpp/scalar_tensor_test cpp/undefined_tensor_test cpp/extension_backend_test cpp/lazy_tensor_test cpp/tensor_iterator_test cpp/Dimname_test cpp/Dict_test cpp/NamedTensor_test cpp/cpu_generator_test cpp/legacy_vmap_test cpp/operators_test 2025-07-24T04:37:37.2479897Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. 2025-07-24T04:37:37.2482168Z import pkg_resources 2025-07-24T04:37:39.3731706Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/pytorch/test/.pytorch-disabled-tests.json 2025-07-24T04:37:39.3971647Z Found test times from artifacts 2025-07-24T04:37:39.4987238Z Found test times from artifacts 2025-07-24T04:37:39.5025397Z Running 25% of tests based on TD 2025-07-24T04:37:39.5031055Z Running parallel tests on 8 processes 2025-07-24T04:37:39.5031640Z Name: tests to run (est. time: 0.0min) 2025-07-24T04:37:39.5032285Z Serial tests (0): 2025-07-24T04:37:39.5032522Z Parallel tests (5): 2025-07-24T04:37:39.5032775Z cpp/Dict_test 1/1 2025-07-24T04:37:39.5032993Z cpp/Dimname_test 1/1 2025-07-24T04:37:39.5033236Z cpp/NamedTensor_test 1/1 2025-07-24T04:37:39.5033493Z cpp/apply_utils_test 1/1 2025-07-24T04:37:39.5033736Z cpp/atest 1/1 2025-07-24T04:37:39.5033983Z Name: excluded (est. time: 0.0min) 2025-07-24T04:37:39.5034246Z Serial tests (0): 2025-07-24T04:37:39.5034466Z Parallel tests (14): 2025-07-24T04:37:39.5034693Z cpp/basic 1/1 2025-07-24T04:37:39.5034917Z cpp/broadcast_test 1/1 2025-07-24T04:37:39.5035170Z cpp/cpu_generator_test 1/1 2025-07-24T04:37:39.5035431Z cpp/dlconvertor_test 1/1 2025-07-24T04:37:39.5035681Z cpp/extension_backend_test 1/1 2025-07-24T04:37:39.5035957Z cpp/lazy_tensor_test 1/1 2025-07-24T04:37:39.5036204Z cpp/legacy_vmap_test 1/1 2025-07-24T04:37:39.5036450Z cpp/native_test 1/1 2025-07-24T04:37:39.5036689Z cpp/operators_test 1/1 2025-07-24T04:37:39.5036940Z cpp/scalar_tensor_test 1/1 2025-07-24T04:37:39.5037190Z cpp/scalar_test 1/1 2025-07-24T04:37:39.5037657Z cpp/tensor_iterator_test 1/1 2025-07-24T04:37:39.5037922Z cpp/undefined_tensor_test 1/1 2025-07-24T04:37:39.5038180Z cpp/wrapdim_test 1/1 2025-07-24T04:37:39.5038504Z Running cpp/Dict_test 1/1 ... [2025-07-24 04:37:39.503732] 2025-07-24T04:37:39.5038851Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2025-07-24T04:37:39.5045170Z Executing ['pytest', '/var/lib/jenkins/pytorch/build/bin/Dict_test', '-m', 'serial', '-v', '-vv', '-rfEX', '-n', '8', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-7ef7f4285a7ceb3b.xml', '-x', '--reruns=2'] ... [2025-07-24 04:37:39.504336] 2025-07-24T04:37:40.5718663Z 2025-07-24T04:37:40.5719949Z cpp/Dict_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.Dict_test_1.1_fd143b70e247bf72_.log 2025-07-24T04:37:40.5720875Z 2025-07-24T04:37:40.5721337Z GITHUB_RUN_ID, GITHUB_RUN_ATTEMPT, or ARTIFACTS_FILE_SUFFIX not set, not uploading 2025-07-24T04:37:40.5722057Z Uploading artifacts took 0.00 seconds 2025-07-24T04:37:40.5728622Z Running cpp/Dimname_test 1/1 ... [2025-07-24 04:37:40.572029] 2025-07-24T04:37:40.5729229Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2025-07-24T04:37:41.6394728Z Executing ['pytest', '/var/lib/jenkins/pytorch/build/bin/Dimname_test', '-m', 'serial', '-v', '-vv', '-rfEX', '-n', '8', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-179de786d2c21026.xml', '-x', '--reruns=2'] ... [2025-07-24 04:37:40.572377] 2025-07-24T04:37:41.6396295Z 2025-07-24T04:37:41.6397027Z cpp/Dimname_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.Dimname_test_1.1_e0232da21b1b8ad7_.log 2025-07-24T04:37:41.6397928Z 2025-07-24T04:37:41.6398274Z Running cpp/NamedTensor_test 1/1 ... [2025-07-24 04:37:41.639088] 2025-07-24T04:37:41.6398909Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2025-07-24T04:37:41.6400508Z Executing ['pytest', '/var/lib/jenkins/pytorch/build/bin/NamedTensor_test', '-m', 'serial', '-v', '-vv', '-rfEX', '-n', '8', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-d8ee4d984b7e7d4b.xml', '-x', '--reruns=2'] ... [2025-07-24 04:37:41.639452] 2025-07-24T04:37:42.7066659Z 2025-07-24T04:37:42.7068133Z cpp/NamedTensor_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.NamedTensor_test_1.1_f672c04899ca8881_.log 2025-07-24T04:37:42.7069068Z 2025-07-24T04:37:42.7069385Z Running cpp/apply_utils_test 1/1 ... [2025-07-24 04:37:42.706557] 2025-07-24T04:37:42.7070072Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2025-07-24T04:37:42.7073516Z Executing ['pytest', '/var/lib/jenkins/pytorch/build/bin/apply_utils_test', '-m', 'serial', '-v', '-vv', '-rfEX', '-n', '8', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-e6b1c29ce9025ae2.xml', '-x', '--reruns=2'] ... [2025-07-24 04:37:42.707105] 2025-07-24T04:37:43.7738478Z 2025-07-24T04:37:43.7739775Z cpp/apply_utils_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.apply_utils_test_1.1_83d7ff74f0b568ab_.log 2025-07-24T04:37:43.7740768Z 2025-07-24T04:37:43.7741055Z Running cpp/atest 1/1 ... [2025-07-24 04:37:43.773643] 2025-07-24T04:37:43.7741629Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2025-07-24T04:37:43.7743277Z Executing ['pytest', '/var/lib/jenkins/pytorch/build/bin/atest', '-m', 'serial', '-v', '-vv', '-rfEX', '-n', '8', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-951ecc43b7ec935b.xml', '-x', '--reruns=2'] ... [2025-07-24 04:37:43.774024] 2025-07-24T04:37:44.8407359Z 2025-07-24T04:37:44.8408500Z cpp/atest 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.atest_1.1_4d0a2e91bbb0e767_.log 2025-07-24T04:37:44.8409454Z 2025-07-24T04:37:48.4362255Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. 2025-07-24T04:37:48.4365350Z import pkg_resources 2025-07-24T04:37:48.4537376Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. 2025-07-24T04:37:48.4539420Z import pkg_resources 2025-07-24T04:37:48.4816402Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. 2025-07-24T04:37:48.4818409Z import pkg_resources 2025-07-24T04:37:48.4820363Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. 2025-07-24T04:37:48.4822362Z import pkg_resources 2025-07-24T04:37:48.4880779Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. 2025-07-24T04:37:48.4882886Z import pkg_resources 2025-07-24T04:37:48.5030719Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. 2025-07-24T04:37:48.5032177Z import pkg_resources 2025-07-24T04:37:48.5048351Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. 2025-07-24T04:37:48.5050354Z import pkg_resources 2025-07-24T04:37:48.5073150Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81. 2025-07-24T04:37:48.5075541Z import pkg_resources 2025-07-24T04:37:49.0584439Z Running cpp/Dict_test 1/1 ... [2025-07-24 04:37:49.058225] 2025-07-24T04:37:49.0584908Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2025-07-24T04:37:49.0589651Z Executing ['pytest', '/var/lib/jenkins/pytorch/build/bin/Dict_test', '-m', 'not serial', '-v', '-vv', '-rfEX', '-n', '8', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-efc1a24455eecd36.xml', '-x', '--reruns=2'] ... [2025-07-24 04:37:49.058736] 2025-07-24T04:37:49.1622831Z Running cpp/Dimname_test 1/1 ... [2025-07-24 04:37:49.162107] 2025-07-24T04:37:49.1623277Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2025-07-24T04:37:49.1623737Z Running cpp/NamedTensor_test 1/1 ... [2025-07-24 04:37:49.162146] 2025-07-24T04:37:49.1624198Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2025-07-24T04:37:49.1628118Z Executing ['pytest', '/var/lib/jenkins/pytorch/build/bin/Dimname_test', '-m', 'not serial', '-v', '-vv', '-rfEX', '-n', '8', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-bd59d4501e834beb.xml', '-x', '--reruns=2'] ... [2025-07-24 04:37:49.162604] 2025-07-24T04:37:49.1635132Z Executing ['pytest', '/var/lib/jenkins/pytorch/build/bin/NamedTensor_test', '-m', 'not serial', '-v', '-vv', '-rfEX', '-n', '8', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-3177e44f9f6563eb.xml', '-x', '--reruns=2'] ... [2025-07-24 04:37:49.162678] 2025-07-24T04:37:49.1824299Z Running cpp/apply_utils_test 1/1 ... [2025-07-24 04:37:49.182222] 2025-07-24T04:37:49.1824788Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2025-07-24T04:37:49.1829704Z Executing ['pytest', '/var/lib/jenkins/pytorch/build/bin/apply_utils_test', '-m', 'not serial', '-v', '-vv', '-rfEX', '-n', '8', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-fc68c4b8d0a920d0.xml', '-x', '--reruns=2'] ... [2025-07-24 04:37:49.182720] 2025-07-24T04:37:49.1901890Z Running cpp/atest 1/1 ... [2025-07-24 04:37:49.190024] 2025-07-24T04:37:49.1902336Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2025-07-24T04:37:49.1907643Z Executing ['pytest', '/var/lib/jenkins/pytorch/build/bin/atest', '-m', 'not serial', '-v', '-vv', '-rfEX', '-n', '8', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-013c885fdf2efba8.xml', '-x', '--reruns=2'] ... [2025-07-24 04:37:49.190535] 2025-07-24T04:37:50.1760509Z 2025-07-24T04:37:50.1761288Z cpp/Dict_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.Dict_test_1.1_0210d620eb06e2ad_.log 2025-07-24T04:37:50.1761893Z 2025-07-24T04:37:50.2299588Z 2025-07-24T04:37:50.2302023Z cpp/NamedTensor_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.NamedTensor_test_1.1_ffe41231487ea850_.log 2025-07-24T04:37:50.2303024Z 2025-07-24T04:37:50.2798940Z 2025-07-24T04:37:50.2799645Z cpp/Dimname_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.Dimname_test_1.1_b119a0ee6e8ed2ae_.log 2025-07-24T04:37:50.2800726Z 2025-07-24T04:37:50.2997895Z 2025-07-24T04:37:50.2998723Z cpp/apply_utils_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.apply_utils_test_1.1_17ccd3c9843071d1_.log 2025-07-24T04:37:50.3000130Z 2025-07-24T04:37:50.3079561Z 2025-07-24T04:37:50.3080214Z cpp/atest 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.atest_1.1_cae48305061b9e27_.log 2025-07-24T04:37:50.3081815Z 2025-07-24T04:37:50.9877773Z Running test batch 'tests to run' cost 11.48 seconds 2025-07-24T04:37:51.6397946Z + run_if_exists tensor_interop_test 2025-07-24T04:37:51.6398496Z + local test_name=tensor_interop_test 2025-07-24T04:37:51.6399008Z + [[ -x build/bin/tensor_interop_test ]] 2025-07-24T04:37:51.6399581Z + echo 'Warning: tensor_interop_test does not exist.' 2025-07-24T04:37:51.6400158Z Warning: tensor_interop_test does not exist. 2025-07-24T04:37:51.6400642Z + run_if_exists cudnn_test 2025-07-24T04:37:51.6401053Z + local test_name=cudnn_test 2025-07-24T04:37:51.6402173Z + [[ -x build/bin/cudnn_test ]] 2025-07-24T04:37:51.6402787Z + echo 'Warning: cudnn_test does not exist.' 2025-07-24T04:37:51.6403294Z Warning: cudnn_test does not exist. 2025-07-24T04:37:51.6403732Z + run_if_exists cuda_generator_test 2025-07-24T04:37:51.6404173Z + local test_name=cuda_generator_test 2025-07-24T04:37:51.6404644Z + [[ -x build/bin/cuda_generator_test ]] 2025-07-24T04:37:51.6405194Z + echo 'Warning: cuda_generator_test does not exist.' 2025-07-24T04:37:51.6405745Z Warning: cuda_generator_test does not exist. 2025-07-24T04:37:51.6406216Z + run_if_exists apply_test 2025-07-24T04:37:51.6406608Z + local test_name=apply_test 2025-07-24T04:37:51.6407083Z + [[ -x build/bin/apply_test ]] 2025-07-24T04:37:51.6407626Z + echo 'Warning: apply_test does not exist.' 2025-07-24T04:37:51.6408216Z Warning: apply_test does not exist. 2025-07-24T04:37:51.6408745Z + run_if_exists stream_test 2025-07-24T04:37:51.6409232Z + local test_name=stream_test 2025-07-24T04:37:51.6409745Z + [[ -x build/bin/stream_test ]] 2025-07-24T04:37:51.6410304Z + echo 'Warning: stream_test does not exist.' 2025-07-24T04:37:51.6410948Z Warning: stream_test does not exist. 2025-07-24T04:37:51.6411266Z + run_if_exists cuda_half_test 2025-07-24T04:37:51.6411528Z + local test_name=cuda_half_test 2025-07-24T04:37:51.6411788Z + [[ -x build/bin/cuda_half_test ]] 2025-07-24T04:37:51.6412079Z + echo 'Warning: cuda_half_test does not exist.' 2025-07-24T04:37:51.6412390Z Warning: cuda_half_test does not exist. 2025-07-24T04:37:51.6412671Z + run_if_exists cuda_vectorized_test 2025-07-24T04:37:51.6412951Z + local test_name=cuda_vectorized_test 2025-07-24T04:37:51.6413246Z + [[ -x build/bin/cuda_vectorized_test ]] 2025-07-24T04:37:51.6413575Z + echo 'Warning: cuda_vectorized_test does not exist.' 2025-07-24T04:37:51.6413912Z Warning: cuda_vectorized_test does not exist. 2025-07-24T04:37:51.6414224Z + run_if_exists cuda_distributions_test 2025-07-24T04:37:51.6414527Z + local test_name=cuda_distributions_test 2025-07-24T04:37:51.6414843Z + [[ -x build/bin/cuda_distributions_test ]] 2025-07-24T04:37:51.6415197Z + echo 'Warning: cuda_distributions_test does not exist.' 2025-07-24T04:37:51.6415567Z Warning: cuda_distributions_test does not exist. 2025-07-24T04:37:51.6415884Z + run_if_exists cuda_optional_test 2025-07-24T04:37:51.6416157Z + local test_name=cuda_optional_test 2025-07-24T04:37:51.6416441Z + [[ -x build/bin/cuda_optional_test ]] 2025-07-24T04:37:51.6416759Z + echo 'Warning: cuda_optional_test does not exist.' 2025-07-24T04:37:51.6417091Z Warning: cuda_optional_test does not exist. 2025-07-24T04:37:51.6417393Z + run_if_exists cuda_tensor_interop_test 2025-07-24T04:37:51.6417684Z + local test_name=cuda_tensor_interop_test 2025-07-24T04:37:51.6417989Z + [[ -x build/bin/cuda_tensor_interop_test ]] 2025-07-24T04:37:51.6418321Z + echo 'Warning: cuda_tensor_interop_test does not exist.' 2025-07-24T04:37:51.6418668Z Warning: cuda_tensor_interop_test does not exist. 2025-07-24T04:37:51.6418968Z + run_if_exists cuda_complex_test 2025-07-24T04:37:51.6419228Z + local test_name=cuda_complex_test 2025-07-24T04:37:51.6419495Z + [[ -x build/bin/cuda_complex_test ]] 2025-07-24T04:37:51.6419791Z + echo 'Warning: cuda_complex_test does not exist.' 2025-07-24T04:37:51.6420105Z Warning: cuda_complex_test does not exist. 2025-07-24T04:37:51.6420383Z + run_if_exists cuda_complex_math_test 2025-07-24T04:37:51.6420652Z + local test_name=cuda_complex_math_test 2025-07-24T04:37:51.6420927Z + [[ -x build/bin/cuda_complex_math_test ]] 2025-07-24T04:37:51.6421247Z + echo 'Warning: cuda_complex_math_test does not exist.' 2025-07-24T04:37:51.6421579Z Warning: cuda_complex_math_test does not exist. 2025-07-24T04:37:51.6421869Z + run_if_exists cuda_cub_test 2025-07-24T04:37:51.6422112Z + local test_name=cuda_cub_test 2025-07-24T04:37:51.6422364Z + [[ -x build/bin/cuda_cub_test ]] 2025-07-24T04:37:51.6422636Z + echo 'Warning: cuda_cub_test does not exist.' 2025-07-24T04:37:51.6422925Z Warning: cuda_cub_test does not exist. 2025-07-24T04:37:51.6423356Z + run_if_exists cuda_atomic_ops_test 2025-07-24T04:37:51.6423627Z + local test_name=cuda_atomic_ops_test 2025-07-24T04:37:51.6423904Z + [[ -x build/bin/cuda_atomic_ops_test ]] 2025-07-24T04:37:51.6424211Z + echo 'Warning: cuda_atomic_ops_test does not exist.' 2025-07-24T04:37:51.6424533Z Warning: cuda_atomic_ops_test does not exist. 2025-07-24T04:37:51.6424804Z + '[' OFF == ON ']' 2025-07-24T04:37:51.6425010Z + [[ -n '' ]] 2025-07-24T04:37:51.6425207Z + assert_git_not_dirty 2025-07-24T04:37:51.6425451Z + [[ linux-jammy-rocm-py3.10 != *rocm* ]] 2025-07-24T04:37:51.6425730Z + sccache_epilogue 2025-07-24T04:37:51.6425980Z + echo '::group::Sccache Compilation Log' 2025-07-24T04:37:51.6426571Z ##[group]Sccache Compilation Log 2025-07-24T04:37:51.6426877Z + echo '=================== sccache compilation log ===================' 2025-07-24T04:37:51.6427248Z =================== sccache compilation log =================== 2025-07-24T04:37:51.6427770Z + python /var/lib/jenkins/pytorch/.ci/pytorch/print_sccache_log.py /var/lib/jenkins/sccache_error.log 2025-07-24T04:37:51.6569928Z + echo '=========== If your build fails, please take a look at the log above for possible reasons ===========' 2025-07-24T04:37:51.6571490Z =========== If your build fails, please take a look at the log above for possible reasons =========== 2025-07-24T04:37:51.6572218Z + sccache --show-stats 2025-07-24T04:37:51.6608401Z Compile requests 746 2025-07-24T04:37:51.6608996Z Compile requests executed 81 2025-07-24T04:37:51.6609532Z Cache hits 14 2025-07-24T04:37:51.6610075Z Cache hits (C/C++) 14 2025-07-24T04:37:51.6610579Z Cache misses 67 2025-07-24T04:37:51.6611089Z Cache misses (C/C++) 61 2025-07-24T04:37:51.6611599Z Cache misses (HIP) 6 2025-07-24T04:37:51.6612052Z Cache hits rate 17.28 % 2025-07-24T04:37:51.6612511Z Cache hits rate (C/C++) 18.67 % 2025-07-24T04:37:51.6612972Z Cache hits rate (HIP) 0.00 % 2025-07-24T04:37:51.6613414Z Cache timeouts 0 2025-07-24T04:37:51.6613852Z Cache read errors 0 2025-07-24T04:37:51.6614284Z Forced recaches 0 2025-07-24T04:37:51.6614719Z Cache write errors 0 2025-07-24T04:37:51.6615150Z Cache errors 0 2025-07-24T04:37:51.6615572Z Compilations 67 2025-07-24T04:37:51.6616005Z Compilation failures 0 2025-07-24T04:37:51.6616458Z Non-cacheable compilations 0 2025-07-24T04:37:51.6616901Z Non-cacheable calls 18 2025-07-24T04:37:51.6617349Z Non-compilation calls 647 2025-07-24T04:37:51.6617831Z Unsupported compiler calls 0 2025-07-24T04:37:51.6618143Z Average cache write 0.000 s 2025-07-24T04:37:51.6618433Z Average compiler 8.489 s 2025-07-24T04:37:51.6618715Z Average cache read hit 0.000 s 2025-07-24T04:37:51.6619008Z Failed distributed compilations 0 2025-07-24T04:37:51.6619200Z 2025-07-24T04:37:51.6619295Z Non-cacheable reasons: 2025-07-24T04:37:51.6619536Z unknown source language 16 2025-07-24T04:37:51.6619804Z -E 2 2025-07-24T04:37:51.6619979Z 2025-07-24T04:37:51.6620160Z Cache location Local disk: "/var/lib/jenkins/.cache/sccache" 2025-07-24T04:37:51.6620542Z Use direct/preprocessor mode? yes 2025-07-24T04:37:51.6620820Z Version (client) 0.10.0 2025-07-24T04:37:51.6621097Z Cache size 3 MiB 2025-07-24T04:37:51.6621381Z Max cache size 10 GiB 2025-07-24T04:37:51.6621666Z + sccache --stop-server 2025-07-24T04:37:51.6649813Z Stopping sccache server... 2025-07-24T04:37:51.6653682Z Compile requests 746 2025-07-24T04:37:51.6654138Z Compile requests executed 81 2025-07-24T04:37:51.6654601Z Cache hits 14 2025-07-24T04:37:51.6655338Z Cache hits (C/C++) 14 2025-07-24T04:37:51.6655783Z Cache misses 67 2025-07-24T04:37:51.6656211Z Cache misses (C/C++) 61 2025-07-24T04:37:51.6656636Z Cache misses (HIP) 6 2025-07-24T04:37:51.6657073Z Cache hits rate 17.28 % 2025-07-24T04:37:51.6657516Z Cache hits rate (C/C++) 18.67 % 2025-07-24T04:37:51.6657963Z Cache hits rate (HIP) 0.00 % 2025-07-24T04:37:51.6658244Z Cache timeouts 0 2025-07-24T04:37:51.6658499Z Cache read errors 0 2025-07-24T04:37:51.6658753Z Forced recaches 0 2025-07-24T04:37:51.6659006Z Cache write errors 0 2025-07-24T04:37:51.6659264Z Cache errors 0 2025-07-24T04:37:51.6659515Z Compilations 67 2025-07-24T04:37:51.6659776Z Compilation failures 0 2025-07-24T04:37:51.6660043Z Non-cacheable compilations 0 2025-07-24T04:37:51.6660310Z Non-cacheable calls 18 2025-07-24T04:37:51.6660573Z Non-compilation calls 647 2025-07-24T04:37:51.6660992Z Unsupported compiler calls 0 2025-07-24T04:37:51.6661266Z Average cache write 0.000 s 2025-07-24T04:37:51.6661546Z Average compiler 8.489 s 2025-07-24T04:37:51.6661816Z Average cache read hit 0.000 s 2025-07-24T04:37:51.6662093Z Failed distributed compilations 0 2025-07-24T04:37:51.6662280Z 2025-07-24T04:37:51.6662367Z Non-cacheable reasons: 2025-07-24T04:37:51.6662598Z unknown source language 16 2025-07-24T04:37:51.6662858Z -E 2 2025-07-24T04:37:51.6663021Z 2025-07-24T04:37:51.6663191Z Cache location Local disk: "/var/lib/jenkins/.cache/sccache" 2025-07-24T04:37:51.6663557Z Use direct/preprocessor mode? yes 2025-07-24T04:37:51.6663825Z Version (client) 0.10.0 2025-07-24T04:37:51.6664103Z Cache size 3 MiB 2025-07-24T04:37:51.6664372Z Max cache size 10 GiB 2025-07-24T04:37:51.6664636Z + echo ::endgroup:: 2025-07-24T04:37:51.6665009Z ##[endgroup] 2025-07-24T04:37:51.6777598Z ##[group]Run # copy test results back to the mounted workspace, needed sudo, resulting permissions were correct 2025-07-24T04:37:51.6778384Z # copy test results back to the mounted workspace, needed sudo, resulting permissions were correct 2025-07-24T04:37:51.6779329Z docker exec -t "a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835" sh -c "cd ../pytorch && sudo cp -R test/test-reports ../workspace/test" 2025-07-24T04:37:51.6807608Z shell: /usr/bin/bash -e {0} 2025-07-24T04:37:51.6807867Z env: 2025-07-24T04:37:51.6808069Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:37:51.6808459Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:37:51.6809031Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:37:51.6809593Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:37:51.6810654Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:37:51.6811529Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:37:51.6811814Z AWS_REGION: us-east-1 2025-07-24T04:37:51.6812140Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:37:51.6812511Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:37:51.6818635Z AWS_SESSION_TOKEN: *** 2025-07-24T04:37:51.6819060Z CONTAINER_NAME: a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:37:51.6819522Z ##[endgroup] 2025-07-24T04:37:51.7815706Z ##[group]Run cat test/**/*_toprint.log || true 2025-07-24T04:37:51.7816082Z cat test/**/*_toprint.log || true 2025-07-24T04:37:51.7846962Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:37:51.7847303Z env: 2025-07-24T04:37:51.7847504Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:37:51.7847882Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:37:51.7848456Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:37:51.7848966Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:37:51.7849800Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:37:51.7850548Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:37:51.7850797Z AWS_REGION: us-east-1 2025-07-24T04:37:51.7851125Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:37:51.7851458Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:37:51.7856874Z AWS_SESSION_TOKEN: *** 2025-07-24T04:37:51.7857273Z CONTAINER_NAME: a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:37:51.7857691Z ##[endgroup] 2025-07-24T04:37:51.8017602Z cat: 'test/**/*_toprint.log': No such file or directory 2025-07-24T04:37:51.8134484Z Prepare all required actions 2025-07-24T04:37:51.8135156Z Getting action download info 2025-07-24T04:37:52.1280974Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2025-07-24T04:37:52.6782784Z Download action repository 'actions/upload-artifact@v4' (SHA:ea165f8d65b6e75b540449e92b4886f43607fa02) 2025-07-24T04:37:53.3161805Z ##[group]Run ./.github/actions/upload-test-artifacts 2025-07-24T04:37:53.3162138Z with: 2025-07-24T04:37:53.3162476Z use-gha: true 2025-07-24T04:37:53.3162784Z file-suffix: test-default-1-2-linux.rocm.gpu.2_46615439117 2025-07-24T04:37:53.3163140Z s3-bucket: gha-artifacts 2025-07-24T04:37:53.3163378Z env: 2025-07-24T04:37:53.3163580Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:37:53.3163950Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:37:53.3164499Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:37:53.3165084Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:37:53.3165951Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:37:53.3166712Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:37:53.3166974Z AWS_REGION: us-east-1 2025-07-24T04:37:53.3167271Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:37:53.3167618Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:37:53.3172682Z AWS_SESSION_TOKEN: *** 2025-07-24T04:37:53.3173075Z CONTAINER_NAME: a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:37:53.3173519Z ##[endgroup] 2025-07-24T04:37:53.3243701Z ##[group]Run actions/upload-artifact@v4 2025-07-24T04:37:53.3243987Z with: 2025-07-24T04:37:53.3244342Z name: test-jsons-runattempt1-test-default-1-2-linux.rocm.gpu.2_46615439117.zip 2025-07-24T04:37:53.3244775Z retention-days: 14 2025-07-24T04:37:53.3245026Z if-no-files-found: warn 2025-07-24T04:37:53.3245261Z path: test/**/*.json 2025-07-24T04:37:53.3245485Z compression-level: 6 2025-07-24T04:37:53.3245699Z overwrite: false 2025-07-24T04:37:53.3245922Z include-hidden-files: false 2025-07-24T04:37:53.3246151Z env: 2025-07-24T04:37:53.3246339Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:37:53.3246702Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:37:53.3247239Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:37:53.3247787Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:37:53.3248623Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:37:53.3249365Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:37:53.3249609Z AWS_REGION: us-east-1 2025-07-24T04:37:53.3249892Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:37:53.3250216Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:37:53.3255284Z AWS_SESSION_TOKEN: *** 2025-07-24T04:37:53.3255702Z CONTAINER_NAME: a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:37:53.3256142Z ##[endgroup] 2025-07-24T04:37:54.0391698Z With the provided path, there will be 7 files uploaded 2025-07-24T04:37:54.0397283Z Artifact name is valid! 2025-07-24T04:37:54.0398495Z Root directory input is valid! 2025-07-24T04:37:54.2226472Z Beginning upload of artifact content to blob storage 2025-07-24T04:37:54.4546566Z Uploaded bytes 42778 2025-07-24T04:37:54.4984964Z Finished uploading artifact content to blob storage! 2025-07-24T04:37:54.4987836Z SHA256 digest of uploaded artifact zip is ed5e615c209c343fe6fabbc7370ea6c94411f1dacd8e75dc97e5b80c9eedf56b 2025-07-24T04:37:54.4989180Z Finalizing artifact upload 2025-07-24T04:37:54.6000768Z Artifact test-jsons-runattempt1-test-default-1-2-linux.rocm.gpu.2_46615439117.zip.zip successfully finalized. Artifact ID 3603096284 2025-07-24T04:37:54.6003236Z Artifact test-jsons-runattempt1-test-default-1-2-linux.rocm.gpu.2_46615439117.zip has been successfully uploaded! Final size is 42778 bytes. Artifact ID is 3603096284 2025-07-24T04:37:54.6006907Z Artifact download URL: https://github.com/pytorch/pytorch/actions/runs/16487210679/artifacts/3603096284 2025-07-24T04:37:54.6238465Z ##[group]Run actions/upload-artifact@v4 2025-07-24T04:37:54.6238798Z with: 2025-07-24T04:37:54.6239191Z name: test-reports-runattempt1-test-default-1-2-linux.rocm.gpu.2_46615439117.zip 2025-07-24T04:37:54.6239667Z retention-days: 14 2025-07-24T04:37:54.6239919Z if-no-files-found: ignore 2025-07-24T04:37:54.6240202Z path: test/**/*.xml test/**/*.csv 2025-07-24T04:37:54.6240498Z compression-level: 6 2025-07-24T04:37:54.6240756Z overwrite: false 2025-07-24T04:37:54.6241012Z include-hidden-files: false 2025-07-24T04:37:54.6241284Z env: 2025-07-24T04:37:54.6241550Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:37:54.6241978Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:37:54.6242627Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:37:54.6243169Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:37:54.6244081Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:37:54.6244892Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:37:54.6245181Z AWS_REGION: us-east-1 2025-07-24T04:37:54.6245519Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:37:54.6245871Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:37:54.6250974Z AWS_SESSION_TOKEN: *** 2025-07-24T04:37:54.6251382Z CONTAINER_NAME: a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:37:54.6251808Z ##[endgroup] 2025-07-24T04:37:55.3901254Z With the provided path, there will be 15 files uploaded 2025-07-24T04:37:55.3906492Z Artifact name is valid! 2025-07-24T04:37:55.3907685Z Root directory input is valid! 2025-07-24T04:37:55.5499274Z Beginning upload of artifact content to blob storage 2025-07-24T04:37:55.7906939Z Uploaded bytes 36360 2025-07-24T04:37:55.8390173Z Finished uploading artifact content to blob storage! 2025-07-24T04:37:55.8393165Z SHA256 digest of uploaded artifact zip is 88fc64720ec47427b965d07d27dc15f37273f948429682c78a4fc6239710a295 2025-07-24T04:37:55.8394489Z Finalizing artifact upload 2025-07-24T04:37:55.9352853Z Artifact test-reports-runattempt1-test-default-1-2-linux.rocm.gpu.2_46615439117.zip.zip successfully finalized. Artifact ID 3603096359 2025-07-24T04:37:55.9354915Z Artifact test-reports-runattempt1-test-default-1-2-linux.rocm.gpu.2_46615439117.zip has been successfully uploaded! Final size is 36360 bytes. Artifact ID is 3603096359 2025-07-24T04:37:55.9363054Z Artifact download URL: https://github.com/pytorch/pytorch/actions/runs/16487210679/artifacts/3603096359 2025-07-24T04:37:55.9604950Z ##[group]Run actions/upload-artifact@v4 2025-07-24T04:37:55.9605285Z with: 2025-07-24T04:37:55.9605656Z name: logs-runattempt1-test-default-1-2-linux.rocm.gpu.2_46615439117.zip 2025-07-24T04:37:55.9606093Z retention-days: 14 2025-07-24T04:37:55.9606350Z if-no-files-found: ignore 2025-07-24T04:37:55.9606648Z path: usage_log.txt test/**/*.log 2025-07-24T04:37:55.9606951Z compression-level: 6 2025-07-24T04:37:55.9607200Z overwrite: false 2025-07-24T04:37:55.9607451Z include-hidden-files: false 2025-07-24T04:37:55.9607718Z env: 2025-07-24T04:37:55.9607950Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:37:55.9608356Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:37:55.9608931Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:37:55.9609465Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:37:55.9610898Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:37:55.9611725Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:37:55.9612016Z AWS_REGION: us-east-1 2025-07-24T04:37:55.9612347Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:37:55.9612719Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:37:55.9617805Z AWS_SESSION_TOKEN: *** 2025-07-24T04:37:55.9618216Z CONTAINER_NAME: a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:37:55.9618663Z ##[endgroup] 2025-07-24T04:37:56.7288736Z Multiple search paths detected. Calculating the least common ancestor of all paths 2025-07-24T04:37:56.7290803Z The least common ancestor is /var/home/pytorchci/actions-runner/_work/pytorch/pytorch. This will be the root directory of the artifact 2025-07-24T04:37:56.7291505Z With the provided path, there will be 13 files uploaded 2025-07-24T04:37:56.7296286Z Artifact name is valid! 2025-07-24T04:37:56.7297383Z Root directory input is valid! 2025-07-24T04:37:56.8902126Z Beginning upload of artifact content to blob storage 2025-07-24T04:37:57.1244078Z Uploaded bytes 41148 2025-07-24T04:37:57.1733576Z Finished uploading artifact content to blob storage! 2025-07-24T04:37:57.1736379Z SHA256 digest of uploaded artifact zip is a884213391f0edbc4207dcea9738f004fee2967abba8a85d2198ff56a2296526 2025-07-24T04:37:57.1737811Z Finalizing artifact upload 2025-07-24T04:37:57.2761664Z Artifact logs-runattempt1-test-default-1-2-linux.rocm.gpu.2_46615439117.zip.zip successfully finalized. Artifact ID 3603096445 2025-07-24T04:37:57.2763758Z Artifact logs-runattempt1-test-default-1-2-linux.rocm.gpu.2_46615439117.zip has been successfully uploaded! Final size is 41148 bytes. Artifact ID is 3603096445 2025-07-24T04:37:57.2768038Z Artifact download URL: https://github.com/pytorch/pytorch/actions/runs/16487210679/artifacts/3603096445 2025-07-24T04:37:57.3016710Z ##[group]Run # shellcheck disable=SC2156 2025-07-24T04:37:57.3017150Z # shellcheck disable=SC2156 2025-07-24T04:37:57.3017681Z find . -iname "core.[1-9]*" -exec docker exec "${CONTAINER_NAME}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2025-07-24T04:37:57.3044802Z shell: /usr/bin/bash -e {0} 2025-07-24T04:37:57.3045085Z env: 2025-07-24T04:37:57.3045310Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:37:57.3045718Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:37:57.3046303Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:37:57.3046851Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:37:57.3047728Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:37:57.3048552Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:37:57.3048834Z AWS_REGION: us-east-1 2025-07-24T04:37:57.3049149Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:37:57.3049524Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:37:57.3054603Z AWS_SESSION_TOKEN: *** 2025-07-24T04:37:57.3055035Z CONTAINER_NAME: a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:37:57.3055476Z ##[endgroup] 2025-07-24T04:37:57.7168632Z ##[group]Run aws-actions/configure-aws-credentials@ececac1a45f3b08a01d2dd070d28d111c5fe6722 2025-07-24T04:37:57.7169139Z with: 2025-07-24T04:37:57.7169531Z role-to-assume: arn:aws:iam::308535385114:role/gha_workflow_upload-benchmark-results 2025-07-24T04:37:57.7169987Z role-duration-seconds: 18000 2025-07-24T04:37:57.7170270Z aws-region: us-east-1 2025-07-24T04:37:57.7170547Z audience: sts.amazonaws.com 2025-07-24T04:37:57.7170815Z env: 2025-07-24T04:37:57.7171030Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:37:57.7171443Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:37:57.7172305Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:37:57.7172840Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:37:57.7173779Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:37:57.7174563Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:37:57.7174846Z AWS_REGION: us-east-1 2025-07-24T04:37:57.7175193Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:37:57.7175561Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:37:57.7180651Z AWS_SESSION_TOKEN: *** 2025-07-24T04:37:57.7181070Z CONTAINER_NAME: a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:37:57.7181526Z ##[endgroup] 2025-07-24T04:37:58.0017294Z Assuming role with OIDC 2025-07-24T04:37:58.1553692Z Authenticated as assumedRoleId AROAUPVRELQNA5GQHA6IA:GitHubActions 2025-07-24T04:37:58.2178930Z ##[group]Run pytorch/test-infra/.github/actions/upload-benchmark-results@main 2025-07-24T04:37:58.2179414Z with: 2025-07-24T04:37:58.2179694Z benchmark-results-dir: test/test-reports 2025-07-24T04:37:58.2180021Z dry-run: false 2025-07-24T04:37:58.2180286Z schema-version: v3 2025-07-24T04:37:58.2180813Z github-token: *** 2025-07-24T04:37:58.2181068Z env: 2025-07-24T04:37:58.2181317Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:37:58.2181747Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:37:58.2182335Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:37:58.2182902Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:37:58.2183820Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:37:58.2184676Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:37:58.2184971Z AWS_REGION: us-east-1 2025-07-24T04:37:58.2185273Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:37:58.2185651Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:37:58.2191472Z AWS_SESSION_TOKEN: *** 2025-07-24T04:37:58.2191904Z CONTAINER_NAME: a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:37:58.2192367Z ##[endgroup] 2025-07-24T04:37:58.2211131Z ##[group]Run set -eux 2025-07-24T04:37:58.2211427Z set -eux 2025-07-24T04:37:58.2211802Z python3 -mpip install boto3==1.35.33 psutil==7.0.0 pynvml==12.0.0 2025-07-24T04:37:58.2212218Z  2025-07-24T04:37:58.2212450Z DEVICE_NAME="" 2025-07-24T04:37:58.2212726Z DEVICE_TYPE="" 2025-07-24T04:37:58.2212976Z  2025-07-24T04:37:58.2213229Z if command -v nvidia-smi; then 2025-07-24T04:37:58.2213679Z  # NB: I'm using PyTorch here to get the device name, however, it needs to 2025-07-24T04:37:58.2214215Z  # install the correct version of PyTorch manually for now. Any PyTorch 2025-07-24T04:37:58.2214736Z  # version is fine, I just use 2.7.1 to satify PYPIDEP linter 2025-07-24T04:37:58.2215158Z  python3 -mpip install torch==2.7.1 2025-07-24T04:37:58.2215493Z elif command -v rocminfo; then 2025-07-24T04:37:58.2215920Z  # NB: Installing torch on ROCm runner with pip here causes CI to fail 2025-07-24T04:37:58.2216433Z  # with a memoryview is too large error only on MI300 runners. Is pip 2025-07-24T04:37:58.2216948Z  # version on ROCm runner there too old? As a workaround, let's use the 2025-07-24T04:37:58.2217421Z  # GPU device name coming from rocminfo instead 2025-07-24T04:37:58.2217785Z  DEVICE_NAME=rocm 2025-07-24T04:37:58.2218251Z  DEVICE_TYPE=$(rocminfo | grep "Marketing Name" | tail -n1 | awk -F':' '{print $2}' | xargs) 2025-07-24T04:37:58.2218720Z fi 2025-07-24T04:37:58.2218946Z  2025-07-24T04:37:58.2219468Z echo "DEVICE_NAME=$DEVICE_NAME" >> $GITHUB_ENV 2025-07-24T04:37:58.2219864Z echo "DEVICE_TYPE=$DEVICE_TYPE" >> $GITHUB_ENV 2025-07-24T04:37:58.2249914Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:37:58.2250298Z env: 2025-07-24T04:37:58.2250537Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:37:58.2250946Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:37:58.2251552Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:37:58.2252115Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:37:58.2252992Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:37:58.2253810Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:37:58.2254325Z AWS_REGION: us-east-1 2025-07-24T04:37:58.2254641Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:37:58.2255015Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:37:58.2260869Z AWS_SESSION_TOKEN: *** 2025-07-24T04:37:58.2261320Z CONTAINER_NAME: a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:37:58.2261771Z ##[endgroup] 2025-07-24T04:37:58.2314346Z + python3 -mpip install boto3==1.35.33 psutil==7.0.0 pynvml==12.0.0 2025-07-24T04:37:58.5345598Z Defaulting to user installation because normal site-packages is not writeable 2025-07-24T04:37:58.6455425Z Requirement already satisfied: boto3==1.35.33 in /var/home/pytorchci/.local/lib/python3.10/site-packages (1.35.33) 2025-07-24T04:37:58.6460350Z Requirement already satisfied: psutil==7.0.0 in /var/home/pytorchci/.local/lib/python3.10/site-packages (7.0.0) 2025-07-24T04:37:58.6466053Z Requirement already satisfied: pynvml==12.0.0 in /var/home/pytorchci/.local/lib/python3.10/site-packages (12.0.0) 2025-07-24T04:37:58.6506685Z Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /usr/lib/python3/dist-packages (from boto3==1.35.33) (0.10.0) 2025-07-24T04:37:58.6513209Z Requirement already satisfied: s3transfer<0.11.0,>=0.10.0 in /var/home/pytorchci/.local/lib/python3.10/site-packages (from boto3==1.35.33) (0.10.4) 2025-07-24T04:37:58.6518249Z Requirement already satisfied: botocore<1.36.0,>=1.35.33 in /var/home/pytorchci/.local/lib/python3.10/site-packages (from boto3==1.35.33) (1.35.99) 2025-07-24T04:37:58.6709302Z Requirement already satisfied: nvidia-ml-py<13.0.0a0,>=12.0.0 in /var/home/pytorchci/.local/lib/python3.10/site-packages (from pynvml==12.0.0) (12.575.51) 2025-07-24T04:37:58.6769174Z Requirement already satisfied: urllib3!=2.2.0,<3,>=1.25.4 in /usr/lib/python3/dist-packages (from botocore<1.36.0,>=1.35.33->boto3==1.35.33) (1.26.5) 2025-07-24T04:37:58.6774458Z Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /var/home/pytorchci/.local/lib/python3.10/site-packages (from botocore<1.36.0,>=1.35.33->boto3==1.35.33) (2.9.0.post0) 2025-07-24T04:37:58.6826793Z Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.36.0,>=1.35.33->boto3==1.35.33) (1.16.0) 2025-07-24T04:37:59.0228453Z + DEVICE_NAME= 2025-07-24T04:37:59.0228908Z + DEVICE_TYPE= 2025-07-24T04:37:59.0229359Z + command -v nvidia-smi 2025-07-24T04:37:59.0229841Z + command -v rocminfo 2025-07-24T04:37:59.0230616Z /opt/rocm/bin/rocminfo 2025-07-24T04:37:59.0231078Z + DEVICE_NAME=rocm 2025-07-24T04:37:59.0242079Z ++ rocminfo 2025-07-24T04:37:59.0245663Z ++ grep 'Marketing Name' 2025-07-24T04:37:59.0246968Z ++ tail -n1 2025-07-24T04:37:59.0249910Z ++ awk -F: '{print $2}' 2025-07-24T04:37:59.0251999Z ++ xargs 2025-07-24T04:37:59.1733651Z + DEVICE_TYPE='AMD Instinct MI250X/MI250' 2025-07-24T04:37:59.1734251Z + echo DEVICE_NAME=rocm 2025-07-24T04:37:59.1734767Z + echo 'DEVICE_TYPE=AMD Instinct MI250X/MI250' 2025-07-24T04:37:59.1759840Z ##[group]Run set -eux 2025-07-24T04:37:59.1760133Z set -eux 2025-07-24T04:37:59.1760659Z  2025-07-24T04:37:59.1760918Z if [[ -z "${GITHUB_TOKEN}" ]]; then 2025-07-24T04:37:59.1761274Z  echo "Missing github-token input" 2025-07-24T04:37:59.1761580Z  exit 1 2025-07-24T04:37:59.1761819Z fi 2025-07-24T04:37:59.1790164Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:37:59.1790550Z env: 2025-07-24T04:37:59.1790787Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:37:59.1791210Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:37:59.1791812Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:37:59.1792364Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:37:59.1793255Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:37:59.1794292Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:37:59.1794593Z AWS_REGION: us-east-1 2025-07-24T04:37:59.1794911Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:37:59.1795291Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:37:59.1801157Z AWS_SESSION_TOKEN: *** 2025-07-24T04:37:59.1801588Z CONTAINER_NAME: a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:37:59.1802039Z DEVICE_NAME: rocm 2025-07-24T04:37:59.1802298Z DEVICE_TYPE: AMD Instinct MI250X/MI250 2025-07-24T04:37:59.1802839Z GITHUB_TOKEN: *** 2025-07-24T04:37:59.1803084Z ##[endgroup] 2025-07-24T04:37:59.1855871Z + [[ -z *** ]] 2025-07-24T04:37:59.1903334Z ##[group]Run pytorch/test-infra/.github/actions/get-workflow-job-id@main 2025-07-24T04:37:59.1903778Z with: 2025-07-24T04:37:59.1904143Z github-token: *** 2025-07-24T04:37:59.1904403Z env: 2025-07-24T04:37:59.1904644Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:37:59.1905059Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:37:59.1925282Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:37:59.1925889Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:37:59.1926804Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:37:59.1927620Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:37:59.1927936Z AWS_REGION: us-east-1 2025-07-24T04:37:59.1928252Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:37:59.1928649Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:37:59.1934833Z AWS_SESSION_TOKEN: *** 2025-07-24T04:37:59.1935280Z CONTAINER_NAME: a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:37:59.1935743Z DEVICE_NAME: rocm 2025-07-24T04:37:59.1936025Z DEVICE_TYPE: AMD Instinct MI250X/MI250 2025-07-24T04:37:59.1936342Z ##[endgroup] 2025-07-24T04:37:59.1953159Z ##[group]Run set -eux 2025-07-24T04:37:59.1953469Z set -eux 2025-07-24T04:37:59.1953717Z  2025-07-24T04:37:59.1954167Z python3 "${GITHUB_ACTION_PATH}/../../scripts/get_workflow_job_id.py" "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2025-07-24T04:37:59.1982208Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:37:59.1982593Z env: 2025-07-24T04:37:59.1982840Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:37:59.1983263Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:37:59.1983861Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:37:59.1984416Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:37:59.1985315Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:37:59.1986127Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:37:59.1986678Z AWS_REGION: us-east-1 2025-07-24T04:37:59.1986989Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:37:59.1987387Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:37:59.1993268Z AWS_SESSION_TOKEN: *** 2025-07-24T04:37:59.1993707Z CONTAINER_NAME: a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:37:59.1994168Z DEVICE_NAME: rocm 2025-07-24T04:37:59.1994456Z DEVICE_TYPE: AMD Instinct MI250X/MI250 2025-07-24T04:37:59.1994889Z GITHUB_TOKEN: *** 2025-07-24T04:37:59.1995143Z ##[endgroup] 2025-07-24T04:37:59.2045489Z + python3 /var/home/pytorchci/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/get-workflow-job-id/../../scripts/get_workflow_job_id.py 16487210679 gpud4f2 2025-07-24T04:38:00.1277166Z setting job-id=46615439117 2025-07-24T04:38:00.1277990Z setting job-name=linux-jammy-rocm-py3.10 / test (default, 1, 2, linux.rocm.gpu.2) 2025-07-24T04:38:00.1444833Z ##[group]Run set -eux 2025-07-24T04:38:00.1445401Z set -eux 2025-07-24T04:38:00.1445680Z  2025-07-24T04:38:00.1446067Z python3 "${GITHUB_ACTION_PATH}/../../scripts/benchmarks/gather_metadata.py" \ 2025-07-24T04:38:00.1446579Z  --schema-version "${SCHEMA_VERSION}" \ 2025-07-24T04:38:00.1446927Z  --repo "${REPO}" \ 2025-07-24T04:38:00.1447242Z  --head-branch "${HEAD_BRANCH}" \ 2025-07-24T04:38:00.1447576Z  --head-sha "${HEAD_SHA}" \ 2025-07-24T04:38:00.1447936Z  --workflow-id "${WORKFLOW_RUN_ID}" \ 2025-07-24T04:38:00.1448300Z  --run-attempt "${RUN_ATTEMPT}" \ 2025-07-24T04:38:00.1448643Z  --job-id "${JOB_ID}" \ 2025-07-24T04:38:00.1448945Z  --job-name "${JOB_NAME}" 2025-07-24T04:38:00.1476937Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:38:00.1477333Z env: 2025-07-24T04:38:00.1477578Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:38:00.1478008Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:38:00.1478635Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:38:00.1479200Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:38:00.1480085Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:38:00.1480916Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:38:00.1481226Z AWS_REGION: us-east-1 2025-07-24T04:38:00.1481574Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:38:00.1481971Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:38:00.1487905Z AWS_SESSION_TOKEN: *** 2025-07-24T04:38:00.1488343Z CONTAINER_NAME: a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:38:00.1488806Z DEVICE_NAME: rocm 2025-07-24T04:38:00.1489090Z DEVICE_TYPE: AMD Instinct MI250X/MI250 2025-07-24T04:38:00.1489407Z SCHEMA_VERSION: v3 2025-07-24T04:38:00.1489700Z REPO: pytorch/pytorch 2025-07-24T04:38:00.1489999Z HEAD_BRANCH: refs/tags/ciflow/trunk/149961 2025-07-24T04:38:00.1490356Z HEAD_SHA: 27c8ef1839e0747b36769efb192befc9308ff93c 2025-07-24T04:38:00.1490701Z WORKFLOW_RUN_ID: 16487210679 2025-07-24T04:38:00.1490993Z RUN_ATTEMPT: 1 2025-07-24T04:38:00.1491227Z JOB_ID: 46615439117 2025-07-24T04:38:00.1491603Z JOB_NAME: linux-jammy-rocm-py3.10 / test (default, 1, 2, linux.rocm.gpu.2) 2025-07-24T04:38:00.1492018Z ##[endgroup] 2025-07-24T04:38:00.1544208Z + python3 /var/home/pytorchci/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/upload-benchmark-results/../../scripts/benchmarks/gather_metadata.py --schema-version v3 --repo pytorch/pytorch --head-branch refs/tags/ciflow/trunk/149961 --head-sha 27c8ef1839e0747b36769efb192befc9308ff93c --workflow-id 16487210679 --run-attempt 1 --job-id 46615439117 --job-name 'linux-jammy-rocm-py3.10 / test (default, 1, 2, linux.rocm.gpu.2)' 2025-07-24T04:38:00.1875018Z ##[group]Run set -eux 2025-07-24T04:38:00.1875572Z set -eux 2025-07-24T04:38:00.1875816Z  2025-07-24T04:38:00.1876230Z python3 "${GITHUB_ACTION_PATH}/../../scripts/benchmarks/gather_runners_info.py" 2025-07-24T04:38:00.1902067Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:38:00.1902474Z env: 2025-07-24T04:38:00.1902719Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:38:00.1903176Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:38:00.1903789Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:38:00.1904349Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:38:00.1905237Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:38:00.1906281Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:38:00.1906607Z AWS_REGION: us-east-1 2025-07-24T04:38:00.1906944Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:38:00.1907341Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:38:00.1913218Z AWS_SESSION_TOKEN: *** 2025-07-24T04:38:00.1913643Z CONTAINER_NAME: a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:38:00.1914098Z DEVICE_NAME: rocm 2025-07-24T04:38:00.1914369Z DEVICE_TYPE: AMD Instinct MI250X/MI250 2025-07-24T04:38:00.1914677Z ##[endgroup] 2025-07-24T04:38:00.1962445Z + python3 /var/home/pytorchci/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/upload-benchmark-results/../../scripts/benchmarks/gather_runners_info.py 2025-07-24T04:38:01.0057433Z /var/home/pytorchci/.local/lib/python3.10/site-packages/torch/_subclasses/functional_tensor.py:276: UserWarning: Failed to initialize NumPy: No module named 'numpy' (Triggered internally at /pytorch/torch/csrc/utils/tensor_numpy.cpp:81.) 2025-07-24T04:38:01.0058863Z cpu = _conversion_method_template(device=torch.device("cpu")) 2025-07-24T04:38:01.6627317Z ##[group]Run set -eux 2025-07-24T04:38:01.6627634Z set -eux 2025-07-24T04:38:01.6627885Z  2025-07-24T04:38:01.6628158Z # TODO (huydhn): Implement this part 2025-07-24T04:38:01.6628562Z echo "dependencies={}" >> "${GITHUB_OUTPUT}" 2025-07-24T04:38:01.6657879Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:38:01.6658274Z env: 2025-07-24T04:38:01.6658526Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:38:01.6658969Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:38:01.6659582Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:38:01.6660163Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:38:01.6661121Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:38:01.6661959Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:38:01.6662263Z AWS_REGION: us-east-1 2025-07-24T04:38:01.6662614Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:38:01.6663002Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:38:01.6668829Z AWS_SESSION_TOKEN: *** 2025-07-24T04:38:01.6669263Z CONTAINER_NAME: a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:38:01.6669722Z DEVICE_NAME: rocm 2025-07-24T04:38:01.6670007Z DEVICE_TYPE: AMD Instinct MI250X/MI250 2025-07-24T04:38:01.6670332Z ##[endgroup] 2025-07-24T04:38:01.6720638Z + echo 'dependencies={}' 2025-07-24T04:38:01.6754462Z ##[group]Run set -eux 2025-07-24T04:38:01.6754787Z set -eux 2025-07-24T04:38:01.6755038Z  2025-07-24T04:38:01.6755326Z if [[ ! -d "${BENCHMARK_RESULTS_DIR}" ]]; then 2025-07-24T04:38:01.6755751Z  echo "${BENCHMARK_RESULTS_DIR} does not exist, skipping" 2025-07-24T04:38:01.6756260Z  # We don't want the job to fail if the directory doesn't exist 2025-07-24T04:38:01.6756930Z  exit 0 2025-07-24T04:38:01.6757167Z fi 2025-07-24T04:38:01.6757388Z  2025-07-24T04:38:01.6757619Z if [[ "${DRY_RUN}" == "true" ]]; then 2025-07-24T04:38:01.6758073Z  python3 "${GITHUB_ACTION_PATH}/../../scripts/upload_benchmark_results.py" \ 2025-07-24T04:38:01.6758593Z  --benchmark-results-dir "${BENCHMARK_RESULTS_DIR}" \ 2025-07-24T04:38:01.6758989Z  --metadata "${BENCHMARK_METADATA}" \ 2025-07-24T04:38:01.6759329Z  --runners "${RUNNER_INFO}" \ 2025-07-24T04:38:01.6759674Z  --dependencies "${DEPENDENCIES}" \ 2025-07-24T04:38:01.6759987Z  --dry-run 2025-07-24T04:38:01.6760245Z else 2025-07-24T04:38:01.6760609Z  python3 "${GITHUB_ACTION_PATH}/../../scripts/upload_benchmark_results.py" \ 2025-07-24T04:38:01.6761322Z  --benchmark-results-dir "${BENCHMARK_RESULTS_DIR}" \ 2025-07-24T04:38:01.6761734Z  --metadata "${BENCHMARK_METADATA}" \ 2025-07-24T04:38:01.6762070Z  --runners "${RUNNER_INFO}" \ 2025-07-24T04:38:01.6762653Z  --dependencies "${DEPENDENCIES}" 2025-07-24T04:38:01.6762973Z fi 2025-07-24T04:38:01.6791124Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:38:01.6791519Z env: 2025-07-24T04:38:01.6791758Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:38:01.6792174Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:38:01.6792752Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:38:01.6793309Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:38:01.6794208Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:38:01.6795010Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:38:01.6795306Z AWS_REGION: us-east-1 2025-07-24T04:38:01.6795671Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:38:01.6796065Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:38:01.6801928Z AWS_SESSION_TOKEN: *** 2025-07-24T04:38:01.6802446Z CONTAINER_NAME: a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:38:01.6802900Z DEVICE_NAME: rocm 2025-07-24T04:38:01.6803180Z DEVICE_TYPE: AMD Instinct MI250X/MI250 2025-07-24T04:38:01.6803518Z BENCHMARK_RESULTS_DIR: test/test-reports 2025-07-24T04:38:01.6803828Z DRY_RUN: false 2025-07-24T04:38:01.6804970Z BENCHMARK_METADATA: {"timestamp": 1753331880, "schema_version": "v3", "name": "linux-jammy-rocm-py3.10 / test (default, 1, 2, linux.rocm.gpu.2)", "repo": "pytorch/pytorch", "head_branch": "refs/tags/ciflow/trunk/149961", "head_sha": "27c8ef1839e0747b36769efb192befc9308ff93c", "workflow_id": 16487210679, "run_attempt": 1, "job_id": 46615439117} 2025-07-24T04:38:01.6806549Z RUNNER_INFO: [{"cpu_info": "x86_64", "cpu_count": 128, "avail_mem_in_gb": 1007, "extra_info": {"hostname": "gpud4f2.jax.cs.cpe.ice.amd.com"}, "name": "rocm", "type": "AMD Instinct MI250X/MI250"}] 2025-07-24T04:38:01.6807217Z DEPENDENCIES: {} 2025-07-24T04:38:01.6807457Z ##[endgroup] 2025-07-24T04:38:01.6860785Z + [[ ! -d test/test-reports ]] 2025-07-24T04:38:01.6861142Z + [[ false == \t\r\u\e ]] 2025-07-24T04:38:01.6863867Z + python3 /var/home/pytorchci/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/upload-benchmark-results/../../scripts/upload_benchmark_results.py --benchmark-results-dir test/test-reports --metadata '{"timestamp": 1753331880, "schema_version": "v3", "name": "linux-jammy-rocm-py3.10 / test (default, 1, 2, linux.rocm.gpu.2)", "repo": "pytorch/pytorch", "head_branch": "refs/tags/ciflow/trunk/149961", "head_sha": "27c8ef1839e0747b36769efb192befc9308ff93c", "workflow_id": 16487210679, "run_attempt": 1, "job_id": 46615439117}' --runners '[{"cpu_info": "x86_64", "cpu_count": 128, "avail_mem_in_gb": 1007, "extra_info": {"hostname": "gpud4f2.jax.cs.cpe.ice.amd.com"}, "name": "rocm", "type": "AMD Instinct MI250X/MI250"}]' --dependencies '{}' 2025-07-24T04:38:01.8389150Z /var/home/pytorchci/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/upload-benchmark-results/../../scripts/upload_benchmark_results.py:236: UserWarning: {'included': [{'test_file': 'cpp/Dict_test'}, {'test_file': 'cpp/Dimname_test'}, {'test_file': 'cpp/NamedTensor_test'}, {'test_file': 'cpp/apply_utils_test'}, {'test_file': 'cpp/atest'}], 'excluded': [{'test_file': 'cpp/basic'}, {'test_file': 'cpp/broadcast_test'}, {'test_file': 'cpp/cpu_generator_test'}, {'test_file': 'cpp/dlconvertor_test'}, {'test_file': 'cpp/extension_backend_test'}, {'test_file': 'cpp/lazy_tensor_test'}, {'test_file': 'cpp/legacy_vmap_test'}, {'test_file': 'cpp/native_test'}, {'test_file': 'cpp/operators_test'}, {'test_file': 'cpp/scalar_tensor_test'}, {'test_file': 'cpp/scalar_test'}, {'test_file': 'cpp/tensor_iterator_test'}, {'test_file': 'cpp/undefined_tensor_test'}, {'test_file': 'cpp/wrapdim_test'}]} from test/test-reports/td_exclusions-57efd3df1d784e2e256a.json is not a benchmark record, skipping 2025-07-24T04:38:01.8393805Z warn(f"{result} from {filepath} is not a benchmark record, skipping") 2025-07-24T04:38:01.8395761Z /var/home/pytorchci/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/upload-benchmark-results/../../scripts/upload_benchmark_results.py:236: UserWarning: {'included': [{'test_file': 'inductor/test_torchinductor'}, {'test_file': 'test_autograd'}, {'test_file': 'test_torch'}], 'excluded': [{'test_file': 'test_ops'}, {'test_file': 'test_binary_ufuncs'}, {'test_file': 'test_nn'}, {'test_file': 'test_unary_ufuncs'}, {'test_file': 'test_cuda'}]} from test/test-reports/td_exclusions-ff2bb2aa9c58ae082e66.json is not a benchmark record, skipping 2025-07-24T04:38:01.8397677Z warn(f"{result} from {filepath} is not a benchmark record, skipping") 2025-07-24T04:38:01.8634081Z Prepare all required actions 2025-07-24T04:38:01.8634593Z Getting action download info 2025-07-24T04:38:01.8662677Z ##[group]Run ./.github/actions/teardown-rocm 2025-07-24T04:38:01.8663017Z env: 2025-07-24T04:38:01.8663246Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:38:01.8663664Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:38:01.8664259Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:38:01.8664816Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:38:01.8665720Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:38:01.8666527Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:38:01.8666822Z AWS_REGION: us-east-1 2025-07-24T04:38:01.8667162Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:38:01.8667621Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:38:01.8673495Z AWS_SESSION_TOKEN: *** 2025-07-24T04:38:01.8673929Z CONTAINER_NAME: a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:38:01.8674395Z DEVICE_NAME: rocm 2025-07-24T04:38:01.8674658Z DEVICE_TYPE: AMD Instinct MI250X/MI250 2025-07-24T04:38:01.8674971Z ##[endgroup] 2025-07-24T04:38:01.8690775Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2025-07-24T04:38:01.8691461Z # ignore expansion of "docker ps -q" since it could be empty 2025-07-24T04:38:01.8692253Z # shellcheck disable=SC2046 2025-07-24T04:38:01.8692756Z docker stop $(docker ps -q) || true 2025-07-24T04:38:01.8693293Z # Prune all stopped containers. 2025-07-24T04:38:01.8715804Z docker container prune -f 2025-07-24T04:38:01.8741321Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:38:01.8741725Z env: 2025-07-24T04:38:01.8741978Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:38:01.8742417Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:38:01.8743291Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:38:01.8743867Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:38:01.8744766Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:38:01.8745576Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:38:01.8745867Z AWS_REGION: us-east-1 2025-07-24T04:38:01.8746188Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:38:01.8746585Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:38:01.8752474Z AWS_SESSION_TOKEN: *** 2025-07-24T04:38:01.8752922Z CONTAINER_NAME: a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:38:01.8753389Z DEVICE_NAME: rocm 2025-07-24T04:38:01.8753663Z DEVICE_TYPE: AMD Instinct MI250X/MI250 2025-07-24T04:38:01.8753988Z ##[endgroup] 2025-07-24T04:38:12.6166765Z a4a81da2c3fc 2025-07-24T04:38:20.8305438Z Deleted Containers: 2025-07-24T04:38:20.8306282Z a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:38:20.8306874Z 2025-07-24T04:38:20.8307098Z Total reclaimed space: 9.985GB 2025-07-24T04:38:20.8375575Z Prepare all required actions 2025-07-24T04:38:20.8404855Z ##[group]Run ./.github/actions/diskspace-cleanup 2025-07-24T04:38:20.8405194Z with: 2025-07-24T04:38:20.8405435Z diskspace-cutoff: 70 2025-07-24T04:38:20.8405692Z env: 2025-07-24T04:38:20.8405914Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:38:20.8406317Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:38:20.8406924Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:38:20.8407473Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:38:20.8408668Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:38:20.8409525Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:38:20.8409820Z AWS_REGION: us-east-1 2025-07-24T04:38:20.8410135Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:38:20.8410519Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:38:20.8416439Z AWS_SESSION_TOKEN: *** 2025-07-24T04:38:20.8416871Z CONTAINER_NAME: a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:38:20.8417320Z DEVICE_NAME: rocm 2025-07-24T04:38:20.8417589Z DEVICE_TYPE: AMD Instinct MI250X/MI250 2025-07-24T04:38:20.8417897Z ##[endgroup] 2025-07-24T04:38:20.8433588Z ##[group]Run set -ex 2025-07-24T04:38:20.8433877Z set -ex 2025-07-24T04:38:20.8434123Z diskspace_cutoff=70 2025-07-24T04:38:20.8434500Z docker_root_dir=$(docker info -f '{{.DockerRootDir}}') 2025-07-24T04:38:20.8434898Z if [ ! -d "$docker_root_dir" ]; then 2025-07-24T04:38:20.8435403Z  echo "Docker root directory ($docker_root_dir) does not exist. Skipping disk space check." 2025-07-24T04:38:20.8435868Z  exit 0 2025-07-24T04:38:20.8436101Z fi 2025-07-24T04:38:20.8436508Z diskspace=$(df -H --output=pcent ${docker_root_dir} | sed -n 2p | sed 's/%//' | sed 's/ //') 2025-07-24T04:38:20.8437311Z msg="Please file an issue on pytorch/pytorch reporting the faulty runner. Include a link to the runner logs so the runner can be identified" 2025-07-24T04:38:20.8438004Z if [[ "$diskspace" -ge "$diskspace_cutoff" ]] ; then 2025-07-24T04:38:20.8438377Z  docker system prune -af 2025-07-24T04:38:20.8438862Z  diskspace_new=$(df -H --output=pcent ${docker_root_dir} | sed -n 2p | sed 's/%//' | sed 's/ //') 2025-07-24T04:38:20.8439390Z  if [[ "$diskspace_new" -gt "$diskspace_cutoff" ]] ; then 2025-07-24T04:38:20.8439955Z  echo "Error: Available diskspace is less than $diskspace_cutoff percent. Not enough diskspace." 2025-07-24T04:38:20.8440696Z  echo "$msg" 2025-07-24T04:38:20.8440964Z  exit 1 2025-07-24T04:38:20.8441213Z  else 2025-07-24T04:38:20.8441500Z  difference=$((diskspace - diskspace_new)) 2025-07-24T04:38:20.8441903Z  echo "Diskspace saved: $difference percent" 2025-07-24T04:38:20.8442241Z  fi 2025-07-24T04:38:20.8442534Z fi 2025-07-24T04:38:20.8471929Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-07-24T04:38:20.8472304Z env: 2025-07-24T04:38:20.8472532Z GIT_DEFAULT_BRANCH: main 2025-07-24T04:38:20.8472953Z RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts 2025-07-24T04:38:20.8473541Z RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results 2025-07-24T04:38:20.8474095Z RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs 2025-07-24T04:38:20.8474991Z GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host 2025-07-24T04:38:20.8475806Z AWS_DEFAULT_REGION: us-east-1 2025-07-24T04:38:20.8476102Z AWS_REGION: us-east-1 2025-07-24T04:38:20.8476410Z AWS_ACCESS_KEY_ID: *** 2025-07-24T04:38:20.8476783Z AWS_SECRET_ACCESS_KEY: *** 2025-07-24T04:38:20.8482746Z AWS_SESSION_TOKEN: *** 2025-07-24T04:38:20.8483162Z CONTAINER_NAME: a4a81da2c3fc1d001b78d9c9c45c38ba808983297131b9acf30556dfb2e92835 2025-07-24T04:38:20.8483614Z DEVICE_NAME: rocm 2025-07-24T04:38:20.8483874Z DEVICE_TYPE: AMD Instinct MI250X/MI250 2025-07-24T04:38:20.8484168Z ##[endgroup] 2025-07-24T04:38:20.8535740Z + diskspace_cutoff=70 2025-07-24T04:38:20.8541546Z ++ docker info -f '{{.DockerRootDir}}' 2025-07-24T04:38:20.9103919Z + docker_root_dir=/media/4TB/docker-rootless 2025-07-24T04:38:20.9104593Z + '[' '!' -d /media/4TB/docker-rootless ']' 2025-07-24T04:38:20.9116589Z ++ df -H --output=pcent /media/4TB/docker-rootless 2025-07-24T04:38:20.9118759Z ++ sed -n 2p 2025-07-24T04:38:20.9119387Z ++ sed s/%// 2025-07-24T04:38:20.9122124Z ++ sed 's/ //' 2025-07-24T04:38:20.9149127Z + diskspace=65 2025-07-24T04:38:20.9150105Z + msg='Please file an issue on pytorch/pytorch reporting the faulty runner. Include a link to the runner logs so the runner can be identified' 2025-07-24T04:38:20.9151140Z + [[ 65 -ge 70 ]] 2025-07-24T04:38:20.9199601Z Post job cleanup. 2025-07-24T04:38:20.9246782Z Post job cleanup. 2025-07-24T04:38:21.0566482Z Post job cleanup. 2025-07-24T04:38:21.0968948Z Logging out of registry 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-07-24T04:38:21.1354967Z Post job cleanup. 2025-07-24T04:38:21.2730491Z Post job cleanup. 2025-07-24T04:38:21.2771865Z Post job cleanup. 2025-07-24T04:38:21.3792952Z [command]/usr/bin/git version 2025-07-24T04:38:21.3837803Z git version 2.34.1 2025-07-24T04:38:21.3873672Z Copying '/var/home/pytorchci/.gitconfig' to '/var/home/pytorchci/actions-runner/_work/_temp/df94569a-2215-40fc-9465-21e1cb7b868c/.gitconfig' 2025-07-24T04:38:21.3885617Z Temporarily overriding HOME='/var/home/pytorchci/actions-runner/_work/_temp/df94569a-2215-40fc-9465-21e1cb7b868c' before making global git config changes 2025-07-24T04:38:21.3886517Z Adding repository directory to the temporary git global config as a safe directory 2025-07-24T04:38:21.3898139Z [command]/usr/bin/git config --global --add safe.directory /var/home/pytorchci/actions-runner/_work/pytorch/pytorch 2025-07-24T04:38:21.3942547Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2025-07-24T04:38:21.3986399Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2025-07-24T04:38:21.4401438Z Entering 'android/libs/fbjni' 2025-07-24T04:38:21.4479178Z Entering 'third_party/FP16' 2025-07-24T04:38:21.4556165Z Entering 'third_party/FXdiv' 2025-07-24T04:38:21.4634826Z Entering 'third_party/NNPACK' 2025-07-24T04:38:21.4708076Z Entering 'third_party/NVTX' 2025-07-24T04:38:21.4787917Z Entering 'third_party/VulkanMemoryAllocator' 2025-07-24T04:38:21.4866806Z Entering 'third_party/XNNPACK' 2025-07-24T04:38:21.4963667Z Entering 'third_party/aiter' 2025-07-24T04:38:21.5039374Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-07-24T04:38:21.5126419Z Entering 'third_party/benchmark' 2025-07-24T04:38:21.5200623Z Entering 'third_party/composable_kernel' 2025-07-24T04:38:21.5283793Z Entering 'third_party/cpp-httplib' 2025-07-24T04:38:21.5356134Z Entering 'third_party/cpuinfo' 2025-07-24T04:38:21.5430329Z Entering 'third_party/cudnn_frontend' 2025-07-24T04:38:21.5508791Z Entering 'third_party/cutlass' 2025-07-24T04:38:21.5599261Z Entering 'third_party/fbgemm' 2025-07-24T04:38:21.5676277Z Entering 'third_party/fbgemm/external/asmjit' 2025-07-24T04:38:21.5748303Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-07-24T04:38:21.5830872Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-07-24T04:38:21.5908300Z Entering 'third_party/fbgemm/external/cutlass' 2025-07-24T04:38:21.5991729Z Entering 'third_party/fbgemm/external/googletest' 2025-07-24T04:38:21.6062000Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-07-24T04:38:21.6138954Z Entering 'third_party/fbgemm/external/json' 2025-07-24T04:38:21.6215297Z Entering 'third_party/flash-attention' 2025-07-24T04:38:21.6281159Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-07-24T04:38:21.6359791Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-07-24T04:38:21.6447387Z Entering 'third_party/flatbuffers' 2025-07-24T04:38:21.6525174Z Entering 'third_party/fmt' 2025-07-24T04:38:21.6598507Z Entering 'third_party/gemmlowp/gemmlowp' 2025-07-24T04:38:21.6670892Z Entering 'third_party/gloo' 2025-07-24T04:38:21.6744666Z Entering 'third_party/googletest' 2025-07-24T04:38:21.6822724Z Entering 'third_party/ideep' 2025-07-24T04:38:21.6900482Z Entering 'third_party/ideep/mkl-dnn' 2025-07-24T04:38:21.6981689Z Entering 'third_party/ittapi' 2025-07-24T04:38:21.7061832Z Entering 'third_party/kineto' 2025-07-24T04:38:21.7139132Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-07-24T04:38:21.7209542Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-07-24T04:38:21.7280590Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-07-24T04:38:21.7349373Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-07-24T04:38:21.7421070Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-07-24T04:38:21.7491371Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-07-24T04:38:21.7559572Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-07-24T04:38:21.7627406Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-07-24T04:38:21.7694136Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-07-24T04:38:21.7772422Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-07-24T04:38:21.7842891Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-07-24T04:38:21.7908616Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-07-24T04:38:21.7981476Z Entering 'third_party/kleidiai' 2025-07-24T04:38:21.8054598Z Entering 'third_party/mimalloc' 2025-07-24T04:38:21.8134994Z Entering 'third_party/nlohmann' 2025-07-24T04:38:21.8216635Z Entering 'third_party/onnx' 2025-07-24T04:38:21.8314332Z Entering 'third_party/onnx/third_party/pybind11' 2025-07-24T04:38:21.8397046Z Entering 'third_party/opentelemetry-cpp' 2025-07-24T04:38:21.8470938Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-07-24T04:38:21.8542334Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-07-24T04:38:21.8613990Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-07-24T04:38:21.8674998Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-07-24T04:38:21.8745600Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-07-24T04:38:21.8816731Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-07-24T04:38:21.8893122Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-07-24T04:38:21.8959305Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-07-24T04:38:21.9031572Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-07-24T04:38:21.9108956Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-07-24T04:38:21.9203482Z Entering 'third_party/pocketfft' 2025-07-24T04:38:21.9276570Z Entering 'third_party/protobuf' 2025-07-24T04:38:21.9354770Z Entering 'third_party/protobuf/third_party/benchmark' 2025-07-24T04:38:21.9425194Z Entering 'third_party/protobuf/third_party/googletest' 2025-07-24T04:38:21.9504811Z Entering 'third_party/psimd' 2025-07-24T04:38:21.9584611Z Entering 'third_party/pthreadpool' 2025-07-24T04:38:21.9659617Z Entering 'third_party/pybind11' 2025-07-24T04:38:21.9737790Z Entering 'third_party/python-peachpy' 2025-07-24T04:38:21.9817198Z Entering 'third_party/sleef' 2025-07-24T04:38:21.9897689Z Entering 'third_party/tensorpipe' 2025-07-24T04:38:21.9976342Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-07-24T04:38:22.0044463Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-07-24T04:38:22.0112099Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-07-24T04:38:22.0180851Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-07-24T04:38:22.0254282Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-07-24T04:38:22.0362648Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2025-07-24T04:38:22.0398976Z http.https://github.com/.extraheader 2025-07-24T04:38:22.0410085Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2025-07-24T04:38:22.0455227Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2025-07-24T04:38:22.0867669Z Entering 'android/libs/fbjni' 2025-07-24T04:38:22.0909696Z http.https://github.com/.extraheader 2025-07-24T04:38:22.0961534Z Entering 'third_party/FP16' 2025-07-24T04:38:22.1004378Z http.https://github.com/.extraheader 2025-07-24T04:38:22.1063491Z Entering 'third_party/FXdiv' 2025-07-24T04:38:22.1104665Z http.https://github.com/.extraheader 2025-07-24T04:38:22.1160830Z Entering 'third_party/NNPACK' 2025-07-24T04:38:22.1195673Z http.https://github.com/.extraheader 2025-07-24T04:38:22.1250727Z Entering 'third_party/NVTX' 2025-07-24T04:38:22.1288195Z http.https://github.com/.extraheader 2025-07-24T04:38:22.1343288Z Entering 'third_party/VulkanMemoryAllocator' 2025-07-24T04:38:22.1380542Z http.https://github.com/.extraheader 2025-07-24T04:38:22.1430909Z Entering 'third_party/XNNPACK' 2025-07-24T04:38:22.1466820Z http.https://github.com/.extraheader 2025-07-24T04:38:22.1533954Z Entering 'third_party/aiter' 2025-07-24T04:38:22.1568827Z http.https://github.com/.extraheader 2025-07-24T04:38:22.1619773Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-07-24T04:38:22.1658091Z http.https://github.com/.extraheader 2025-07-24T04:38:22.1727451Z Entering 'third_party/benchmark' 2025-07-24T04:38:22.1767119Z http.https://github.com/.extraheader 2025-07-24T04:38:22.1823037Z Entering 'third_party/composable_kernel' 2025-07-24T04:38:22.1861305Z http.https://github.com/.extraheader 2025-07-24T04:38:22.1923535Z Entering 'third_party/cpp-httplib' 2025-07-24T04:38:22.1967356Z http.https://github.com/.extraheader 2025-07-24T04:38:22.2021477Z Entering 'third_party/cpuinfo' 2025-07-24T04:38:22.2060175Z http.https://github.com/.extraheader 2025-07-24T04:38:22.2111515Z Entering 'third_party/cudnn_frontend' 2025-07-24T04:38:22.2146701Z http.https://github.com/.extraheader 2025-07-24T04:38:22.2202191Z Entering 'third_party/cutlass' 2025-07-24T04:38:22.2244638Z http.https://github.com/.extraheader 2025-07-24T04:38:22.2311525Z Entering 'third_party/fbgemm' 2025-07-24T04:38:22.2355299Z http.https://github.com/.extraheader 2025-07-24T04:38:22.2413996Z Entering 'third_party/fbgemm/external/asmjit' 2025-07-24T04:38:22.2451456Z http.https://github.com/.extraheader 2025-07-24T04:38:22.2503343Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-07-24T04:38:22.2544756Z http.https://github.com/.extraheader 2025-07-24T04:38:22.2610344Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-07-24T04:38:22.2647898Z http.https://github.com/.extraheader 2025-07-24T04:38:22.2696352Z Entering 'third_party/fbgemm/external/cutlass' 2025-07-24T04:38:22.2738562Z http.https://github.com/.extraheader 2025-07-24T04:38:22.2801112Z Entering 'third_party/fbgemm/external/googletest' 2025-07-24T04:38:22.2838795Z http.https://github.com/.extraheader 2025-07-24T04:38:22.2894476Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-07-24T04:38:22.2936786Z http.https://github.com/.extraheader 2025-07-24T04:38:22.2988712Z Entering 'third_party/fbgemm/external/json' 2025-07-24T04:38:22.3023568Z http.https://github.com/.extraheader 2025-07-24T04:38:22.3079311Z Entering 'third_party/flash-attention' 2025-07-24T04:38:22.3119428Z http.https://github.com/.extraheader 2025-07-24T04:38:22.3175688Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-07-24T04:38:22.3217527Z http.https://github.com/.extraheader 2025-07-24T04:38:22.3277846Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-07-24T04:38:22.3317959Z http.https://github.com/.extraheader 2025-07-24T04:38:22.3391026Z Entering 'third_party/flatbuffers' 2025-07-24T04:38:22.3429024Z http.https://github.com/.extraheader 2025-07-24T04:38:22.3487714Z Entering 'third_party/fmt' 2025-07-24T04:38:22.3529613Z http.https://github.com/.extraheader 2025-07-24T04:38:22.3584234Z Entering 'third_party/gemmlowp/gemmlowp' 2025-07-24T04:38:22.3622274Z http.https://github.com/.extraheader 2025-07-24T04:38:22.3674549Z Entering 'third_party/gloo' 2025-07-24T04:38:22.3710059Z http.https://github.com/.extraheader 2025-07-24T04:38:22.3765818Z Entering 'third_party/googletest' 2025-07-24T04:38:22.3808302Z http.https://github.com/.extraheader 2025-07-24T04:38:22.3859051Z Entering 'third_party/ideep' 2025-07-24T04:38:22.3900404Z http.https://github.com/.extraheader 2025-07-24T04:38:22.3952359Z Entering 'third_party/ideep/mkl-dnn' 2025-07-24T04:38:22.3999321Z http.https://github.com/.extraheader 2025-07-24T04:38:22.4071800Z Entering 'third_party/ittapi' 2025-07-24T04:38:22.4109798Z http.https://github.com/.extraheader 2025-07-24T04:38:22.4165605Z Entering 'third_party/kineto' 2025-07-24T04:38:22.4208147Z http.https://github.com/.extraheader 2025-07-24T04:38:22.4258879Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-07-24T04:38:22.4301260Z http.https://github.com/.extraheader 2025-07-24T04:38:22.4351258Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-07-24T04:38:22.4388818Z http.https://github.com/.extraheader 2025-07-24T04:38:22.4446232Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-07-24T04:38:22.4482722Z http.https://github.com/.extraheader 2025-07-24T04:38:22.4537424Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-07-24T04:38:22.4567848Z http.https://github.com/.extraheader 2025-07-24T04:38:22.4617364Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-07-24T04:38:22.4653041Z http.https://github.com/.extraheader 2025-07-24T04:38:22.4703640Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-07-24T04:38:22.4743063Z http.https://github.com/.extraheader 2025-07-24T04:38:22.4801117Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-07-24T04:38:22.4839821Z http.https://github.com/.extraheader 2025-07-24T04:38:22.4897049Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-07-24T04:38:22.4931618Z http.https://github.com/.extraheader 2025-07-24T04:38:22.4980772Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-07-24T04:38:22.5021799Z http.https://github.com/.extraheader 2025-07-24T04:38:22.5075823Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-07-24T04:38:22.5111620Z http.https://github.com/.extraheader 2025-07-24T04:38:22.5171980Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-07-24T04:38:22.5210296Z http.https://github.com/.extraheader 2025-07-24T04:38:22.5259475Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-07-24T04:38:22.5301744Z http.https://github.com/.extraheader 2025-07-24T04:38:22.5359568Z Entering 'third_party/kleidiai' 2025-07-24T04:38:22.5397958Z http.https://github.com/.extraheader 2025-07-24T04:38:22.5451076Z Entering 'third_party/mimalloc' 2025-07-24T04:38:22.5492215Z http.https://github.com/.extraheader 2025-07-24T04:38:22.5542039Z Entering 'third_party/nlohmann' 2025-07-24T04:38:22.5583718Z http.https://github.com/.extraheader 2025-07-24T04:38:22.5638947Z Entering 'third_party/onnx' 2025-07-24T04:38:22.5681126Z http.https://github.com/.extraheader 2025-07-24T04:38:22.5758030Z Entering 'third_party/onnx/third_party/pybind11' 2025-07-24T04:38:22.5793512Z http.https://github.com/.extraheader 2025-07-24T04:38:22.5856338Z Entering 'third_party/opentelemetry-cpp' 2025-07-24T04:38:22.5893892Z http.https://github.com/.extraheader 2025-07-24T04:38:22.5948506Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-07-24T04:38:22.5986344Z http.https://github.com/.extraheader 2025-07-24T04:38:22.6036699Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-07-24T04:38:22.6072976Z http.https://github.com/.extraheader 2025-07-24T04:38:22.6127223Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-07-24T04:38:22.6164542Z http.https://github.com/.extraheader 2025-07-24T04:38:22.6215379Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-07-24T04:38:22.6245524Z http.https://github.com/.extraheader 2025-07-24T04:38:22.6301666Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-07-24T04:38:22.6343859Z http.https://github.com/.extraheader 2025-07-24T04:38:22.6396123Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-07-24T04:38:22.6432831Z http.https://github.com/.extraheader 2025-07-24T04:38:22.6481438Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-07-24T04:38:22.6516515Z http.https://github.com/.extraheader 2025-07-24T04:38:22.6567313Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-07-24T04:38:22.6612980Z http.https://github.com/.extraheader 2025-07-24T04:38:22.6672484Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-07-24T04:38:22.6712436Z http.https://github.com/.extraheader 2025-07-24T04:38:22.6773760Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-07-24T04:38:22.6815768Z http.https://github.com/.extraheader 2025-07-24T04:38:22.6893302Z Entering 'third_party/pocketfft' 2025-07-24T04:38:22.6936227Z http.https://github.com/.extraheader 2025-07-24T04:38:22.6989224Z Entering 'third_party/protobuf' 2025-07-24T04:38:22.7022129Z http.https://github.com/.extraheader 2025-07-24T04:38:22.7082556Z Entering 'third_party/protobuf/third_party/benchmark' 2025-07-24T04:38:22.7128295Z http.https://github.com/.extraheader 2025-07-24T04:38:22.7185157Z Entering 'third_party/protobuf/third_party/googletest' 2025-07-24T04:38:22.7222852Z http.https://github.com/.extraheader 2025-07-24T04:38:22.7278596Z Entering 'third_party/psimd' 2025-07-24T04:38:22.7317392Z http.https://github.com/.extraheader 2025-07-24T04:38:22.7370451Z Entering 'third_party/pthreadpool' 2025-07-24T04:38:22.7408809Z http.https://github.com/.extraheader 2025-07-24T04:38:22.7463730Z Entering 'third_party/pybind11' 2025-07-24T04:38:22.7499643Z http.https://github.com/.extraheader 2025-07-24T04:38:22.7552477Z Entering 'third_party/python-peachpy' 2025-07-24T04:38:22.7587621Z http.https://github.com/.extraheader 2025-07-24T04:38:22.7641683Z Entering 'third_party/sleef' 2025-07-24T04:38:22.7684868Z http.https://github.com/.extraheader 2025-07-24T04:38:22.7736480Z Entering 'third_party/tensorpipe' 2025-07-24T04:38:22.7779178Z http.https://github.com/.extraheader 2025-07-24T04:38:22.7829882Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-07-24T04:38:22.7866036Z http.https://github.com/.extraheader 2025-07-24T04:38:22.7919700Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-07-24T04:38:22.7958198Z http.https://github.com/.extraheader 2025-07-24T04:38:22.8013376Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-07-24T04:38:22.8049195Z http.https://github.com/.extraheader 2025-07-24T04:38:22.8103280Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-07-24T04:38:22.8140693Z http.https://github.com/.extraheader 2025-07-24T04:38:22.8188526Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-07-24T04:38:22.8225169Z http.https://github.com/.extraheader 2025-07-24T04:38:22.8472164Z Cleaning up orphan processes