2025-03-14T04:36:36.0155512Z Current runner version: '2.322.0'
2025-03-14T04:36:36.0163450Z Runner name: 'i-0d8b0d116e8f11da9'
2025-03-14T04:36:36.0164455Z Runner group name: 'Default'
2025-03-14T04:36:36.0165511Z Machine name: 'ip-10-0-69-51'
2025-03-14T04:36:36.0170527Z ##[group]GITHUB_TOKEN Permissions
2025-03-14T04:36:36.0173505Z Actions: read
2025-03-14T04:36:36.0174246Z Attestations: read
2025-03-14T04:36:36.0175039Z Checks: read
2025-03-14T04:36:36.0175782Z Contents: read
2025-03-14T04:36:36.0176453Z Deployments: read
2025-03-14T04:36:36.0177221Z Discussions: read
2025-03-14T04:36:36.0177874Z Issues: read
2025-03-14T04:36:36.0178534Z Metadata: read
2025-03-14T04:36:36.0179275Z Packages: read
2025-03-14T04:36:36.0179968Z Pages: read
2025-03-14T04:36:36.0180673Z PullRequests: read
2025-03-14T04:36:36.0181436Z RepositoryProjects: read
2025-03-14T04:36:36.0182212Z SecurityEvents: read
2025-03-14T04:36:36.0182972Z Statuses: read
2025-03-14T04:36:36.0183728Z ##[endgroup]
2025-03-14T04:36:36.0187604Z Secret source: Actions
2025-03-14T04:36:36.0188631Z Prepare workflow directory
2025-03-14T04:36:36.4036001Z Prepare all required actions
2025-03-14T04:36:36.4080580Z Getting action download info
2025-03-14T04:36:36.5809527Z Download action repository 'pytorch/test-infra@main' (SHA:de00dac6adc071cb2f9861380a0ed3947b93e5cc)
2025-03-14T04:36:38.1148410Z Download action repository 'pytorch/pytorch@main' (SHA:bea181ff7eeead9fcdd806e286846296c4ab2d67)
2025-03-14T04:36:52.1340575Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0)
2025-03-14T04:36:52.3164923Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a)
2025-03-14T04:36:52.6402827Z Getting action download info
2025-03-14T04:36:52.7473575Z Download action repository 'actions/checkout@v4' (SHA:11bd71901bbe5b1630ceea73d27597364c9af683)
2025-03-14T04:36:53.0109168Z Getting action download info
2025-03-14T04:36:53.0997698Z Download action repository 'nick-fields/retry@v3.0.0' (SHA:7152eba30c6575329ac0576536151aca5a72780e)
2025-03-14T04:36:53.3239224Z Getting action download info
2025-03-14T04:36:53.4286539Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482)
2025-03-14T04:36:53.6887817Z Getting action download info
2025-03-14T04:36:53.8222899Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/heads/main (aed0b7a742a2d7b7901790622829cbd2135049a4)
2025-03-14T04:36:53.8224918Z ##[group] Inputs
2025-03-14T04:36:53.8225340Z   build-environment: linux-focal-cuda12.4-py3.10-gcc9-sm75
2025-03-14T04:36:53.8226100Z   test-matrix: {"include": [{"config": "pr_time_benchmarks", "shard": 1, "num_shards": 1, "runner": "linux.g4dn.metal.nvidia.gpu"}]}
2025-03-14T04:36:53.8227400Z   docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:36:53.8228408Z   sync-tag: 
2025-03-14T04:36:53.8229200Z   timeout-minutes: 240
2025-03-14T04:36:53.8229510Z   use-gha: 
2025-03-14T04:36:53.8229755Z   dashboard-tag: 
2025-03-14T04:36:53.8230017Z   s3-bucket: gha-artifacts
2025-03-14T04:36:53.8230336Z   aws-role-to-assume: 
2025-03-14T04:36:53.8230922Z   disable-monitor: false
2025-03-14T04:36:53.8231221Z ##[endgroup]
2025-03-14T04:36:53.8231758Z Complete job name: cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu)
2025-03-14T04:36:53.9129649Z A job started hook has been configured by the self-hosted runner administrator
2025-03-14T04:36:53.9230141Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh'
2025-03-14T04:36:53.9238488Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:36:53.9239226Z ##[endgroup]
2025-03-14T04:36:55.2698178Z Runner Type: linux.g4dn.metal.nvidia.gpu
2025-03-14T04:36:55.2698774Z Instance Type: g4dn.metal
2025-03-14T04:36:55.2699099Z AMI Name: unknown
2025-03-14T04:36:55.2735421Z AMI ID: ami-08b5b3a93ed654d19
2025-03-14T04:37:00.3532182Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main
2025-03-14T04:37:00.3532688Z with:
2025-03-14T04:37:00.3533396Z   github-secret: ***
2025-03-14T04:37:00.3534191Z   instructions: All testing is done inside the container, to start an interactive session run:
  docker exec -it $(docker container ps --format '{{.ID}}') bash

2025-03-14T04:37:00.3535044Z   activate-with-label: false
2025-03-14T04:37:00.3535365Z   label: with-ssh
2025-03-14T04:37:00.3535654Z   remove-existing-keys: true
2025-03-14T04:37:00.3535981Z   fail-silently: true
2025-03-14T04:37:00.3536263Z env:
2025-03-14T04:37:00.3536510Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:37:00.3536821Z ##[endgroup]
2025-03-14T04:37:00.4711933Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info.
2025-03-14T04:37:00.4713030Z Not on pull request and ciflow reference could not be extracted, skipping adding ssh keys
2025-03-14T04:37:00.4862099Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main
2025-03-14T04:37:00.4862662Z with:
2025-03-14T04:37:00.4862943Z   no-sudo: true
2025-03-14T04:37:00.4863248Z   submodules: recursive
2025-03-14T04:37:00.4863582Z   fetch-depth: 0
2025-03-14T04:37:00.4863852Z env:
2025-03-14T04:37:00.4864136Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:37:00.4864462Z ##[endgroup]
2025-03-14T04:37:00.4952593Z ##[group]Run echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT"
2025-03-14T04:37:00.4953693Z echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT"
2025-03-14T04:37:00.4962084Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:37:00.4962549Z env:
2025-03-14T04:37:00.4962870Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:37:00.4963270Z ##[endgroup]
2025-03-14T04:37:00.5059854Z ##[group]Run # Use all available CPUs for fetching
2025-03-14T04:37:00.5060381Z # Use all available CPUs for fetching
2025-03-14T04:37:00.5060776Z cd "${GITHUB_WORKSPACE}"
2025-03-14T04:37:00.5061156Z git config --global fetch.parallel 0
2025-03-14T04:37:00.5061594Z git config --global submodule.fetchJobs 0
2025-03-14T04:37:00.5061979Z 
2025-03-14T04:37:00.5062386Z # Clean workspace. The default checkout action should also do this, but
2025-03-14T04:37:00.5062916Z # do it here as well just in case
2025-03-14T04:37:00.5063263Z if [[ -d .git ]]; then
2025-03-14T04:37:00.5063597Z   if [ -z "${NO_SUDO}" ]; then
2025-03-14T04:37:00.5063952Z     sudo git clean -ffdx
2025-03-14T04:37:00.5064275Z   else
2025-03-14T04:37:00.5064548Z     git clean -ffdx
2025-03-14T04:37:00.5064852Z   fi
2025-03-14T04:37:00.5065100Z fi
2025-03-14T04:37:00.5071352Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:37:00.5071789Z env:
2025-03-14T04:37:00.5072137Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:37:00.5072462Z   NO_SUDO: true
2025-03-14T04:37:00.5072723Z ##[endgroup]
2025-03-14T04:37:00.6257530Z Removing .additional_ci_files/
2025-03-14T04:37:00.6259078Z Removing benchmarks/dynamo/pr_time_benchmarks/benchmarks/__pycache__/
2025-03-14T04:37:00.6260508Z Removing build/
2025-03-14T04:37:00.6261385Z Removing dist/
2025-03-14T04:37:00.6262279Z Removing logs-test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_38754319774.zip
2025-03-14T04:37:00.6263098Z Removing test-jsons-test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_38754319774.zip
2025-03-14T04:37:00.6263988Z Removing test-reports-test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_38754319774.zip
2025-03-14T04:37:00.6264647Z Removing test/test-reports/
2025-03-14T04:37:00.6265011Z Removing tools/__pycache__/
2025-03-14T04:37:00.6265376Z Removing tools/stats/__pycache__/
2025-03-14T04:37:00.6265864Z Removing tools/stats/upload_utilization_stats/__pycache__/
2025-03-14T04:37:00.6266609Z Removing usage_log.txt
2025-03-14T04:37:00.6337166Z ##[group]Run actions/checkout@v4
2025-03-14T04:37:00.6337555Z with:
2025-03-14T04:37:00.6337887Z   ref: aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:37:00.6338297Z   fetch-depth: 0
2025-03-14T04:37:00.6338613Z   submodules: recursive
2025-03-14T04:37:00.6338949Z   show-progress: false
2025-03-14T04:37:00.6339294Z   repository: pytorch/pytorch
2025-03-14T04:37:00.6339766Z   token: ***
2025-03-14T04:37:00.6340057Z   ssh-strict: true
2025-03-14T04:37:00.6340359Z   ssh-user: git
2025-03-14T04:37:00.6340665Z   persist-credentials: true
2025-03-14T04:37:00.6341016Z   clean: true
2025-03-14T04:37:00.6341335Z   sparse-checkout-cone-mode: true
2025-03-14T04:37:00.6341688Z   fetch-tags: false
2025-03-14T04:37:00.6341994Z   lfs: false
2025-03-14T04:37:00.6342300Z   set-safe-directory: true
2025-03-14T04:37:00.6342634Z env:
2025-03-14T04:37:00.6342914Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:37:00.6343432Z ##[endgroup]
2025-03-14T04:37:00.7448942Z Syncing repository: pytorch/pytorch
2025-03-14T04:37:00.7450471Z ##[group]Getting Git version info
2025-03-14T04:37:00.7451040Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch'
2025-03-14T04:37:00.7451855Z [command]/usr/bin/git version
2025-03-14T04:37:00.7452214Z git version 2.47.1
2025-03-14T04:37:00.7460352Z ##[endgroup]
2025-03-14T04:37:00.7469403Z Copying '/home/ec2-user/.gitconfig' to '/home/ec2-user/actions-runner/_work/_temp/603d915e-b02a-426a-a5d8-5f30bf852d7e/.gitconfig'
2025-03-14T04:37:00.7486881Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/603d915e-b02a-426a-a5d8-5f30bf852d7e' before making global git config changes
2025-03-14T04:37:00.7490049Z Adding repository directory to the temporary git global config as a safe directory
2025-03-14T04:37:00.7492483Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch
2025-03-14T04:37:00.7529855Z [command]/usr/bin/git config --local --get remote.origin.url
2025-03-14T04:37:00.7546753Z https://github.com/pytorch/pytorch
2025-03-14T04:37:00.7562616Z ##[group]Removing previously created refs, to avoid conflicts
2025-03-14T04:37:00.7564477Z [command]/usr/bin/git rev-parse --symbolic-full-name --verify --quiet HEAD
2025-03-14T04:37:00.7582563Z HEAD
2025-03-14T04:37:00.7616096Z ##[endgroup]
2025-03-14T04:37:00.7616984Z [command]/usr/bin/git submodule status
2025-03-14T04:37:00.7922343Z  7e1e1fe3858c63c251c637ae41a20de425dde96f android/libs/fbjni (v0.1.0-12-g7e1e1fe)
2025-03-14T04:37:00.7986129Z  4dfe081cf6bcd15db339cf2680b9281b8451eeb3 third_party/FP16 (4dfe081)
2025-03-14T04:37:00.8061807Z  b408327ac2a15ec3e43352421954f5b1967701d1 third_party/FXdiv (b408327)
2025-03-14T04:37:00.8150135Z  c07e3a0400713d546e0dea2d5466dd22ea389c73 third_party/NNPACK (c07e3a0)
2025-03-14T04:37:00.8176379Z  e170594ac7cf1dac584da473d4ca9301087090c1 third_party/NVTX (v3.1.0)
2025-03-14T04:37:00.8249538Z  a6bfc237255a6bac1513f7c1ebde6d8aed6b5191 third_party/VulkanMemoryAllocator (v2.1.0-705-ga6bfc23)
2025-03-14T04:37:00.8754779Z  51a0103656eff6fc9bfd39a4597923c4b542c883 third_party/XNNPACK (remotes/origin/ds/ndk-1243-g51a010365)
2025-03-14T04:37:00.8785564Z  0d98dba29d66e93259db7daa53a9327df767a415 third_party/benchmark (v1.6.1)
2025-03-14T04:37:00.8815478Z  8086bbe3a78d931eb96fe12fdc014082e18d18d3 third_party/composable_kernel (mock-tag-test-6-g8086bbe3a)
2025-03-14T04:37:00.8948260Z  3b6597bba913d51161383657829b7e644e59c006 third_party/cpp-httplib (v0.15.3-20-g3b6597b)
2025-03-14T04:37:00.9054202Z  1e83a2fdd3102f65c6f1fb602c1b320486218a99 third_party/cpuinfo (1e83a2f)
2025-03-14T04:37:00.9088651Z  91b7532f3386768bba4f444ee7672b497f34da8a third_party/cudnn_frontend (v0.5-44-g91b7532)
2025-03-14T04:37:00.9178844Z  afa1772203677c5118fcd82537a9c8fefbcc7008 third_party/cutlass (v3.8.0)
2025-03-14T04:37:00.9817463Z  3147391d946bb4b6c68edd901f2add6ac1f31f8c third_party/eigen (3.4.0)
2025-03-14T04:37:01.0171511Z  dbc3157bf256f1339b3fa1fef2be89ac4078be0e third_party/fbgemm (v0.4.1-446-gdbc3157b)
2025-03-14T04:37:01.0259641Z  979702c87a8713a8e0a5e9fee122b90d2ef13be5 third_party/flash-attention (v2.7.4)
2025-03-14T04:37:01.0285944Z  01834de25e4bf3975a9a00e816292b1ad0fe184b third_party/flatbuffers (v23.3.3)
2025-03-14T04:37:01.0699036Z  123913715afeb8a437e6388b4473fcc4753e1c9a third_party/fmt (11.1.4)
2025-03-14T04:37:01.0795985Z  3fb5c176c17c765a3492cd2f0321b0dab712f350 third_party/gemmlowp/gemmlowp (remotes/origin/revert-87-master-135-g3fb5c17)
2025-03-14T04:37:01.0907850Z  5354032ea08eadd7fc4456477f7f7c6308818509 third_party/gloo (5354032)
2025-03-14T04:37:01.1145994Z  b514bdc898e2951020cbdca1304b75f5950d1f59 third_party/googletest (release-1.8.0-3484-gb514bdc8)
2025-03-14T04:37:01.1235219Z  719d8e6cd7f7a0e01b155657526d693acf97c2b3 third_party/ideep (pytorch-rls-v3.7.1)
2025-03-14T04:37:01.1290448Z  5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42 third_party/ittapi (v3.23.0-14-g5b8a7d7)
2025-03-14T04:37:01.1541630Z  2859721fd9e73d3ca1c56f827dbc64e6d68f78a2 third_party/kineto (remotes/origin/sraikund/test-53-g2859721)
2025-03-14T04:37:01.1562617Z  ef685a13cfbe8d418aa2ed34350e21e4938358b6 third_party/kleidiai (v1.3.0)
2025-03-14T04:37:01.1590760Z  b66e3214d8a104669c2ec05ae91ebc26a8f5ab78 third_party/mimalloc (v1.8.2)
2025-03-14T04:37:01.2056128Z  87cda1d6646592ac5866dc703c8e1839046a6806 third_party/nlohmann (v3.10.1-113-g87cda1d6)
2025-03-14T04:37:01.2347885Z  b8baa8446686496da4cc8fda09f2b6fe65c2a02c third_party/onnx (v1.17.0)
2025-03-14T04:37:01.2373423Z  a799f4aed9c94b765dcdaabaeab7d5e7e2310878 third_party/opentelemetry-cpp (v1.14.2)
2025-03-14T04:37:01.2396328Z  9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe third_party/pocketfft (release_for_eigen-11-g9d3ab05)
2025-03-14T04:37:01.2810787Z  d1eca4e4b421cd2997495c4b4e65cea6be4e9b8a third_party/protobuf (v3.7.0-rc.2-1279-gd1eca4e4b)
2025-03-14T04:37:01.2878932Z  072586a71b55b7f8c584153d223e95687148a900 third_party/psimd (heads/master)
2025-03-14T04:37:01.2931020Z  4fe0e1e183925bf8cfa6aae24237e724a96479b8 third_party/pthreadpool (0.1-144-g4fe0e1e)
2025-03-14T04:37:01.2980380Z  a2e59f0e7065404b44dfe92a28aca47ba1378dc4 third_party/pybind11 (v2.11.0-182-ga2e59f0e)
2025-03-14T04:37:01.3043339Z  f45429b087dd7d5bc78bb40dc7cf06425c252d67 third_party/python-peachpy (remotes/origin/pre-generated)
2025-03-14T04:37:01.3151835Z  56e1f79cb140fb9326d612d0be06b5250565cade third_party/sleef (3.7-33-g56e1f79)
2025-03-14T04:37:01.3210659Z  52791a2fd214b2a9dc5759d36725909c1daa7f2e third_party/tensorpipe (remotes/origin/master)
2025-03-14T04:37:01.3270277Z ##[group]Cleaning the repository
2025-03-14T04:37:01.3271339Z [command]/usr/bin/git clean -ffdx
2025-03-14T04:37:01.3412309Z [command]/usr/bin/git reset --hard HEAD
2025-03-14T04:37:02.3927694Z Updating files:  86% (14255/16540)
2025-03-14T04:37:02.4068648Z Updating files:  87% (14390/16540)
2025-03-14T04:37:02.4219001Z Updating files:  88% (14556/16540)
2025-03-14T04:37:02.4404754Z Updating files:  89% (14721/16540)
2025-03-14T04:37:02.4516033Z Updating files:  90% (14886/16540)
2025-03-14T04:37:02.4646651Z Updating files:  91% (15052/16540)
2025-03-14T04:37:02.4800594Z Updating files:  92% (15217/16540)
2025-03-14T04:37:02.5048462Z Updating files:  93% (15383/16540)
2025-03-14T04:37:02.5233865Z Updating files:  94% (15548/16540)
2025-03-14T04:37:02.5380289Z Updating files:  95% (15713/16540)
2025-03-14T04:37:02.5511874Z Updating files:  96% (15879/16540)
2025-03-14T04:37:02.5779794Z Updating files:  97% (16044/16540)
2025-03-14T04:37:02.5926587Z Updating files:  98% (16210/16540)
2025-03-14T04:37:02.6070793Z Updating files:  99% (16375/16540)
2025-03-14T04:37:02.6071778Z Updating files: 100% (16540/16540)
2025-03-14T04:37:02.6072182Z Updating files: 100% (16540/16540), done.
2025-03-14T04:37:02.6243130Z HEAD is now at b4745db9048 [MPS] Add support for `i0e` in eager. (#149174)
2025-03-14T04:37:02.6317678Z ##[endgroup]
2025-03-14T04:37:02.6319049Z ##[group]Disabling automatic garbage collection
2025-03-14T04:37:02.6324077Z [command]/usr/bin/git config --local gc.auto 0
2025-03-14T04:37:02.6354604Z ##[endgroup]
2025-03-14T04:37:02.6356402Z ##[group]Setting up auth
2025-03-14T04:37:02.6360440Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2025-03-14T04:37:02.6388107Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
2025-03-14T04:37:02.6656519Z Entering 'android/libs/fbjni'
2025-03-14T04:37:02.6713260Z Entering 'third_party/FP16'
2025-03-14T04:37:02.6763916Z Entering 'third_party/FXdiv'
2025-03-14T04:37:02.6821035Z Entering 'third_party/NNPACK'
2025-03-14T04:37:02.6868876Z Entering 'third_party/NVTX'
2025-03-14T04:37:02.6923412Z Entering 'third_party/VulkanMemoryAllocator'
2025-03-14T04:37:02.6973584Z Entering 'third_party/XNNPACK'
2025-03-14T04:37:02.7037680Z Entering 'third_party/benchmark'
2025-03-14T04:37:02.7083726Z Entering 'third_party/composable_kernel'
2025-03-14T04:37:02.7148804Z Entering 'third_party/cpp-httplib'
2025-03-14T04:37:02.7203213Z Entering 'third_party/cpuinfo'
2025-03-14T04:37:02.7249914Z Entering 'third_party/cudnn_frontend'
2025-03-14T04:37:02.7303741Z Entering 'third_party/cutlass'
2025-03-14T04:37:02.7360049Z Entering 'third_party/eigen'
2025-03-14T04:37:02.7409269Z Entering 'third_party/fbgemm'
2025-03-14T04:37:02.7461276Z Entering 'third_party/fbgemm/third_party/asmjit'
2025-03-14T04:37:02.7506763Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2025-03-14T04:37:02.7560468Z Entering 'third_party/fbgemm/third_party/cutlass'
2025-03-14T04:37:02.7613999Z Entering 'third_party/fbgemm/third_party/googletest'
2025-03-14T04:37:02.7661835Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2025-03-14T04:37:02.7713679Z Entering 'third_party/flash-attention'
2025-03-14T04:37:02.7770797Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-03-14T04:37:02.7829074Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-03-14T04:37:02.7893890Z Entering 'third_party/flatbuffers'
2025-03-14T04:37:02.7954303Z Entering 'third_party/fmt'
2025-03-14T04:37:02.8009878Z Entering 'third_party/gemmlowp/gemmlowp'
2025-03-14T04:37:02.8062281Z Entering 'third_party/gloo'
2025-03-14T04:37:02.8110411Z Entering 'third_party/googletest'
2025-03-14T04:37:02.8162377Z Entering 'third_party/ideep'
2025-03-14T04:37:02.8208993Z Entering 'third_party/ideep/mkl-dnn'
2025-03-14T04:37:02.8268164Z Entering 'third_party/ittapi'
2025-03-14T04:37:02.8320328Z Entering 'third_party/kineto'
2025-03-14T04:37:02.8371669Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-03-14T04:37:02.8417657Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-03-14T04:37:02.8473933Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-03-14T04:37:02.8527956Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-03-14T04:37:02.8582055Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-03-14T04:37:02.8629780Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-03-14T04:37:02.8683848Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-03-14T04:37:02.8741491Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-03-14T04:37:02.8801281Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-03-14T04:37:02.8859785Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-03-14T04:37:02.8909151Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-03-14T04:37:02.8956421Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-03-14T04:37:02.9010506Z Entering 'third_party/kleidiai'
2025-03-14T04:37:02.9058022Z Entering 'third_party/mimalloc'
2025-03-14T04:37:02.9113539Z Entering 'third_party/nlohmann'
2025-03-14T04:37:02.9170531Z Entering 'third_party/onnx'
2025-03-14T04:37:02.9240186Z Entering 'third_party/onnx/third_party/pybind11'
2025-03-14T04:37:02.9292736Z Entering 'third_party/opentelemetry-cpp'
2025-03-14T04:37:02.9340405Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-03-14T04:37:02.9397384Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-03-14T04:37:02.9451458Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-03-14T04:37:02.9497572Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-03-14T04:37:02.9544147Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-03-14T04:37:02.9595805Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-03-14T04:37:02.9642645Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-03-14T04:37:02.9691188Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-03-14T04:37:02.9742110Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-03-14T04:37:02.9790562Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-03-14T04:37:02.9859344Z Entering 'third_party/pocketfft'
2025-03-14T04:37:02.9909600Z Entering 'third_party/protobuf'
2025-03-14T04:37:02.9963439Z Entering 'third_party/protobuf/third_party/benchmark'
2025-03-14T04:37:03.0021409Z Entering 'third_party/protobuf/third_party/googletest'
2025-03-14T04:37:03.0071701Z Entering 'third_party/psimd'
2025-03-14T04:37:03.0127357Z Entering 'third_party/pthreadpool'
2025-03-14T04:37:03.0180293Z Entering 'third_party/pybind11'
2025-03-14T04:37:03.0228729Z Entering 'third_party/python-peachpy'
2025-03-14T04:37:03.0280591Z Entering 'third_party/sleef'
2025-03-14T04:37:03.0329156Z Entering 'third_party/tensorpipe'
2025-03-14T04:37:03.0375546Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-03-14T04:37:03.0423253Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-03-14T04:37:03.0479565Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-03-14T04:37:03.0539298Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-03-14T04:37:03.0583239Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-03-14T04:37:03.0649836Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2025-03-14T04:37:03.0677706Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
2025-03-14T04:37:03.0939565Z Entering 'android/libs/fbjni'
2025-03-14T04:37:03.0990098Z Entering 'third_party/FP16'
2025-03-14T04:37:03.1041591Z Entering 'third_party/FXdiv'
2025-03-14T04:37:03.1091565Z Entering 'third_party/NNPACK'
2025-03-14T04:37:03.1144378Z Entering 'third_party/NVTX'
2025-03-14T04:37:03.1199936Z Entering 'third_party/VulkanMemoryAllocator'
2025-03-14T04:37:03.1251754Z Entering 'third_party/XNNPACK'
2025-03-14T04:37:03.1318125Z Entering 'third_party/benchmark'
2025-03-14T04:37:03.1367746Z Entering 'third_party/composable_kernel'
2025-03-14T04:37:03.1421834Z Entering 'third_party/cpp-httplib'
2025-03-14T04:37:03.1470349Z Entering 'third_party/cpuinfo'
2025-03-14T04:37:03.1522600Z Entering 'third_party/cudnn_frontend'
2025-03-14T04:37:03.1579678Z Entering 'third_party/cutlass'
2025-03-14T04:37:03.1639743Z Entering 'third_party/eigen'
2025-03-14T04:37:03.1691919Z Entering 'third_party/fbgemm'
2025-03-14T04:37:03.1745658Z Entering 'third_party/fbgemm/third_party/asmjit'
2025-03-14T04:37:03.1796409Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2025-03-14T04:37:03.1858332Z Entering 'third_party/fbgemm/third_party/cutlass'
2025-03-14T04:37:03.1914237Z Entering 'third_party/fbgemm/third_party/googletest'
2025-03-14T04:37:03.1969464Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2025-03-14T04:37:03.2031442Z Entering 'third_party/flash-attention'
2025-03-14T04:37:03.2080554Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-03-14T04:37:03.2132646Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-03-14T04:37:03.2197844Z Entering 'third_party/flatbuffers'
2025-03-14T04:37:03.2254004Z Entering 'third_party/fmt'
2025-03-14T04:37:03.2308614Z Entering 'third_party/gemmlowp/gemmlowp'
2025-03-14T04:37:03.2361080Z Entering 'third_party/gloo'
2025-03-14T04:37:03.2422278Z Entering 'third_party/googletest'
2025-03-14T04:37:03.2472582Z Entering 'third_party/ideep'
2025-03-14T04:37:03.2520604Z Entering 'third_party/ideep/mkl-dnn'
2025-03-14T04:37:03.2575361Z Entering 'third_party/ittapi'
2025-03-14T04:37:03.2632191Z Entering 'third_party/kineto'
2025-03-14T04:37:03.2680492Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-03-14T04:37:03.2726840Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-03-14T04:37:03.2777142Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-03-14T04:37:03.2831768Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-03-14T04:37:03.2887372Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-03-14T04:37:03.2938070Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-03-14T04:37:03.2992192Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-03-14T04:37:03.3041720Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-03-14T04:37:03.3090426Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-03-14T04:37:03.3138682Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-03-14T04:37:03.3191917Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-03-14T04:37:03.3242451Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-03-14T04:37:03.3302164Z Entering 'third_party/kleidiai'
2025-03-14T04:37:03.3353573Z Entering 'third_party/mimalloc'
2025-03-14T04:37:03.3401358Z Entering 'third_party/nlohmann'
2025-03-14T04:37:03.3459154Z Entering 'third_party/onnx'
2025-03-14T04:37:03.3521941Z Entering 'third_party/onnx/third_party/pybind11'
2025-03-14T04:37:03.3578437Z Entering 'third_party/opentelemetry-cpp'
2025-03-14T04:37:03.3632148Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-03-14T04:37:03.3688537Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-03-14T04:37:03.3740921Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-03-14T04:37:03.3790033Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-03-14T04:37:03.3840077Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-03-14T04:37:03.3887025Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-03-14T04:37:03.3936395Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-03-14T04:37:03.3984227Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-03-14T04:37:03.4039574Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-03-14T04:37:03.4090249Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-03-14T04:37:03.4165881Z Entering 'third_party/pocketfft'
2025-03-14T04:37:03.4221226Z Entering 'third_party/protobuf'
2025-03-14T04:37:03.4279207Z Entering 'third_party/protobuf/third_party/benchmark'
2025-03-14T04:37:03.4326520Z Entering 'third_party/protobuf/third_party/googletest'
2025-03-14T04:37:03.4378967Z Entering 'third_party/psimd'
2025-03-14T04:37:03.4433156Z Entering 'third_party/pthreadpool'
2025-03-14T04:37:03.4481010Z Entering 'third_party/pybind11'
2025-03-14T04:37:03.4542727Z Entering 'third_party/python-peachpy'
2025-03-14T04:37:03.4591794Z Entering 'third_party/sleef'
2025-03-14T04:37:03.4651068Z Entering 'third_party/tensorpipe'
2025-03-14T04:37:03.4701286Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-03-14T04:37:03.4758135Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-03-14T04:37:03.4807043Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-03-14T04:37:03.4860739Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-03-14T04:37:03.4911368Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-03-14T04:37:03.4994517Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic ***
2025-03-14T04:37:03.5042817Z ##[endgroup]
2025-03-14T04:37:03.5043461Z ##[group]Fetching the repository
2025-03-14T04:37:03.5049765Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --no-recurse-submodules origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/*
2025-03-14T04:37:03.7383626Z From https://github.com/pytorch/pytorch
2025-03-14T04:37:03.7384363Z  - [deleted]                 (none)                 -> ciflow/inductor/149162
2025-03-14T04:37:03.7681272Z  - [deleted]                 (none)                 -> ciflow/inductor/149167
2025-03-14T04:37:04.1542641Z    aed0b7a742a..bea181ff7ee  main                   -> origin/main
2025-03-14T04:37:04.1553768Z  t [tag update]              ciflow/inductor/149027 -> ciflow/inductor/149027
2025-03-14T04:37:04.1557446Z  * [new tag]                 ciflow/trunk/149064    -> ciflow/trunk/149064
2025-03-14T04:37:04.1990890Z [command]/usr/bin/git rev-parse --verify --quiet aed0b7a742a2d7b7901790622829cbd2135049a4^{object}
2025-03-14T04:37:04.2016968Z aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:37:04.2019638Z ##[endgroup]
2025-03-14T04:37:04.2020170Z ##[group]Determining the checkout info
2025-03-14T04:37:04.2020925Z ##[endgroup]
2025-03-14T04:37:04.2025665Z [command]/usr/bin/git sparse-checkout disable
2025-03-14T04:37:04.5122468Z [command]/usr/bin/git config --local --unset-all extensions.worktreeConfig
2025-03-14T04:37:04.5147697Z ##[group]Checking out the ref
2025-03-14T04:37:04.5149742Z [command]/usr/bin/git checkout --progress --force aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:37:04.6210825Z Previous HEAD position was b4745db9048 [MPS] Add support for `i0e` in eager. (#149174)
2025-03-14T04:37:04.6216834Z HEAD is now at aed0b7a742a [c10d] Add param recording for uniqueID broadcasting and allgather (#149166)
2025-03-14T04:37:04.6245791Z ##[endgroup]
2025-03-14T04:37:04.6246367Z ##[group]Setting up auth for fetching submodules
2025-03-14T04:37:04.6250671Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic ***
2025-03-14T04:37:04.6293143Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf
2025-03-14T04:37:04.6317966Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com:
2025-03-14T04:37:04.6346676Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com:
2025-03-14T04:37:04.6369596Z ##[endgroup]
2025-03-14T04:37:04.6370162Z ##[group]Fetching submodules
2025-03-14T04:37:04.6371682Z [command]/usr/bin/git submodule sync --recursive
2025-03-14T04:37:04.6638577Z Synchronizing submodule url for 'android/libs/fbjni'
2025-03-14T04:37:04.6663626Z Synchronizing submodule url for 'third_party/FP16'
2025-03-14T04:37:04.6690429Z Synchronizing submodule url for 'third_party/FXdiv'
2025-03-14T04:37:04.6723951Z Synchronizing submodule url for 'third_party/NNPACK'
2025-03-14T04:37:04.6749608Z Synchronizing submodule url for 'third_party/NVTX'
2025-03-14T04:37:04.6780078Z Synchronizing submodule url for 'third_party/VulkanMemoryAllocator'
2025-03-14T04:37:04.6801666Z Synchronizing submodule url for 'third_party/XNNPACK'
2025-03-14T04:37:04.6842008Z Synchronizing submodule url for 'third_party/benchmark'
2025-03-14T04:37:04.6863209Z Synchronizing submodule url for 'third_party/composable_kernel'
2025-03-14T04:37:04.6891935Z Synchronizing submodule url for 'third_party/cpp-httplib'
2025-03-14T04:37:04.6913734Z Synchronizing submodule url for 'third_party/cpuinfo'
2025-03-14T04:37:04.6941059Z Synchronizing submodule url for 'third_party/cudnn_frontend'
2025-03-14T04:37:04.6972250Z Synchronizing submodule url for 'third_party/cutlass'
2025-03-14T04:37:04.7007075Z Synchronizing submodule url for 'third_party/eigen'
2025-03-14T04:37:04.7033645Z Synchronizing submodule url for 'third_party/fbgemm'
2025-03-14T04:37:04.7054333Z Synchronizing submodule url for 'third_party/fbgemm/third_party/asmjit'
2025-03-14T04:37:04.7078596Z Synchronizing submodule url for 'third_party/fbgemm/third_party/cpuinfo'
2025-03-14T04:37:04.7107792Z Synchronizing submodule url for 'third_party/fbgemm/third_party/cutlass'
2025-03-14T04:37:04.7132659Z Synchronizing submodule url for 'third_party/fbgemm/third_party/googletest'
2025-03-14T04:37:04.7151616Z Synchronizing submodule url for 'third_party/fbgemm/third_party/hipify_torch'
2025-03-14T04:37:04.7176540Z Synchronizing submodule url for 'third_party/flash-attention'
2025-03-14T04:37:04.7195401Z Synchronizing submodule url for 'third_party/flash-attention/csrc/composable_kernel'
2025-03-14T04:37:04.7220166Z Synchronizing submodule url for 'third_party/flash-attention/csrc/cutlass'
2025-03-14T04:37:04.7257795Z Synchronizing submodule url for 'third_party/flatbuffers'
2025-03-14T04:37:04.7282300Z Synchronizing submodule url for 'third_party/fmt'
2025-03-14T04:37:04.7303677Z Synchronizing submodule url for 'third_party/gemmlowp/gemmlowp'
2025-03-14T04:37:04.7324602Z Synchronizing submodule url for 'third_party/gloo'
2025-03-14T04:37:04.7344163Z Synchronizing submodule url for 'third_party/googletest'
2025-03-14T04:37:04.7363882Z Synchronizing submodule url for 'third_party/ideep'
2025-03-14T04:37:04.7386081Z Synchronizing submodule url for 'third_party/ideep/mkl-dnn'
2025-03-14T04:37:04.7415161Z Synchronizing submodule url for 'third_party/ittapi'
2025-03-14T04:37:04.7437672Z Synchronizing submodule url for 'third_party/kineto'
2025-03-14T04:37:04.7459153Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog'
2025-03-14T04:37:04.7476003Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-03-14T04:37:04.7493492Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-03-14T04:37:04.7512117Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-03-14T04:37:04.7539341Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-03-14T04:37:04.7561141Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-03-14T04:37:04.7587699Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-03-14T04:37:04.7607030Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-03-14T04:37:04.7624920Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-03-14T04:37:04.7643935Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-03-14T04:37:04.7667466Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/fmt'
2025-03-14T04:37:04.7694867Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/googletest'
2025-03-14T04:37:04.7731747Z Synchronizing submodule url for 'third_party/kleidiai'
2025-03-14T04:37:04.7753309Z Synchronizing submodule url for 'third_party/mimalloc'
2025-03-14T04:37:04.7772126Z Synchronizing submodule url for 'third_party/nlohmann'
2025-03-14T04:37:04.7791832Z Synchronizing submodule url for 'third_party/onnx'
2025-03-14T04:37:04.7825860Z Synchronizing submodule url for 'third_party/onnx/third_party/pybind11'
2025-03-14T04:37:04.7853368Z Synchronizing submodule url for 'third_party/opentelemetry-cpp'
2025-03-14T04:37:04.7877841Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-03-14T04:37:04.7896407Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/googletest'
2025-03-14T04:37:04.7914883Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-03-14T04:37:04.7938519Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-03-14T04:37:04.7965863Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-03-14T04:37:04.7994648Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-03-14T04:37:04.8013759Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-03-14T04:37:04.8033338Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-03-14T04:37:04.8051924Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-03-14T04:37:04.8075570Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-03-14T04:37:04.8115690Z Synchronizing submodule url for 'third_party/pocketfft'
2025-03-14T04:37:04.8142428Z Synchronizing submodule url for 'third_party/protobuf'
2025-03-14T04:37:04.8165646Z Synchronizing submodule url for 'third_party/protobuf/third_party/benchmark'
2025-03-14T04:37:04.8189467Z Synchronizing submodule url for 'third_party/protobuf/third_party/googletest'
2025-03-14T04:37:04.8212712Z Synchronizing submodule url for 'third_party/psimd'
2025-03-14T04:37:04.8234280Z Synchronizing submodule url for 'third_party/pthreadpool'
2025-03-14T04:37:04.8260793Z Synchronizing submodule url for 'third_party/pybind11'
2025-03-14T04:37:04.8281806Z Synchronizing submodule url for 'third_party/python-peachpy'
2025-03-14T04:37:04.8310151Z Synchronizing submodule url for 'third_party/sleef'
2025-03-14T04:37:04.8341786Z Synchronizing submodule url for 'third_party/tensorpipe'
2025-03-14T04:37:04.8366644Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/googletest'
2025-03-14T04:37:04.8381555Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/libnop'
2025-03-14T04:37:04.8408848Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/libuv'
2025-03-14T04:37:04.8427425Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/pybind11'
2025-03-14T04:37:04.8446586Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-03-14T04:37:04.8491582Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive
2025-03-14T04:37:04.8925246Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f'
2025-03-14T04:37:04.9048440Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3'
2025-03-14T04:37:04.9131237Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1'
2025-03-14T04:37:04.9391441Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73'
2025-03-14T04:37:04.9776043Z Submodule path 'third_party/NVTX': checked out 'e170594ac7cf1dac584da473d4ca9301087090c1'
2025-03-14T04:37:05.0173907Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191'
2025-03-14T04:37:05.8693374Z Submodule path 'third_party/XNNPACK': checked out '51a0103656eff6fc9bfd39a4597923c4b542c883'
2025-03-14T04:37:05.8945426Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415'
2025-03-14T04:37:06.1847944Z Submodule path 'third_party/composable_kernel': checked out '8086bbe3a78d931eb96fe12fdc014082e18d18d3'
2025-03-14T04:37:06.2292091Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006'
2025-03-14T04:37:06.3333428Z Submodule path 'third_party/cpuinfo': checked out '1e83a2fdd3102f65c6f1fb602c1b320486218a99'
2025-03-14T04:37:06.3688297Z Submodule path 'third_party/cudnn_frontend': checked out '91b7532f3386768bba4f444ee7672b497f34da8a'
2025-03-14T04:37:07.0337148Z Submodule path 'third_party/cutlass': checked out 'afa1772203677c5118fcd82537a9c8fefbcc7008'
2025-03-14T04:37:07.3107770Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c'
2025-03-14T04:37:07.3801263Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e'
2025-03-14T04:37:07.4253478Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81'
2025-03-14T04:37:07.5239756Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3'
2025-03-14T04:37:07.9917347Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6'
2025-03-14T04:37:08.0393441Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796'
2025-03-14T04:37:08.0504839Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1'
2025-03-14T04:37:08.1297264Z Submodule path 'third_party/flash-attention': checked out '979702c87a8713a8e0a5e9fee122b90d2ef13be5'
2025-03-14T04:37:08.4030363Z Submodule path 'third_party/flash-attention/csrc/composable_kernel': checked out '888317e698e9803c62bd38568abc9e05d7709f33'
2025-03-14T04:37:09.0448123Z Submodule path 'third_party/flash-attention/csrc/cutlass': checked out 'c506e16788cb08416a4a57e11a9067beeee29420'
2025-03-14T04:37:09.1963419Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b'
2025-03-14T04:37:09.2291613Z Submodule path 'third_party/fmt': checked out '123913715afeb8a437e6388b4473fcc4753e1c9a'
2025-03-14T04:37:09.2694477Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350'
2025-03-14T04:37:09.2971871Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509'
2025-03-14T04:37:09.3433626Z Submodule path 'third_party/googletest': checked out 'b514bdc898e2951020cbdca1304b75f5950d1f59'
2025-03-14T04:37:09.3568632Z Submodule path 'third_party/ideep': checked out '719d8e6cd7f7a0e01b155657526d693acf97c2b3'
2025-03-14T04:37:09.9631944Z Submodule path 'third_party/ideep/mkl-dnn': checked out '8d263e693366ef8db40acc569cc7d8edf644556d'
2025-03-14T04:37:09.9791969Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42'
2025-03-14T04:37:10.0704628Z Submodule path 'third_party/kineto': checked out '2859721fd9e73d3ca1c56f827dbc64e6d68f78a2'
2025-03-14T04:37:10.1594814Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e'
2025-03-14T04:37:10.3549839Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9'
2025-03-14T04:37:10.3745107Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400'
2025-03-14T04:37:10.4120188Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05'
2025-03-14T04:37:10.4254749Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067'
2025-03-14T04:37:10.4342013Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4'
2025-03-14T04:37:10.4535902Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446'
2025-03-14T04:37:10.4967375Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850'
2025-03-14T04:37:10.6066195Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5'
2025-03-14T04:37:10.6248665Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150'
2025-03-14T04:37:10.6646869Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164'
2025-03-14T04:37:10.7122369Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347'
2025-03-14T04:37:10.7518948Z Submodule path 'third_party/kleidiai': checked out 'ef685a13cfbe8d418aa2ed34350e21e4938358b6'
2025-03-14T04:37:10.7909593Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78'
2025-03-14T04:37:10.9011942Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806'
2025-03-14T04:37:11.3129716Z Submodule path 'third_party/onnx': checked out 'b8baa8446686496da4cc8fda09f2b6fe65c2a02c'
2025-03-14T04:37:11.3519751Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '3e9dfa2866941655c56877882565e7577de6fc7b'
2025-03-14T04:37:11.4348206Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878'
2025-03-14T04:37:11.4572605Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2'
2025-03-14T04:37:11.5020420Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1'
2025-03-14T04:37:11.5156384Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa'
2025-03-14T04:37:11.6311139Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d'
2025-03-14T04:37:11.6448915Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce'
2025-03-14T04:37:11.6605745Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5'
2025-03-14T04:37:11.6781938Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d'
2025-03-14T04:37:11.9436989Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4'
2025-03-14T04:37:11.9916287Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929'
2025-03-14T04:37:12.5548081Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50'
2025-03-14T04:37:12.5666094Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe'
2025-03-14T04:37:12.8724418Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a'
2025-03-14T04:37:12.8885827Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8'
2025-03-14T04:37:12.9420062Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081'
2025-03-14T04:37:12.9510989Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900'
2025-03-14T04:37:12.9640232Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8'
2025-03-14T04:37:13.0029086Z Submodule path 'third_party/pybind11': checked out 'a2e59f0e7065404b44dfe92a28aca47ba1378dc4'
2025-03-14T04:37:13.0316692Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67'
2025-03-14T04:37:13.0759884Z Submodule path 'third_party/sleef': checked out '56e1f79cb140fb9326d612d0be06b5250565cade'
2025-03-14T04:37:13.1049230Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e'
2025-03-14T04:37:13.1506228Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e'
2025-03-14T04:37:13.1669037Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281'
2025-03-14T04:37:13.2326245Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242'
2025-03-14T04:37:13.2626136Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef'
2025-03-14T04:37:13.2713574Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5'
2025-03-14T04:37:13.2763581Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0
2025-03-14T04:37:13.3050575Z Entering 'android/libs/fbjni'
2025-03-14T04:37:13.3095505Z Entering 'third_party/FP16'
2025-03-14T04:37:13.3133367Z Entering 'third_party/FXdiv'
2025-03-14T04:37:13.3177934Z Entering 'third_party/NNPACK'
2025-03-14T04:37:13.3219340Z Entering 'third_party/NVTX'
2025-03-14T04:37:13.3266726Z Entering 'third_party/VulkanMemoryAllocator'
2025-03-14T04:37:13.3303777Z Entering 'third_party/XNNPACK'
2025-03-14T04:37:13.3356048Z Entering 'third_party/benchmark'
2025-03-14T04:37:13.3391286Z Entering 'third_party/composable_kernel'
2025-03-14T04:37:13.3445548Z Entering 'third_party/cpp-httplib'
2025-03-14T04:37:13.3481994Z Entering 'third_party/cpuinfo'
2025-03-14T04:37:13.3526461Z Entering 'third_party/cudnn_frontend'
2025-03-14T04:37:13.3567897Z Entering 'third_party/cutlass'
2025-03-14T04:37:13.3624729Z Entering 'third_party/eigen'
2025-03-14T04:37:13.3668754Z Entering 'third_party/fbgemm'
2025-03-14T04:37:13.3705941Z Entering 'third_party/fbgemm/third_party/asmjit'
2025-03-14T04:37:13.3741408Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2025-03-14T04:37:13.3777596Z Entering 'third_party/fbgemm/third_party/cutlass'
2025-03-14T04:37:13.3833078Z Entering 'third_party/fbgemm/third_party/googletest'
2025-03-14T04:37:13.3876004Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2025-03-14T04:37:13.3915860Z Entering 'third_party/flash-attention'
2025-03-14T04:37:13.3963486Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-03-14T04:37:13.4009321Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-03-14T04:37:13.4065303Z Entering 'third_party/flatbuffers'
2025-03-14T04:37:13.4110691Z Entering 'third_party/fmt'
2025-03-14T04:37:13.4150725Z Entering 'third_party/gemmlowp/gemmlowp'
2025-03-14T04:37:13.4186948Z Entering 'third_party/gloo'
2025-03-14T04:37:13.4228896Z Entering 'third_party/googletest'
2025-03-14T04:37:13.4274260Z Entering 'third_party/ideep'
2025-03-14T04:37:13.4314262Z Entering 'third_party/ideep/mkl-dnn'
2025-03-14T04:37:13.4360595Z Entering 'third_party/ittapi'
2025-03-14T04:37:13.4407171Z Entering 'third_party/kineto'
2025-03-14T04:37:13.4444038Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-03-14T04:37:13.4484377Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-03-14T04:37:13.4521891Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-03-14T04:37:13.4557604Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-03-14T04:37:13.4600529Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-03-14T04:37:13.4636236Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-03-14T04:37:13.4688822Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-03-14T04:37:13.4724977Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-03-14T04:37:13.4762095Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-03-14T04:37:13.4796446Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-03-14T04:37:13.4838796Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-03-14T04:37:13.4874826Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-03-14T04:37:13.4915331Z Entering 'third_party/kleidiai'
2025-03-14T04:37:13.4965396Z Entering 'third_party/mimalloc'
2025-03-14T04:37:13.5002580Z Entering 'third_party/nlohmann'
2025-03-14T04:37:13.5041192Z Entering 'third_party/onnx'
2025-03-14T04:37:13.5101904Z Entering 'third_party/onnx/third_party/pybind11'
2025-03-14T04:37:13.5147702Z Entering 'third_party/opentelemetry-cpp'
2025-03-14T04:37:13.5191252Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-03-14T04:37:13.5235276Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-03-14T04:37:13.5273785Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-03-14T04:37:13.5309699Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-03-14T04:37:13.5347800Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-03-14T04:37:13.5394434Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-03-14T04:37:13.5429601Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-03-14T04:37:13.5473245Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-03-14T04:37:13.5511002Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-03-14T04:37:13.5547538Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-03-14T04:37:13.5601146Z Entering 'third_party/pocketfft'
2025-03-14T04:37:13.5646048Z Entering 'third_party/protobuf'
2025-03-14T04:37:13.5685156Z Entering 'third_party/protobuf/third_party/benchmark'
2025-03-14T04:37:13.5726788Z Entering 'third_party/protobuf/third_party/googletest'
2025-03-14T04:37:13.5771741Z Entering 'third_party/psimd'
2025-03-14T04:37:13.5815128Z Entering 'third_party/pthreadpool'
2025-03-14T04:37:13.5855367Z Entering 'third_party/pybind11'
2025-03-14T04:37:13.5891678Z Entering 'third_party/python-peachpy'
2025-03-14T04:37:13.5936928Z Entering 'third_party/sleef'
2025-03-14T04:37:13.5973109Z Entering 'third_party/tensorpipe'
2025-03-14T04:37:13.6022741Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-03-14T04:37:13.6063817Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-03-14T04:37:13.6097626Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-03-14T04:37:13.6139617Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-03-14T04:37:13.6184110Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-03-14T04:37:13.6240488Z ##[endgroup]
2025-03-14T04:37:13.6241033Z ##[group]Persisting credentials for submodules
2025-03-14T04:37:13.6247471Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :"
2025-03-14T04:37:13.6530532Z Entering 'android/libs/fbjni'
2025-03-14T04:37:13.6566978Z url.https://github.com/.insteadof
2025-03-14T04:37:13.6567476Z url.https://github.com/.insteadof
2025-03-14T04:37:13.6600588Z Entering 'third_party/FP16'
2025-03-14T04:37:13.6631805Z url.https://github.com/.insteadof
2025-03-14T04:37:13.6632921Z url.https://github.com/.insteadof
2025-03-14T04:37:13.6662979Z Entering 'third_party/FXdiv'
2025-03-14T04:37:13.6692838Z url.https://github.com/.insteadof
2025-03-14T04:37:13.6693963Z url.https://github.com/.insteadof
2025-03-14T04:37:13.6727260Z Entering 'third_party/NNPACK'
2025-03-14T04:37:13.6758495Z url.https://github.com/.insteadof
2025-03-14T04:37:13.6759596Z url.https://github.com/.insteadof
2025-03-14T04:37:13.6795087Z Entering 'third_party/NVTX'
2025-03-14T04:37:13.6827079Z url.https://github.com/.insteadof
2025-03-14T04:37:13.6828192Z url.https://github.com/.insteadof
2025-03-14T04:37:13.6860879Z Entering 'third_party/VulkanMemoryAllocator'
2025-03-14T04:37:13.6892960Z url.https://github.com/.insteadof
2025-03-14T04:37:13.6894061Z url.https://github.com/.insteadof
2025-03-14T04:37:13.6922516Z Entering 'third_party/XNNPACK'
2025-03-14T04:37:13.6958035Z url.https://github.com/.insteadof
2025-03-14T04:37:13.6959162Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7010294Z Entering 'third_party/benchmark'
2025-03-14T04:37:13.7045334Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7046455Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7078882Z Entering 'third_party/composable_kernel'
2025-03-14T04:37:13.7113884Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7114980Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7157130Z Entering 'third_party/cpp-httplib'
2025-03-14T04:37:13.7189692Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7191019Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7227585Z Entering 'third_party/cpuinfo'
2025-03-14T04:37:13.7259287Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7260436Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7295761Z Entering 'third_party/cudnn_frontend'
2025-03-14T04:37:13.7328210Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7328781Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7368547Z Entering 'third_party/cutlass'
2025-03-14T04:37:13.7406063Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7407609Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7446692Z Entering 'third_party/eigen'
2025-03-14T04:37:13.7480248Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7481379Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7515996Z Entering 'third_party/fbgemm'
2025-03-14T04:37:13.7549831Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7550971Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7586442Z Entering 'third_party/fbgemm/third_party/asmjit'
2025-03-14T04:37:13.7620282Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7620749Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7651204Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2025-03-14T04:37:13.7686926Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7688063Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7717013Z Entering 'third_party/fbgemm/third_party/cutlass'
2025-03-14T04:37:13.7753947Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7755081Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7792939Z Entering 'third_party/fbgemm/third_party/googletest'
2025-03-14T04:37:13.7823826Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7824396Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7859724Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2025-03-14T04:37:13.7895858Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7896995Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7929314Z Entering 'third_party/flash-attention'
2025-03-14T04:37:13.7967096Z url.https://github.com/.insteadof
2025-03-14T04:37:13.7968230Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8005280Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-03-14T04:37:13.8034894Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8035796Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8078307Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-03-14T04:37:13.8112024Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8113169Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8157040Z Entering 'third_party/flatbuffers'
2025-03-14T04:37:13.8189911Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8191062Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8227250Z Entering 'third_party/fmt'
2025-03-14T04:37:13.8264909Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8266055Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8300598Z Entering 'third_party/gemmlowp/gemmlowp'
2025-03-14T04:37:13.8331438Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8331978Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8368624Z Entering 'third_party/gloo'
2025-03-14T04:37:13.8402032Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8403190Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8436360Z Entering 'third_party/googletest'
2025-03-14T04:37:13.8469604Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8470745Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8500264Z Entering 'third_party/ideep'
2025-03-14T04:37:13.8536923Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8538062Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8569112Z Entering 'third_party/ideep/mkl-dnn'
2025-03-14T04:37:13.8600127Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8601030Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8645956Z Entering 'third_party/ittapi'
2025-03-14T04:37:13.8677437Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8677967Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8715591Z Entering 'third_party/kineto'
2025-03-14T04:37:13.8748110Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8749239Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8784304Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-03-14T04:37:13.8815326Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8815918Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8849321Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-03-14T04:37:13.8887156Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8888313Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8921968Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-03-14T04:37:13.8952772Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8953278Z url.https://github.com/.insteadof
2025-03-14T04:37:13.8990544Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-03-14T04:37:13.9021998Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9022545Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9055837Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-03-14T04:37:13.9089257Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9089995Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9121147Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-03-14T04:37:13.9154364Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9155506Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9191156Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-03-14T04:37:13.9222261Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9222732Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9257701Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-03-14T04:37:13.9290456Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9291590Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9326384Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-03-14T04:37:13.9357866Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9358425Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9395224Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-03-14T04:37:13.9430755Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9431999Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9467871Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-03-14T04:37:13.9499387Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9499986Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9537423Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-03-14T04:37:13.9574723Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9575865Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9609626Z Entering 'third_party/kleidiai'
2025-03-14T04:37:13.9645850Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9647001Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9679240Z Entering 'third_party/mimalloc'
2025-03-14T04:37:13.9715924Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9717062Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9746951Z Entering 'third_party/nlohmann'
2025-03-14T04:37:13.9779644Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9780790Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9811921Z Entering 'third_party/onnx'
2025-03-14T04:37:13.9846900Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9848075Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9892542Z Entering 'third_party/onnx/third_party/pybind11'
2025-03-14T04:37:13.9927184Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9928317Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9963028Z Entering 'third_party/opentelemetry-cpp'
2025-03-14T04:37:13.9996418Z url.https://github.com/.insteadof
2025-03-14T04:37:13.9997550Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0030773Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-03-14T04:37:14.0060864Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0061759Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0097218Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-03-14T04:37:14.0129446Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0129987Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0169311Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-03-14T04:37:14.0200063Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0200564Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0239026Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-03-14T04:37:14.0269975Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0270521Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0307956Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-03-14T04:37:14.0345942Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0347073Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0377821Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-03-14T04:37:14.0409007Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0409600Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0448132Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-03-14T04:37:14.0484727Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0485862Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0517185Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-03-14T04:37:14.0554612Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0555735Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0589780Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-03-14T04:37:14.0620196Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0620790Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0657302Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-03-14T04:37:14.0695831Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0696978Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0752581Z Entering 'third_party/pocketfft'
2025-03-14T04:37:14.0785925Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0787060Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0818783Z Entering 'third_party/protobuf'
2025-03-14T04:37:14.0850775Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0851972Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0884743Z Entering 'third_party/protobuf/third_party/benchmark'
2025-03-14T04:37:14.0916044Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0916908Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0952402Z Entering 'third_party/protobuf/third_party/googletest'
2025-03-14T04:37:14.0986811Z url.https://github.com/.insteadof
2025-03-14T04:37:14.0987920Z url.https://github.com/.insteadof
2025-03-14T04:37:14.1018858Z Entering 'third_party/psimd'
2025-03-14T04:37:14.1052048Z url.https://github.com/.insteadof
2025-03-14T04:37:14.1052574Z url.https://github.com/.insteadof
2025-03-14T04:37:14.1084957Z Entering 'third_party/pthreadpool'
2025-03-14T04:37:14.1117258Z url.https://github.com/.insteadof
2025-03-14T04:37:14.1117843Z url.https://github.com/.insteadof
2025-03-14T04:37:14.1147986Z Entering 'third_party/pybind11'
2025-03-14T04:37:14.1182071Z url.https://github.com/.insteadof
2025-03-14T04:37:14.1183198Z url.https://github.com/.insteadof
2025-03-14T04:37:14.1217152Z Entering 'third_party/python-peachpy'
2025-03-14T04:37:14.1254414Z url.https://github.com/.insteadof
2025-03-14T04:37:14.1255085Z url.https://github.com/.insteadof
2025-03-14T04:37:14.1287644Z Entering 'third_party/sleef'
2025-03-14T04:37:14.1324772Z url.https://github.com/.insteadof
2025-03-14T04:37:14.1325254Z url.https://github.com/.insteadof
2025-03-14T04:37:14.1364022Z Entering 'third_party/tensorpipe'
2025-03-14T04:37:14.1397407Z url.https://github.com/.insteadof
2025-03-14T04:37:14.1398503Z url.https://github.com/.insteadof
2025-03-14T04:37:14.1430942Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-03-14T04:37:14.1461273Z url.https://github.com/.insteadof
2025-03-14T04:37:14.1479340Z url.https://github.com/.insteadof
2025-03-14T04:37:14.1491816Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-03-14T04:37:14.1527501Z url.https://github.com/.insteadof
2025-03-14T04:37:14.1528645Z url.https://github.com/.insteadof
2025-03-14T04:37:14.1563732Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-03-14T04:37:14.1593016Z url.https://github.com/.insteadof
2025-03-14T04:37:14.1593698Z url.https://github.com/.insteadof
2025-03-14T04:37:14.1626472Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-03-14T04:37:14.1656445Z url.https://github.com/.insteadof
2025-03-14T04:37:14.1657365Z url.https://github.com/.insteadof
2025-03-14T04:37:14.1693715Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-03-14T04:37:14.1730298Z url.https://github.com/.insteadof
2025-03-14T04:37:14.1731119Z url.https://github.com/.insteadof
2025-03-14T04:37:14.1788817Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url"
2025-03-14T04:37:14.2062631Z Entering 'android/libs/fbjni'
2025-03-14T04:37:14.2106695Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config	remote.origin.url
2025-03-14T04:37:14.2123516Z Entering 'third_party/FP16'
2025-03-14T04:37:14.2170592Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config	remote.origin.url
2025-03-14T04:37:14.2188959Z Entering 'third_party/FXdiv'
2025-03-14T04:37:14.2237011Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config	remote.origin.url
2025-03-14T04:37:14.2253619Z Entering 'third_party/NNPACK'
2025-03-14T04:37:14.2301697Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config	remote.origin.url
2025-03-14T04:37:14.2318880Z Entering 'third_party/NVTX'
2025-03-14T04:37:14.2365866Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NVTX/config	remote.origin.url
2025-03-14T04:37:14.2382995Z Entering 'third_party/VulkanMemoryAllocator'
2025-03-14T04:37:14.2434170Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config	remote.origin.url
2025-03-14T04:37:14.2452813Z Entering 'third_party/XNNPACK'
2025-03-14T04:37:14.2503802Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config	remote.origin.url
2025-03-14T04:37:14.2537817Z Entering 'third_party/benchmark'
2025-03-14T04:37:14.2584708Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config	remote.origin.url
2025-03-14T04:37:14.2603222Z Entering 'third_party/composable_kernel'
2025-03-14T04:37:14.2652527Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/composable_kernel/config	remote.origin.url
2025-03-14T04:37:14.2677450Z Entering 'third_party/cpp-httplib'
2025-03-14T04:37:14.2726053Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config	remote.origin.url
2025-03-14T04:37:14.2743369Z Entering 'third_party/cpuinfo'
2025-03-14T04:37:14.2793287Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config	remote.origin.url
2025-03-14T04:37:14.2812061Z Entering 'third_party/cudnn_frontend'
2025-03-14T04:37:14.2857369Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config	remote.origin.url
2025-03-14T04:37:14.2879427Z Entering 'third_party/cutlass'
2025-03-14T04:37:14.2926819Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config	remote.origin.url
2025-03-14T04:37:14.2953216Z Entering 'third_party/eigen'
2025-03-14T04:37:14.3001968Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config	remote.origin.url
2025-03-14T04:37:14.3022979Z Entering 'third_party/fbgemm'
2025-03-14T04:37:14.3070022Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config	remote.origin.url
2025-03-14T04:37:14.3089414Z Entering 'third_party/fbgemm/third_party/asmjit'
2025-03-14T04:37:14.3132150Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config	remote.origin.url
2025-03-14T04:37:14.3155582Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2025-03-14T04:37:14.3203581Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config	remote.origin.url
2025-03-14T04:37:14.3221728Z Entering 'third_party/fbgemm/third_party/cutlass'
2025-03-14T04:37:14.3264225Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config	remote.origin.url
2025-03-14T04:37:14.3292159Z Entering 'third_party/fbgemm/third_party/googletest'
2025-03-14T04:37:14.3338451Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config	remote.origin.url
2025-03-14T04:37:14.3357346Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2025-03-14T04:37:14.3402899Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config	remote.origin.url
2025-03-14T04:37:14.3422765Z Entering 'third_party/flash-attention'
2025-03-14T04:37:14.3472618Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/config	remote.origin.url
2025-03-14T04:37:14.3491973Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-03-14T04:37:14.3543458Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/modules/csrc/composable_kernel/config	remote.origin.url
2025-03-14T04:37:14.3568088Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-03-14T04:37:14.3616789Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/modules/csrc/cutlass/config	remote.origin.url
2025-03-14T04:37:14.3643948Z Entering 'third_party/flatbuffers'
2025-03-14T04:37:14.3686502Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config	remote.origin.url
2025-03-14T04:37:14.3707790Z Entering 'third_party/fmt'
2025-03-14T04:37:14.3749208Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config	remote.origin.url
2025-03-14T04:37:14.3765015Z Entering 'third_party/gemmlowp/gemmlowp'
2025-03-14T04:37:14.3825615Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config	remote.origin.url
2025-03-14T04:37:14.3843095Z Entering 'third_party/gloo'
2025-03-14T04:37:14.3892890Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config	remote.origin.url
2025-03-14T04:37:14.3911422Z Entering 'third_party/googletest'
2025-03-14T04:37:14.3962838Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config	remote.origin.url
2025-03-14T04:37:14.3980667Z Entering 'third_party/ideep'
2025-03-14T04:37:14.4032141Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config	remote.origin.url
2025-03-14T04:37:14.4050155Z Entering 'third_party/ideep/mkl-dnn'
2025-03-14T04:37:14.4093145Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config	remote.origin.url
2025-03-14T04:37:14.4121403Z Entering 'third_party/ittapi'
2025-03-14T04:37:14.4164379Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config	remote.origin.url
2025-03-14T04:37:14.4185513Z Entering 'third_party/kineto'
2025-03-14T04:37:14.4232920Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config	remote.origin.url
2025-03-14T04:37:14.4251175Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-03-14T04:37:14.4296551Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config	remote.origin.url
2025-03-14T04:37:14.4312150Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-03-14T04:37:14.4355702Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config	remote.origin.url
2025-03-14T04:37:14.4377179Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-03-14T04:37:14.4424439Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config	remote.origin.url
2025-03-14T04:37:14.4441983Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-03-14T04:37:14.4484893Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config	remote.origin.url
2025-03-14T04:37:14.4506643Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-03-14T04:37:14.4551573Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config	remote.origin.url
2025-03-14T04:37:14.4568118Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-03-14T04:37:14.4613036Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config	remote.origin.url
2025-03-14T04:37:14.4640685Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-03-14T04:37:14.4684379Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config	remote.origin.url
2025-03-14T04:37:14.4703844Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-03-14T04:37:14.4746698Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config	remote.origin.url
2025-03-14T04:37:14.4767373Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-03-14T04:37:14.4826095Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config	remote.origin.url
2025-03-14T04:37:14.4843530Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-03-14T04:37:14.4886536Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config	remote.origin.url
2025-03-14T04:37:14.4910496Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-03-14T04:37:14.4954415Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config	remote.origin.url
2025-03-14T04:37:14.4976035Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-03-14T04:37:14.5021209Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config	remote.origin.url
2025-03-14T04:37:14.5040155Z Entering 'third_party/kleidiai'
2025-03-14T04:37:14.5083395Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kleidiai/config	remote.origin.url
2025-03-14T04:37:14.5097570Z Entering 'third_party/mimalloc'
2025-03-14T04:37:14.5147030Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config	remote.origin.url
2025-03-14T04:37:14.5163990Z Entering 'third_party/nlohmann'
2025-03-14T04:37:14.5214917Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config	remote.origin.url
2025-03-14T04:37:14.5233484Z Entering 'third_party/onnx'
2025-03-14T04:37:14.5277635Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config	remote.origin.url
2025-03-14T04:37:14.5312735Z Entering 'third_party/onnx/third_party/pybind11'
2025-03-14T04:37:14.5357112Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config	remote.origin.url
2025-03-14T04:37:14.5376930Z Entering 'third_party/opentelemetry-cpp'
2025-03-14T04:37:14.5424417Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config	remote.origin.url
2025-03-14T04:37:14.5443351Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-03-14T04:37:14.5485964Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config	remote.origin.url
2025-03-14T04:37:14.5506740Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-03-14T04:37:14.5552434Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config	remote.origin.url
2025-03-14T04:37:14.5571732Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-03-14T04:37:14.5615036Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config	remote.origin.url
2025-03-14T04:37:14.5636999Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-03-14T04:37:14.5683521Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config	remote.origin.url
2025-03-14T04:37:14.5701793Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-03-14T04:37:14.5747079Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config	remote.origin.url
2025-03-14T04:37:14.5766224Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-03-14T04:37:14.5813365Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config	remote.origin.url
2025-03-14T04:37:14.5830682Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-03-14T04:37:14.5872722Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config	remote.origin.url
2025-03-14T04:37:14.5892848Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-03-14T04:37:14.5935889Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config	remote.origin.url
2025-03-14T04:37:14.5950054Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-03-14T04:37:14.5996653Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config	remote.origin.url
2025-03-14T04:37:14.6020183Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-03-14T04:37:14.6062931Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config	remote.origin.url
2025-03-14T04:37:14.6094832Z Entering 'third_party/pocketfft'
2025-03-14T04:37:14.6145988Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config	remote.origin.url
2025-03-14T04:37:14.6160476Z Entering 'third_party/protobuf'
2025-03-14T04:37:14.6206107Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config	remote.origin.url
2025-03-14T04:37:14.6225663Z Entering 'third_party/protobuf/third_party/benchmark'
2025-03-14T04:37:14.6268898Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config	remote.origin.url
2025-03-14T04:37:14.6286087Z Entering 'third_party/protobuf/third_party/googletest'
2025-03-14T04:37:14.6333750Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config	remote.origin.url
2025-03-14T04:37:14.6351897Z Entering 'third_party/psimd'
2025-03-14T04:37:14.6395236Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config	remote.origin.url
2025-03-14T04:37:14.6412711Z Entering 'third_party/pthreadpool'
2025-03-14T04:37:14.6465869Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config	remote.origin.url
2025-03-14T04:37:14.6486042Z Entering 'third_party/pybind11'
2025-03-14T04:37:14.6532236Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config	remote.origin.url
2025-03-14T04:37:14.6551264Z Entering 'third_party/python-peachpy'
2025-03-14T04:37:14.6595089Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config	remote.origin.url
2025-03-14T04:37:14.6612660Z Entering 'third_party/sleef'
2025-03-14T04:37:14.6661609Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config	remote.origin.url
2025-03-14T04:37:14.6685251Z Entering 'third_party/tensorpipe'
2025-03-14T04:37:14.6732586Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config	remote.origin.url
2025-03-14T04:37:14.6747906Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-03-14T04:37:14.6790102Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config	remote.origin.url
2025-03-14T04:37:14.6804230Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-03-14T04:37:14.6848690Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config	remote.origin.url
2025-03-14T04:37:14.6866064Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-03-14T04:37:14.6910728Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config	remote.origin.url
2025-03-14T04:37:14.6928093Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-03-14T04:37:14.6973797Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config	remote.origin.url
2025-03-14T04:37:14.6985313Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-03-14T04:37:14.7029271Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config	remote.origin.url
2025-03-14T04:37:14.7894036Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:'
2025-03-14T04:37:14.8171879Z Entering 'android/libs/fbjni'
2025-03-14T04:37:14.8214938Z Entering 'third_party/FP16'
2025-03-14T04:37:14.8255958Z Entering 'third_party/FXdiv'
2025-03-14T04:37:14.8291598Z Entering 'third_party/NNPACK'
2025-03-14T04:37:14.8330995Z Entering 'third_party/NVTX'
2025-03-14T04:37:14.8377761Z Entering 'third_party/VulkanMemoryAllocator'
2025-03-14T04:37:14.8418445Z Entering 'third_party/XNNPACK'
2025-03-14T04:37:14.8474261Z Entering 'third_party/benchmark'
2025-03-14T04:37:14.8511734Z Entering 'third_party/composable_kernel'
2025-03-14T04:37:14.8562382Z Entering 'third_party/cpp-httplib'
2025-03-14T04:37:14.8597003Z Entering 'third_party/cpuinfo'
2025-03-14T04:37:14.8636964Z Entering 'third_party/cudnn_frontend'
2025-03-14T04:37:14.8676849Z Entering 'third_party/cutlass'
2025-03-14T04:37:14.8725462Z Entering 'third_party/eigen'
2025-03-14T04:37:14.8763979Z Entering 'third_party/fbgemm'
2025-03-14T04:37:14.8805886Z Entering 'third_party/fbgemm/third_party/asmjit'
2025-03-14T04:37:14.8849865Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2025-03-14T04:37:14.8885062Z Entering 'third_party/fbgemm/third_party/cutlass'
2025-03-14T04:37:14.8933203Z Entering 'third_party/fbgemm/third_party/googletest'
2025-03-14T04:37:14.8970633Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2025-03-14T04:37:14.9015644Z Entering 'third_party/flash-attention'
2025-03-14T04:37:14.9055746Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-03-14T04:37:14.9098620Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-03-14T04:37:14.9149840Z Entering 'third_party/flatbuffers'
2025-03-14T04:37:14.9197018Z Entering 'third_party/fmt'
2025-03-14T04:37:14.9246200Z Entering 'third_party/gemmlowp/gemmlowp'
2025-03-14T04:37:14.9286013Z Entering 'third_party/gloo'
2025-03-14T04:37:14.9327859Z Entering 'third_party/googletest'
2025-03-14T04:37:14.9374232Z Entering 'third_party/ideep'
2025-03-14T04:37:14.9413908Z Entering 'third_party/ideep/mkl-dnn'
2025-03-14T04:37:14.9461171Z Entering 'third_party/ittapi'
2025-03-14T04:37:14.9506659Z Entering 'third_party/kineto'
2025-03-14T04:37:14.9542426Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-03-14T04:37:14.9585389Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-03-14T04:37:14.9623647Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-03-14T04:37:14.9659633Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-03-14T04:37:14.9701624Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-03-14T04:37:14.9745015Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-03-14T04:37:14.9788714Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-03-14T04:37:14.9835778Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-03-14T04:37:14.9875415Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-03-14T04:37:14.9918899Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-03-14T04:37:14.9956722Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-03-14T04:37:14.9990253Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-03-14T04:37:15.0027235Z Entering 'third_party/kleidiai'
2025-03-14T04:37:15.0064323Z Entering 'third_party/mimalloc'
2025-03-14T04:37:15.0107582Z Entering 'third_party/nlohmann'
2025-03-14T04:37:15.0148629Z Entering 'third_party/onnx'
2025-03-14T04:37:15.0210630Z Entering 'third_party/onnx/third_party/pybind11'
2025-03-14T04:37:15.0256609Z Entering 'third_party/opentelemetry-cpp'
2025-03-14T04:37:15.0299718Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-03-14T04:37:15.0340925Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-03-14T04:37:15.0385978Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-03-14T04:37:15.0432600Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-03-14T04:37:15.0474686Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-03-14T04:37:15.0509762Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-03-14T04:37:15.0544753Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-03-14T04:37:15.0586043Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-03-14T04:37:15.0624556Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-03-14T04:37:15.0661444Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-03-14T04:37:15.0723740Z Entering 'third_party/pocketfft'
2025-03-14T04:37:15.0766217Z Entering 'third_party/protobuf'
2025-03-14T04:37:15.0817113Z Entering 'third_party/protobuf/third_party/benchmark'
2025-03-14T04:37:15.0851995Z Entering 'third_party/protobuf/third_party/googletest'
2025-03-14T04:37:15.0894184Z Entering 'third_party/psimd'
2025-03-14T04:37:15.0931581Z Entering 'third_party/pthreadpool'
2025-03-14T04:37:15.0975368Z Entering 'third_party/pybind11'
2025-03-14T04:37:15.1023798Z Entering 'third_party/python-peachpy'
2025-03-14T04:37:15.1066603Z Entering 'third_party/sleef'
2025-03-14T04:37:15.1113326Z Entering 'third_party/tensorpipe'
2025-03-14T04:37:15.1155753Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-03-14T04:37:15.1192341Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-03-14T04:37:15.1234464Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-03-14T04:37:15.1272454Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-03-14T04:37:15.1307577Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-03-14T04:37:15.1358947Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:'
2025-03-14T04:37:15.1644881Z Entering 'android/libs/fbjni'
2025-03-14T04:37:15.1680202Z Entering 'third_party/FP16'
2025-03-14T04:37:15.1729735Z Entering 'third_party/FXdiv'
2025-03-14T04:37:15.1771399Z Entering 'third_party/NNPACK'
2025-03-14T04:37:15.1808293Z Entering 'third_party/NVTX'
2025-03-14T04:37:15.1852652Z Entering 'third_party/VulkanMemoryAllocator'
2025-03-14T04:37:15.1888710Z Entering 'third_party/XNNPACK'
2025-03-14T04:37:15.1947460Z Entering 'third_party/benchmark'
2025-03-14T04:37:15.1982933Z Entering 'third_party/composable_kernel'
2025-03-14T04:37:15.2035023Z Entering 'third_party/cpp-httplib'
2025-03-14T04:37:15.2071826Z Entering 'third_party/cpuinfo'
2025-03-14T04:37:15.2121154Z Entering 'third_party/cudnn_frontend'
2025-03-14T04:37:15.2161937Z Entering 'third_party/cutlass'
2025-03-14T04:37:15.2212932Z Entering 'third_party/eigen'
2025-03-14T04:37:15.2254047Z Entering 'third_party/fbgemm'
2025-03-14T04:37:15.2290208Z Entering 'third_party/fbgemm/third_party/asmjit'
2025-03-14T04:37:15.2329980Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2025-03-14T04:37:15.2372306Z Entering 'third_party/fbgemm/third_party/cutlass'
2025-03-14T04:37:15.2426745Z Entering 'third_party/fbgemm/third_party/googletest'
2025-03-14T04:37:15.2467180Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2025-03-14T04:37:15.2509885Z Entering 'third_party/flash-attention'
2025-03-14T04:37:15.2551462Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-03-14T04:37:15.2590954Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-03-14T04:37:15.2642242Z Entering 'third_party/flatbuffers'
2025-03-14T04:37:15.2683738Z Entering 'third_party/fmt'
2025-03-14T04:37:15.2731776Z Entering 'third_party/gemmlowp/gemmlowp'
2025-03-14T04:37:15.2773495Z Entering 'third_party/gloo'
2025-03-14T04:37:15.2810471Z Entering 'third_party/googletest'
2025-03-14T04:37:15.2852937Z Entering 'third_party/ideep'
2025-03-14T04:37:15.2888791Z Entering 'third_party/ideep/mkl-dnn'
2025-03-14T04:37:15.2938776Z Entering 'third_party/ittapi'
2025-03-14T04:37:15.2987581Z Entering 'third_party/kineto'
2025-03-14T04:37:15.3028285Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-03-14T04:37:15.3062920Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-03-14T04:37:15.3108710Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-03-14T04:37:15.3152821Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-03-14T04:37:15.3187208Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-03-14T04:37:15.3226952Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-03-14T04:37:15.3264802Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-03-14T04:37:15.3311565Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-03-14T04:37:15.3351263Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-03-14T04:37:15.3390978Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-03-14T04:37:15.3431834Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-03-14T04:37:15.3467365Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-03-14T04:37:15.3505553Z Entering 'third_party/kleidiai'
2025-03-14T04:37:15.3551148Z Entering 'third_party/mimalloc'
2025-03-14T04:37:15.3590177Z Entering 'third_party/nlohmann'
2025-03-14T04:37:15.3633670Z Entering 'third_party/onnx'
2025-03-14T04:37:15.3693521Z Entering 'third_party/onnx/third_party/pybind11'
2025-03-14T04:37:15.3742879Z Entering 'third_party/opentelemetry-cpp'
2025-03-14T04:37:15.3790779Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-03-14T04:37:15.3829050Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-03-14T04:37:15.3865111Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-03-14T04:37:15.3907117Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-03-14T04:37:15.3945452Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-03-14T04:37:15.3979995Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-03-14T04:37:15.4015319Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-03-14T04:37:15.4058923Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-03-14T04:37:15.4095100Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-03-14T04:37:15.4132134Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-03-14T04:37:15.4192118Z Entering 'third_party/pocketfft'
2025-03-14T04:37:15.4232555Z Entering 'third_party/protobuf'
2025-03-14T04:37:15.4274656Z Entering 'third_party/protobuf/third_party/benchmark'
2025-03-14T04:37:15.4318244Z Entering 'third_party/protobuf/third_party/googletest'
2025-03-14T04:37:15.4362146Z Entering 'third_party/psimd'
2025-03-14T04:37:15.4403622Z Entering 'third_party/pthreadpool'
2025-03-14T04:37:15.4439724Z Entering 'third_party/pybind11'
2025-03-14T04:37:15.4480754Z Entering 'third_party/python-peachpy'
2025-03-14T04:37:15.4521486Z Entering 'third_party/sleef'
2025-03-14T04:37:15.4561832Z Entering 'third_party/tensorpipe'
2025-03-14T04:37:15.4596895Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-03-14T04:37:15.4637604Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-03-14T04:37:15.4680918Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-03-14T04:37:15.4718617Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-03-14T04:37:15.4752354Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-03-14T04:37:15.4810799Z ##[endgroup]
2025-03-14T04:37:15.4847288Z [command]/usr/bin/git log -1 --format=%H
2025-03-14T04:37:15.4868417Z aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:37:15.5072953Z Prepare all required actions
2025-03-14T04:37:15.5073540Z Getting action download info
2025-03-14T04:37:15.6276139Z ##[group]Run ./.github/actions/setup-linux
2025-03-14T04:37:15.6276506Z env:
2025-03-14T04:37:15.6276762Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:37:15.6277062Z ##[endgroup]
2025-03-14T04:37:15.6326098Z ##[group]Run set -euo pipefail
2025-03-14T04:37:15.6326494Z set -euo pipefail
2025-03-14T04:37:15.6326830Z function get_ec2_metadata() {
2025-03-14T04:37:15.6327408Z   # Pulled from instance metadata endpoint for EC2
2025-03-14T04:37:15.6328106Z   # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html
2025-03-14T04:37:15.6328720Z   category=$1
2025-03-14T04:37:15.6329127Z   # If it is GCP runner (runner name contains gcp), do not run this
2025-03-14T04:37:15.6329607Z   runner_name_str=i-0d8b0d116e8f11da9
2025-03-14T04:37:15.6330027Z   if [[ -f /.inarc ]]; then
2025-03-14T04:37:15.6330416Z     echo "ARC Runner, no info on ec2 metadata"
2025-03-14T04:37:15.6330858Z   elif [[ $runner_name_str == *"gcp"* ]]; then
2025-03-14T04:37:15.6331382Z     echo "Runner is from Google Cloud Platform, No info on ec2 metadata"
2025-03-14T04:37:15.6331859Z   else
2025-03-14T04:37:15.6332792Z     curl -H "X-aws-ec2-metadata-token: $(curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 30")" -fsSL "http://169.254.169.254/latest/meta-data/${category}"
2025-03-14T04:37:15.6333788Z   fi
2025-03-14T04:37:15.6334041Z }
2025-03-14T04:37:15.6334340Z echo "ami-id: $(get_ec2_metadata ami-id)"
2025-03-14T04:37:15.6334816Z echo "instance-id: $(get_ec2_metadata instance-id)"
2025-03-14T04:37:15.6335331Z echo "instance-type: $(get_ec2_metadata instance-type)"
2025-03-14T04:37:15.6335800Z echo "system info $(uname -a)"
2025-03-14T04:37:15.6341630Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:37:15.6342055Z env:
2025-03-14T04:37:15.6342312Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:37:15.6342595Z ##[endgroup]
2025-03-14T04:37:15.6481598Z ami-id: ami-08b5b3a93ed654d19
2025-03-14T04:37:15.6574279Z instance-id: i-0d8b0d116e8f11da9
2025-03-14T04:37:15.6671780Z instance-type: g4dn.metal
2025-03-14T04:37:15.6680492Z system info Linux ip-10-0-69-51.ec2.internal 6.1.129-138.220.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 25 22:18:43 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
2025-03-14T04:37:15.6720268Z ##[group]Run echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT"
2025-03-14T04:37:15.6721330Z echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT"
2025-03-14T04:37:15.6728350Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:37:15.6728811Z env:
2025-03-14T04:37:15.6729085Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:37:15.6729390Z ##[endgroup]
2025-03-14T04:37:15.6829189Z ##[group]Run if systemctl is-active --quiet docker; then
2025-03-14T04:37:15.6829681Z if systemctl is-active --quiet docker; then
2025-03-14T04:37:15.6830104Z     echo "Docker daemon is running...";
2025-03-14T04:37:15.6830455Z else
2025-03-14T04:37:15.6830838Z     echo "Starting docker deamon..." && sudo systemctl start docker;
2025-03-14T04:37:15.6831318Z fi
2025-03-14T04:37:15.6836306Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:37:15.6836727Z env:
2025-03-14T04:37:15.6836966Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:37:15.6837253Z ##[endgroup]
2025-03-14T04:37:15.6903775Z Docker daemon is running...
2025-03-14T04:37:15.6962866Z ##[group]Run nick-fields/retry@v3.0.0
2025-03-14T04:37:15.6963357Z with:
2025-03-14T04:37:15.6963616Z   shell: bash
2025-03-14T04:37:15.6964069Z   timeout_minutes: 5
2025-03-14T04:37:15.6964354Z   max_attempts: 3
2025-03-14T04:37:15.6964637Z   retry_wait_seconds: 30
2025-03-14T04:37:15.6967099Z   command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\")
aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \
    --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com"

# For LF Runners we need to make sure we also login to Meta's ECR docker registry too.
META_AWS_ACCOUNT_ID=308535385114
if [ "$AWS_ACCOUNT_ID" != "$META_AWS_ACCOUNT_ID" ] ; then
    aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \
        --password-stdin "$META_AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com"
fi

2025-03-14T04:37:15.6969716Z   polling_interval_seconds: 1
2025-03-14T04:37:15.6970049Z   warning_on_retry: true
2025-03-14T04:37:15.6970369Z   continue_on_error: false
2025-03-14T04:37:15.6970652Z env:
2025-03-14T04:37:15.6970913Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:37:15.6971230Z   AWS_RETRY_MODE: standard
2025-03-14T04:37:15.6971537Z   AWS_MAX_ATTEMPTS: 5
2025-03-14T04:37:15.6971836Z   AWS_DEFAULT_REGION: us-east-1
2025-03-14T04:37:15.6972161Z ##[endgroup]
2025-03-14T04:37:16.7756799Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json.
2025-03-14T04:37:16.7758306Z Configure a credential helper to remove this warning. See
2025-03-14T04:37:16.7759102Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store
2025-03-14T04:37:16.7759586Z 
2025-03-14T04:37:16.7759695Z Login Succeeded
2025-03-14T04:37:16.7823303Z Command completed after 1 attempt(s).
2025-03-14T04:37:16.7884777Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}"
2025-03-14T04:37:16.7885354Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}"
2025-03-14T04:37:16.7885860Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}"
2025-03-14T04:37:16.7892522Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:37:16.7892978Z env:
2025-03-14T04:37:16.7893256Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:37:16.7893576Z ##[endgroup]
2025-03-14T04:37:16.7989849Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty
2025-03-14T04:37:16.7990718Z # ignore expansion of "docker ps -q" since it could be empty
2025-03-14T04:37:16.7991346Z # shellcheck disable=SC2046
2025-03-14T04:37:16.7991790Z docker stop $(docker ps -q) || true
2025-03-14T04:37:16.7992260Z # Prune all of the docker images
2025-03-14T04:37:16.7992679Z docker system prune -af
2025-03-14T04:37:16.7998387Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:37:16.7998878Z env:
2025-03-14T04:37:16.7999174Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:37:16.7999520Z ##[endgroup]
2025-03-14T04:37:16.8243182Z "docker stop" requires at least 1 argument.
2025-03-14T04:37:16.8243681Z See 'docker stop --help'.
2025-03-14T04:37:16.8243907Z 
2025-03-14T04:37:16.8244123Z Usage:  docker stop [OPTIONS] CONTAINER [CONTAINER...]
2025-03-14T04:37:16.8244459Z 
2025-03-14T04:37:16.8244597Z Stop one or more running containers
2025-03-14T04:37:16.8386026Z Total reclaimed space: 0B
2025-03-14T04:37:16.8427774Z ##[group]Run set +e
2025-03-14T04:37:16.8428124Z set +e
2025-03-14T04:37:16.8428421Z set -x
2025-03-14T04:37:16.8428715Z 
2025-03-14T04:37:16.8429002Z PT_DOMAIN=download.pytorch.org
2025-03-14T04:37:16.8429647Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400,
2025-03-14T04:37:16.8430512Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last
2025-03-14T04:37:16.8431131Z # one is returned at random
2025-03-14T04:37:16.8431608Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1)
2025-03-14T04:37:16.8432074Z 
2025-03-14T04:37:16.8432525Z if [ -z "${RESOLVED_IP}" ]; then
2025-03-14T04:37:16.8433040Z   echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..."
2025-03-14T04:37:16.8433636Z   RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1)
2025-03-14T04:37:16.8434076Z 
2025-03-14T04:37:16.8434352Z   if [ -z "${RESOLVED_IP}" ]; then
2025-03-14T04:37:16.8434794Z     echo "Couldn't resolve ${PT_DOMAIN}, exiting..."
2025-03-14T04:37:16.8435218Z     exit 1
2025-03-14T04:37:16.8435496Z   fi
2025-03-14T04:37:16.8440531Z fi
2025-03-14T04:37:16.8440792Z 
2025-03-14T04:37:16.8441124Z if grep -r "${PT_DOMAIN}" /etc/hosts; then
2025-03-14T04:37:16.8441585Z   # Clean up any old records first
2025-03-14T04:37:16.8442036Z   sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts
2025-03-14T04:37:16.8442433Z fi
2025-03-14T04:37:16.8442708Z 
2025-03-14T04:37:16.8443112Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts
2025-03-14T04:37:16.8443591Z cat /etc/hosts
2025-03-14T04:37:16.8448721Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:37:16.8449199Z env:
2025-03-14T04:37:16.8449476Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:37:16.8449811Z ##[endgroup]
2025-03-14T04:37:16.8471498Z + PT_DOMAIN=download.pytorch.org
2025-03-14T04:37:16.8476671Z ++ dig -4 +short download.pytorch.org
2025-03-14T04:37:16.8477626Z ++ tail -n1
2025-03-14T04:37:16.8875693Z + RESOLVED_IP=18.160.10.28
2025-03-14T04:37:16.8877164Z + '[' -z 18.160.10.28 ']'
2025-03-14T04:37:16.8878207Z + grep -r download.pytorch.org /etc/hosts
2025-03-14T04:37:16.8886711Z 18.160.10.76 download.pytorch.org
2025-03-14T04:37:16.8887911Z + sudo sed -i /download.pytorch.org/d /etc/hosts
2025-03-14T04:37:16.9857439Z + echo '18.160.10.28 download.pytorch.org'
2025-03-14T04:37:16.9858929Z + sudo tee -a /etc/hosts
2025-03-14T04:37:17.1435107Z 18.160.10.28 download.pytorch.org
2025-03-14T04:37:17.1452973Z + cat /etc/hosts
2025-03-14T04:37:17.1461775Z 127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
2025-03-14T04:37:17.1467719Z ::1         localhost6 localhost6.localdomain6
2025-03-14T04:37:17.1468173Z 18.160.10.28 download.pytorch.org
2025-03-14T04:37:17.1639795Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main
2025-03-14T04:37:17.1640326Z with:
2025-03-14T04:37:17.1641248Z   docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:37:17.1642449Z   docker-build-dir: .ci/docker
2025-03-14T04:37:17.1642782Z   working-directory: .
2025-03-14T04:37:17.1643175Z   docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-03-14T04:37:17.1643631Z   force-push: false
2025-03-14T04:37:17.1643895Z env:
2025-03-14T04:37:17.1644139Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:37:17.1644442Z ##[endgroup]
2025-03-14T04:37:17.1673943Z ##[group]Run set -ex
2025-03-14T04:37:17.1674289Z set -ex
2025-03-14T04:37:17.1674548Z 
2025-03-14T04:37:17.1675002Z # If the docker build directory or the build script doesn't exist, the action will
2025-03-14T04:37:17.1675810Z # gracefully return the docker image name as it is.  Pulling docker image in Linux
2025-03-14T04:37:17.1676445Z # job could then download the pre-built image as usual
2025-03-14T04:37:17.1677021Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then
2025-03-14T04:37:17.1677562Z   echo "skip=true" >> "${GITHUB_OUTPUT}"
2025-03-14T04:37:17.1678073Z   echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}"
2025-03-14T04:37:17.1678541Z 
2025-03-14T04:37:17.1678944Z   echo "There is no Docker build script in ${REPO_NAME} repo, skipping..."
2025-03-14T04:37:17.1679447Z   exit 0
2025-03-14T04:37:17.1679718Z else
2025-03-14T04:37:17.1680017Z   echo "skip=false" >> "${GITHUB_OUTPUT}"
2025-03-14T04:37:17.1680398Z fi
2025-03-14T04:37:17.1680652Z 
2025-03-14T04:37:17.1681051Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then
2025-03-14T04:37:17.1681723Z   # The docker image name already includes the ECR prefix and tag, so we can just
2025-03-14T04:37:17.1682327Z   # use it as it is, but first let's extract the tag
2025-03-14T04:37:17.1682869Z   DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}')
2025-03-14T04:37:17.1683596Z   echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}"
2025-03-14T04:37:17.1684151Z   echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}"
2025-03-14T04:37:17.1684620Z else
2025-03-14T04:37:17.1684988Z   DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}")
2025-03-14T04:37:17.1685514Z   echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}"
2025-03-14T04:37:17.1686227Z   echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}"
2025-03-14T04:37:17.1686859Z fi
2025-03-14T04:37:17.1692508Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:37:17.1692976Z env:
2025-03-14T04:37:17.1693259Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:37:17.1693602Z   REPO_NAME: pytorch
2025-03-14T04:37:17.1694635Z   DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:37:17.1695744Z   DOCKER_BUILD_DIR: .ci/docker
2025-03-14T04:37:17.1696205Z   DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-03-14T04:37:17.1696681Z ##[endgroup]
2025-03-14T04:37:17.1720350Z + [[ ! -d .ci/docker ]]
2025-03-14T04:37:17.1720821Z + [[ ! -f .ci/docker/build.sh ]]
2025-03-14T04:37:17.1721327Z + echo skip=false
2025-03-14T04:37:17.1724555Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:aa89d6e739080d90fa18625d57297c6734465849 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]]
2025-03-14T04:37:17.1728930Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:37:17.1730104Z ++ awk -F '[:,]' '{print $2}'
2025-03-14T04:37:17.1749990Z + DOCKER_TAG=aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:37:17.1750567Z + echo docker-tag=aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:37:17.1752456Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:37:17.1786920Z ##[group]Run set +e
2025-03-14T04:37:17.1787266Z set +e
2025-03-14T04:37:17.1787539Z set -x
2025-03-14T04:37:17.1787796Z 
2025-03-14T04:37:17.1788049Z login() {
2025-03-14T04:37:17.1788590Z   aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1"
2025-03-14T04:37:17.1789174Z }
2025-03-14T04:37:17.1789424Z 
2025-03-14T04:37:17.1789675Z retry () {
2025-03-14T04:37:17.1790377Z   $*  || (sleep 1 && $*) || (sleep 2 && $*)
2025-03-14T04:37:17.1790826Z }
2025-03-14T04:37:17.1791115Z 
2025-03-14T04:37:17.1791407Z retry login "${DOCKER_REGISTRY}"
2025-03-14T04:37:17.1791824Z 
2025-03-14T04:37:17.1792119Z START_TIME=$(date +%s)
2025-03-14T04:37:17.1792513Z # Wait up to 120 minutes
2025-03-14T04:37:17.1792986Z while [[ $(( $(date +%s) - 7200 )) -lt $START_TIME ]]; do
2025-03-14T04:37:17.1793587Z   # Check if image already exists, if it does then skip building it
2025-03-14T04:37:17.1794188Z   if docker manifest inspect "${DOCKER_IMAGE}"; then
2025-03-14T04:37:17.1794639Z     exit 0
2025-03-14T04:37:17.1794937Z   fi
2025-03-14T04:37:17.1795243Z 
2025-03-14T04:37:17.1795732Z   # NB: This flag is used by Docker build workflow to push the image to ECR, so we can
2025-03-14T04:37:17.1796548Z   # use this to differentiate between the Docker build and regular build jobs. For the
2025-03-14T04:37:17.1797377Z   # latter, it will wait for the Docker images to become available before continuing
2025-03-14T04:37:17.1798198Z   if [ "${DOCKER_PUSH:-false}" == "true" ]; then
2025-03-14T04:37:17.1798734Z     # It's a Docker build job, let's build the image
2025-03-14T04:37:17.1799197Z     break
2025-03-14T04:37:17.1799511Z   else
2025-03-14T04:37:17.1799975Z     # It's a regular build job, wait for the image to become available
2025-03-14T04:37:17.1800497Z     sleep 300
2025-03-14T04:37:17.1800838Z   fi
2025-03-14T04:37:17.1801147Z done
2025-03-14T04:37:17.1801452Z 
2025-03-14T04:37:17.1801900Z # NB: This part requires a full checkout. Otherwise, the merge base will
2025-03-14T04:37:17.1802808Z # be empty.  The default action would be to continue rebuild the image
2025-03-14T04:37:17.1803389Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then
2025-03-14T04:37:17.1803905Z   # if we're on the base branch then use the parent commit
2025-03-14T04:37:17.1804368Z   MERGE_BASE=$(git rev-parse HEAD~)
2025-03-14T04:37:17.1804747Z else
2025-03-14T04:37:17.1805130Z   # otherwise we're on a PR, so use the most recent base commit
2025-03-14T04:37:17.1805662Z   MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION")
2025-03-14T04:37:17.1806080Z fi
2025-03-14T04:37:17.1806362Z 
2025-03-14T04:37:17.1806659Z if [[ -z "${MERGE_BASE}" ]]; then
2025-03-14T04:37:17.1807091Z   echo "rebuild=true" >> "${GITHUB_OUTPUT}"
2025-03-14T04:37:17.1807497Z 
2025-03-14T04:37:17.1808148Z   echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..."
2025-03-14T04:37:17.1808802Z   exit 0
2025-03-14T04:37:17.1809076Z fi
2025-03-14T04:37:17.1809333Z 
2025-03-14T04:37:17.1809689Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then
2025-03-14T04:37:17.1810458Z   echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit"
2025-03-14T04:37:17.1811148Z   exit 1
2025-03-14T04:37:17.1811422Z fi
2025-03-14T04:37:17.1811684Z 
2025-03-14T04:37:17.1812093Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}")
2025-03-14T04:37:17.1827102Z # If no image exists but the hash is the same as the previous hash then we should error out here
2025-03-14T04:37:17.1827752Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then
2025-03-14T04:37:17.1828506Z   echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch"
2025-03-14T04:37:17.1829342Z   echo "         Will re-build docker image to store in local cache, TTS may be longer"
2025-03-14T04:37:17.1829844Z fi
2025-03-14T04:37:17.1830077Z 
2025-03-14T04:37:17.1830354Z echo "rebuild=true" >> "${GITHUB_OUTPUT}"
2025-03-14T04:37:17.1835216Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:37:17.1835625Z env:
2025-03-14T04:37:17.1835858Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:37:17.1836175Z   DOCKER_BUILD_DIR: .ci/docker
2025-03-14T04:37:17.1836545Z   BASE_REVISION: aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:37:17.1837576Z   DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:37:17.1838617Z   DOCKER_TAG: aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:37:17.1839103Z   DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-03-14T04:37:17.1839528Z   DOCKER_PUSH: 
2025-03-14T04:37:17.1839778Z ##[endgroup]
2025-03-14T04:37:17.1860884Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-03-14T04:37:17.1861393Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-03-14T04:37:17.1864079Z + aws ecr get-login-password --region us-east-1
2025-03-14T04:37:17.1865422Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-03-14T04:37:17.6787655Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json.
2025-03-14T04:37:17.6788422Z Configure a credential helper to remove this warning. See
2025-03-14T04:37:17.6790099Z Login Succeeded
2025-03-14T04:37:17.6792740Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store
2025-03-14T04:37:17.6794031Z 
2025-03-14T04:37:17.6809155Z ++ date +%s
2025-03-14T04:37:17.6818359Z + START_TIME=1741927037
2025-03-14T04:37:17.6823994Z ++ date +%s
2025-03-14T04:37:17.6832192Z + [[ 1741919837 -lt 1741927037 ]]
2025-03-14T04:37:17.6834364Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:37:17.9594214Z {
2025-03-14T04:37:17.9594649Z 	"schemaVersion": 2,
2025-03-14T04:37:17.9595194Z 	"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
2025-03-14T04:37:17.9595748Z 	"config": {
2025-03-14T04:37:17.9596161Z 		"mediaType": "application/vnd.docker.container.image.v1+json",
2025-03-14T04:37:17.9596648Z 		"size": 52676,
2025-03-14T04:37:17.9597145Z 		"digest": "sha256:67c80a026a8320d7a154e47f931cc5dfab55468390062655a56216de80309c6a"
2025-03-14T04:37:17.9597711Z 	},
2025-03-14T04:37:17.9597946Z 	"layers": [
2025-03-14T04:37:17.9598207Z 		{
2025-03-14T04:37:17.9598614Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9599119Z 			"size": 28584506,
2025-03-14T04:37:17.9599877Z 			"digest": "sha256:80888bc6716fcbb8874e75ac88898d3e38e6f1bc55678f0e97ca9d706b7f3733"
2025-03-14T04:37:17.9600453Z 		},
2025-03-14T04:37:17.9600705Z 		{
2025-03-14T04:37:17.9601107Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9601617Z 			"size": 7944698,
2025-03-14T04:37:17.9602136Z 			"digest": "sha256:fbcd35dc5bc3a7bda41926aadd083020f942b001ebac6f1d30480f0f065394c0"
2025-03-14T04:37:17.9602816Z 		},
2025-03-14T04:37:17.9603059Z 		{
2025-03-14T04:37:17.9603561Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9604024Z 			"size": 57593527,
2025-03-14T04:37:17.9604502Z 			"digest": "sha256:c7232af9ae05f7de83f8d6171bd0c35a4dd0a85ebafb15b950dbc08f89ea5fb5"
2025-03-14T04:37:17.9605027Z 		},
2025-03-14T04:37:17.9605235Z 		{
2025-03-14T04:37:17.9605608Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9606071Z 			"size": 185,
2025-03-14T04:37:17.9606536Z 			"digest": "sha256:db6cdef1932a0d9ca6ef9a539e08d491f66d1b1ed81926ae1525375bdd8100cc"
2025-03-14T04:37:17.9607070Z 		},
2025-03-14T04:37:17.9607296Z 		{
2025-03-14T04:37:17.9607668Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9608129Z 			"size": 6886,
2025-03-14T04:37:17.9608579Z 			"digest": "sha256:56dc8550293751a1604e97ac949cfae82ba20cb2a28e034737bafd7382559609"
2025-03-14T04:37:17.9609095Z 		},
2025-03-14T04:37:17.9609322Z 		{
2025-03-14T04:37:17.9609692Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9610160Z 			"size": 1374287670,
2025-03-14T04:37:17.9610874Z 			"digest": "sha256:30c0ea6140d07e2a8deb70d780f277c63cf61836ff33d66eef944728a4bef6bd"
2025-03-14T04:37:17.9611644Z 		},
2025-03-14T04:37:17.9611883Z 		{
2025-03-14T04:37:17.9612284Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9612761Z 			"size": 62641,
2025-03-14T04:37:17.9613258Z 			"digest": "sha256:71bdb1a72c2d6dc97bbdbca82383f0260c4ee87556701e8e606c08a6bb0f0da5"
2025-03-14T04:37:17.9613847Z 		},
2025-03-14T04:37:17.9614089Z 		{
2025-03-14T04:37:17.9614492Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9614986Z 			"size": 1685,
2025-03-14T04:37:17.9615465Z 			"digest": "sha256:4829486be7c30f19f4136fa56adbb3de206ed0bbf0705b59fb2147406778ce38"
2025-03-14T04:37:17.9616179Z 		},
2025-03-14T04:37:17.9616440Z 		{
2025-03-14T04:37:17.9616842Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9617337Z 			"size": 1523,
2025-03-14T04:37:17.9617826Z 			"digest": "sha256:1f4e68d7b5e4224ba1da78ef461ff7f01e8d59c09d39281277521384105a9441"
2025-03-14T04:37:17.9618484Z 		},
2025-03-14T04:37:17.9618729Z 		{
2025-03-14T04:37:17.9619107Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9619590Z 			"size": 2626741021,
2025-03-14T04:37:17.9620054Z 			"digest": "sha256:7c373e2d9b7e82a6878d4a31293dd857915a0fe47d07dce541cea03b043d57fc"
2025-03-14T04:37:17.9620566Z 		},
2025-03-14T04:37:17.9620811Z 		{
2025-03-14T04:37:17.9621169Z + exit 0
2025-03-14T04:37:17.9621582Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9622079Z 			"size": 86545,
2025-03-14T04:37:17.9622551Z 			"digest": "sha256:6223811417458a3c93b84ee3b65f8b08d9e2828b926f0aed863041610d7d95d4"
2025-03-14T04:37:17.9623092Z 		},
2025-03-14T04:37:17.9623340Z 		{
2025-03-14T04:37:17.9623734Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9624397Z 			"size": 1895,
2025-03-14T04:37:17.9624896Z 			"digest": "sha256:f190bb8f8a614ae9dfb5a7487f2edff7eddda39ce04dc2e81524e494e736f5cd"
2025-03-14T04:37:17.9625474Z 		},
2025-03-14T04:37:17.9625721Z 		{
2025-03-14T04:37:17.9626114Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9626615Z 			"size": 643749102,
2025-03-14T04:37:17.9627121Z 			"digest": "sha256:7e229290af344a16aacf855d00ca7d3a140bfb1d25e0e2f1231fe43a7c65ef50"
2025-03-14T04:37:17.9627687Z 		},
2025-03-14T04:37:17.9627987Z 		{
2025-03-14T04:37:17.9628373Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9628862Z 			"size": 704,
2025-03-14T04:37:17.9629343Z 			"digest": "sha256:abf1e0eaf8f455eba7e50175ce110426fdd02b10bca2e9031d32c7251045f5be"
2025-03-14T04:37:17.9630030Z 		},
2025-03-14T04:37:17.9630273Z 		{
2025-03-14T04:37:17.9630684Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9631169Z 			"size": 1258,
2025-03-14T04:37:17.9631635Z 			"digest": "sha256:4545fec1915c8d49e34073e658d69e1c96e35f2b67c231ac3116c035e66cc947"
2025-03-14T04:37:17.9632171Z 		},
2025-03-14T04:37:17.9632411Z 		{
2025-03-14T04:37:17.9632780Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9633275Z 			"size": 484,
2025-03-14T04:37:17.9633745Z 			"digest": "sha256:40ee82023c66685eb67168a0dcff10a495a8babe89330afc48240193f31effd1"
2025-03-14T04:37:17.9634296Z 		},
2025-03-14T04:37:17.9634551Z 		{
2025-03-14T04:37:17.9634944Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9635414Z 			"size": 91727648,
2025-03-14T04:37:17.9635888Z 			"digest": "sha256:279be7c466cf12b535dd29c09162ba935243b009cd19388a956743924d17000e"
2025-03-14T04:37:17.9636430Z 		},
2025-03-14T04:37:17.9636681Z 		{
2025-03-14T04:37:17.9637075Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9637549Z 			"size": 3721,
2025-03-14T04:37:17.9638009Z 			"digest": "sha256:b5cb697686fab1f8832bcd0bb18e736dbee90278db6225b2d5254d9995e083cf"
2025-03-14T04:37:17.9638538Z 		},
2025-03-14T04:37:17.9638770Z 		{
2025-03-14T04:37:17.9639143Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9639603Z 			"size": 1860,
2025-03-14T04:37:17.9640068Z 			"digest": "sha256:07ed5400009b33dd3d23e1f0d12d90abd31a930c269400062e3b7d875fe2cba5"
2025-03-14T04:37:17.9640601Z 		},
2025-03-14T04:37:17.9640841Z 		{
2025-03-14T04:37:17.9641211Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9641679Z 			"size": 700,
2025-03-14T04:37:17.9642148Z 			"digest": "sha256:f895057e0aef06da056dc6e851e9341b646df75ceee25cecf482bd43ab954572"
2025-03-14T04:37:17.9642686Z 		},
2025-03-14T04:37:17.9642934Z 		{
2025-03-14T04:37:17.9643428Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9643901Z 			"size": 477,
2025-03-14T04:37:17.9644359Z 			"digest": "sha256:12b934c44e191a042dde40a1c3ecb53dd7cde5398f5ea036ca7da193143f8ad5"
2025-03-14T04:37:17.9644881Z 		},
2025-03-14T04:37:17.9645120Z 		{
2025-03-14T04:37:17.9645506Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9645985Z 			"size": 2922628648,
2025-03-14T04:37:17.9646451Z 			"digest": "sha256:13e6ba8d75a87900125197a710b3ef6a2eb79f8fd24fdb10aa079f53c9b7baea"
2025-03-14T04:37:17.9646957Z 		},
2025-03-14T04:37:17.9647187Z 		{
2025-03-14T04:37:17.9647563Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9648029Z 			"size": 380,
2025-03-14T04:37:17.9648484Z 			"digest": "sha256:0cddd2dd1bb9d9011f3e537274bbd0eb33c97b6c4b00d22a37d96691f3aaefc3"
2025-03-14T04:37:17.9649019Z 		},
2025-03-14T04:37:17.9649242Z 		{
2025-03-14T04:37:17.9649610Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9650080Z 			"size": 12912,
2025-03-14T04:37:17.9650546Z 			"digest": "sha256:19d8410fa3bca5fd64beeacc7b7e61500e14191d436f62f8366721e5c0acb100"
2025-03-14T04:37:17.9651069Z 		},
2025-03-14T04:37:17.9651289Z 		{
2025-03-14T04:37:17.9651672Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9652150Z 			"size": 864,
2025-03-14T04:37:17.9652619Z 			"digest": "sha256:e305964abe7d3e24292f7aca73eea7e3ca183b966092a8c3f8ff9c67b11d77a9"
2025-03-14T04:37:17.9653166Z 		},
2025-03-14T04:37:17.9653383Z 		{
2025-03-14T04:37:17.9653770Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9654251Z 			"size": 106,
2025-03-14T04:37:17.9654772Z 			"digest": "sha256:48470c6711d9175be4810bc0feb259afe4221f6517fb7a2977763ef52ef78aa5"
2025-03-14T04:37:17.9655311Z 		},
2025-03-14T04:37:17.9655545Z 		{
2025-03-14T04:37:17.9655926Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9656388Z 			"size": 504,
2025-03-14T04:37:17.9656833Z 			"digest": "sha256:102ff554f6800e1e05ff7d583e74649848c045acbe577053de5fa676c92c59ef"
2025-03-14T04:37:17.9657342Z 		},
2025-03-14T04:37:17.9657565Z 		{
2025-03-14T04:37:17.9657938Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9658398Z 			"size": 121477375,
2025-03-14T04:37:17.9658861Z 			"digest": "sha256:862f9a6d0194c5dfe6d0116334919bc47ad58594de38de5794f703c21ced5231"
2025-03-14T04:37:17.9659399Z 		},
2025-03-14T04:37:17.9659624Z 		{
2025-03-14T04:37:17.9659977Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9660433Z 			"size": 109,
2025-03-14T04:37:17.9660898Z 			"digest": "sha256:496d42fc1121f866629d863efebeecdcc2531614c71695c7eb5659e4698964a4"
2025-03-14T04:37:17.9661408Z 		},
2025-03-14T04:37:17.9661630Z 		{
2025-03-14T04:37:17.9661992Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9662452Z 			"size": 490,
2025-03-14T04:37:17.9662895Z 			"digest": "sha256:766900d8eb9cdef5963898de920686983406fd34492f888f8d4cb985eea04cad"
2025-03-14T04:37:17.9663403Z 		},
2025-03-14T04:37:17.9663645Z 		{
2025-03-14T04:37:17.9664018Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9664482Z 			"size": 297,
2025-03-14T04:37:17.9664930Z 			"digest": "sha256:2bf5096e9bc945d652abaa8f1c6229e34040f204d2a2107444eff736f23bdd9b"
2025-03-14T04:37:17.9665444Z 		},
2025-03-14T04:37:17.9665666Z 		{
2025-03-14T04:37:17.9666030Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9666489Z 			"size": 103,
2025-03-14T04:37:17.9666916Z 			"digest": "sha256:17fc7d1c873581c1149f87a16877b25395c860f96c2529225fd5f03d963e9dfa"
2025-03-14T04:37:17.9667426Z 		},
2025-03-14T04:37:17.9667649Z 		{
2025-03-14T04:37:17.9668017Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9668480Z 			"size": 1474,
2025-03-14T04:37:17.9668942Z 			"digest": "sha256:73b57ddfc7ffc74db3e012fdf48c82e65b65ed9e9b5657b989c8ec329d33be36"
2025-03-14T04:37:17.9669547Z 		},
2025-03-14T04:37:17.9669772Z 		{
2025-03-14T04:37:17.9670140Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9670609Z 			"size": 427755722,
2025-03-14T04:37:17.9671082Z 			"digest": "sha256:246047bfca4007bfa19e8d1aaf62ee7695e46d7c7714242c2ddcb10a3f46163b"
2025-03-14T04:37:17.9671609Z 		},
2025-03-14T04:37:17.9671835Z 		{
2025-03-14T04:37:17.9672203Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9672662Z 			"size": 164,
2025-03-14T04:37:17.9673112Z 			"digest": "sha256:5314adad3d9204ac426634b79837a2c900c94d0ef79b7c4a0f60330cf959ce9a"
2025-03-14T04:37:17.9673620Z 		},
2025-03-14T04:37:17.9673845Z 		{
2025-03-14T04:37:17.9674212Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9674678Z 			"size": 802,
2025-03-14T04:37:17.9675129Z 			"digest": "sha256:b095bb71008252bf3f3bf13de52f91a4897d46f8f183bb876cf73db94eea3191"
2025-03-14T04:37:17.9675654Z 		},
2025-03-14T04:37:17.9675881Z 		{
2025-03-14T04:37:17.9676242Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9676701Z 			"size": 36083458,
2025-03-14T04:37:17.9677151Z 			"digest": "sha256:a9b21d6249e6e949b639c37f710d15f98d17344f472e0d74395565789fb9eca8"
2025-03-14T04:37:17.9677655Z 		},
2025-03-14T04:37:17.9677875Z 		{
2025-03-14T04:37:17.9678232Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9678693Z 			"size": 104,
2025-03-14T04:37:17.9679135Z 			"digest": "sha256:32b89bc8b340760d177eadf38704df7f19f132562e4b5b187eafab46266a3647"
2025-03-14T04:37:17.9679648Z 		},
2025-03-14T04:37:17.9679918Z 		{
2025-03-14T04:37:17.9680286Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9680744Z 			"size": 425,
2025-03-14T04:37:17.9681197Z 			"digest": "sha256:0f2b30b2c08b9540847c124be2fd235f40467badb82658acef9a6c3b3fa0e1c4"
2025-03-14T04:37:17.9681719Z 		},
2025-03-14T04:37:17.9681941Z 		{
2025-03-14T04:37:17.9682305Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9682765Z 			"size": 20262180,
2025-03-14T04:37:17.9683215Z 			"digest": "sha256:7082c2fbfc1891093f8535e8986296dac6e3f87196d8b8998962c56a6e2565b6"
2025-03-14T04:37:17.9683724Z 		},
2025-03-14T04:37:17.9683944Z 		{
2025-03-14T04:37:17.9684307Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9684765Z 			"size": 644,
2025-03-14T04:37:17.9685207Z 			"digest": "sha256:6c73e7aaf5429c14469525f72fed370d01013d2237a3d91776165eb7a9bbb138"
2025-03-14T04:37:17.9685715Z 		},
2025-03-14T04:37:17.9685935Z 		{
2025-03-14T04:37:17.9686287Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9686745Z 			"size": 700,
2025-03-14T04:37:17.9687197Z 			"digest": "sha256:f895057e0aef06da056dc6e851e9341b646df75ceee25cecf482bd43ab954572"
2025-03-14T04:37:17.9687715Z 		},
2025-03-14T04:37:17.9687938Z 		{
2025-03-14T04:37:17.9688298Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9688755Z 			"size": 141,
2025-03-14T04:37:17.9689203Z 			"digest": "sha256:f78c26edc26c26888ed13e9cc4c93be93c2fa93bc39182d1f28707b1973de958"
2025-03-14T04:37:17.9689783Z 		},
2025-03-14T04:37:17.9690008Z 		{
2025-03-14T04:37:17.9690628Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9691127Z 			"size": 135,
2025-03-14T04:37:17.9691615Z 			"digest": "sha256:dcd44da9f83c81a881856bb128dfad38507af79d389432a01e933946682a9673"
2025-03-14T04:37:17.9692174Z 		},
2025-03-14T04:37:17.9692415Z 		{
2025-03-14T04:37:17.9692810Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9693310Z 			"size": 5220114625,
2025-03-14T04:37:17.9693809Z 			"digest": "sha256:6379c2fe567231f127d0bad4cd3e070cc5379a9fa4e3ddec8336fdb0ffa9736c"
2025-03-14T04:37:17.9694369Z 		},
2025-03-14T04:37:17.9694605Z 		{
2025-03-14T04:37:17.9694999Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9695600Z 			"size": 196,
2025-03-14T04:37:17.9696083Z 			"digest": "sha256:b88092e39da59e8ef899493088fe07cc4ff20f8591a5273f51006b2b4bf6803c"
2025-03-14T04:37:17.9696639Z 		},
2025-03-14T04:37:17.9696876Z 		{
2025-03-14T04:37:17.9697269Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9697768Z 			"size": 565,
2025-03-14T04:37:17.9698254Z 			"digest": "sha256:4ee03c8b3be2ff4c0449bc1f4f87393dd2736d1098e9f60cf99011b3260b35ab"
2025-03-14T04:37:17.9698814Z 		},
2025-03-14T04:37:17.9699054Z 		{
2025-03-14T04:37:17.9699449Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9699951Z 			"size": 43163863,
2025-03-14T04:37:17.9700444Z 			"digest": "sha256:f82587c7a86dff3e00d7c8a8124a2581914d5d175218835b1378405f75ac55be"
2025-03-14T04:37:17.9700996Z 		},
2025-03-14T04:37:17.9701217Z 		{
2025-03-14T04:37:17.9701611Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9702230Z 			"size": 106,
2025-03-14T04:37:17.9702681Z 			"digest": "sha256:15ec2b791b0b8159dc79e9ed60b4a6c2d1cdebb460852f3a2a6aa29c4872d3ab"
2025-03-14T04:37:17.9703201Z 		},
2025-03-14T04:37:17.9703428Z 		{
2025-03-14T04:37:17.9703794Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9704293Z 			"size": 1401,
2025-03-14T04:37:17.9704755Z 			"digest": "sha256:7b9e114c1cb7e87fb310f9def6252232c6a8b9e5f66feb8d20e4aa851e4810f5"
2025-03-14T04:37:17.9705277Z 		},
2025-03-14T04:37:17.9705498Z 		{
2025-03-14T04:37:17.9705864Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9706326Z 			"size": 700,
2025-03-14T04:37:17.9706858Z 			"digest": "sha256:f895057e0aef06da056dc6e851e9341b646df75ceee25cecf482bd43ab954572"
2025-03-14T04:37:17.9707386Z 		},
2025-03-14T04:37:17.9707611Z 		{
2025-03-14T04:37:17.9707962Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9708431Z 			"size": 139,
2025-03-14T04:37:17.9708876Z 			"digest": "sha256:429a8d451044fccb167a915294d8e90be87835953ae64d129f14348f6f0711e1"
2025-03-14T04:37:17.9709391Z 		},
2025-03-14T04:37:17.9709613Z 		{
2025-03-14T04:37:17.9709981Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9710439Z 			"size": 120,
2025-03-14T04:37:17.9710899Z 			"digest": "sha256:44156619fef12d2fc442c3d0f2bc1c27d2aeacf73fd41aa0ea3cda64e1c355fd"
2025-03-14T04:37:17.9711424Z 		},
2025-03-14T04:37:17.9711649Z 		{
2025-03-14T04:37:17.9712012Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9712478Z 			"size": 6570462764,
2025-03-14T04:37:17.9712951Z 			"digest": "sha256:20ff8019c66be2f3d8a36d3afbcc6853c80eb741555ffe291ea06e822e60d43d"
2025-03-14T04:37:17.9713473Z 		},
2025-03-14T04:37:17.9713690Z 		{
2025-03-14T04:37:17.9714050Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9714494Z 			"size": 176,
2025-03-14T04:37:17.9714948Z 			"digest": "sha256:7809aa0d99c049c0adf075d5be65adf6dafc83e9646868b365cf9201aa8854ab"
2025-03-14T04:37:17.9715466Z 		},
2025-03-14T04:37:17.9715685Z 		{
2025-03-14T04:37:17.9716047Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9716503Z 			"size": 907,
2025-03-14T04:37:17.9716956Z 			"digest": "sha256:204d74e677e106af47076fb10fb911dbf661bcbdfbe06a82a229f5dae97cf3ea"
2025-03-14T04:37:17.9717475Z 		},
2025-03-14T04:37:17.9717695Z 		{
2025-03-14T04:37:17.9718056Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9718515Z 			"size": 700,
2025-03-14T04:37:17.9718971Z 			"digest": "sha256:f895057e0aef06da056dc6e851e9341b646df75ceee25cecf482bd43ab954572"
2025-03-14T04:37:17.9719491Z 		},
2025-03-14T04:37:17.9719709Z 		{
2025-03-14T04:37:17.9720073Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9720529Z 			"size": 135,
2025-03-14T04:37:17.9720982Z 			"digest": "sha256:fe9fb5e11a5a8b9886e0476cdf20148e4cb54c946aeeb91e0cd57f56b9b09019"
2025-03-14T04:37:17.9721560Z 		},
2025-03-14T04:37:17.9721785Z 		{
2025-03-14T04:37:17.9722148Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9722611Z 			"size": 32,
2025-03-14T04:37:17.9723062Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2025-03-14T04:37:17.9723582Z 		},
2025-03-14T04:37:17.9723801Z 		{
2025-03-14T04:37:17.9724165Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9724625Z 			"size": 158,
2025-03-14T04:37:17.9725075Z 			"digest": "sha256:0184b2f1dce717abeda2212b23b56c5a5d2a88b5a9855d51362efb2e409dd3ae"
2025-03-14T04:37:17.9725594Z 		},
2025-03-14T04:37:17.9725814Z 		{
2025-03-14T04:37:17.9726177Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9726636Z 			"size": 1899,
2025-03-14T04:37:17.9727091Z 			"digest": "sha256:a6029d37d2e9316e55b2c15c1858049c3ba789bacbbfbfdec0b7bf3af3a82ae5"
2025-03-14T04:37:17.9727613Z 		},
2025-03-14T04:37:17.9727819Z 		{
2025-03-14T04:37:17.9728181Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9728642Z 			"size": 196364357,
2025-03-14T04:37:17.9729105Z 			"digest": "sha256:f47a1a28158bba9dc4361fe7db1bff78435a6a1dba13ec0c56974a1615630722"
2025-03-14T04:37:17.9729625Z 		},
2025-03-14T04:37:17.9729845Z 		{
2025-03-14T04:37:17.9730215Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9730673Z 			"size": 165,
2025-03-14T04:37:17.9731128Z 			"digest": "sha256:70cfac12558aabd8b776cc73ec814b7eebd746cfb49f586c05b635a7f8f51b88"
2025-03-14T04:37:17.9731650Z 		},
2025-03-14T04:37:17.9731874Z 		{
2025-03-14T04:37:17.9732306Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9732767Z 			"size": 7943,
2025-03-14T04:37:17.9733211Z 			"digest": "sha256:4c209765e93b40d0d78caa92579c0f06df20e463624ba915d0f76e5e77639552"
2025-03-14T04:37:17.9733727Z 		},
2025-03-14T04:37:17.9733950Z 		{
2025-03-14T04:37:17.9734299Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9734757Z 			"size": 8064,
2025-03-14T04:37:17.9735201Z 			"digest": "sha256:46344bfd438c7577210f690342612aab2b2ff8b466d02e41386fc59ed62ec82d"
2025-03-14T04:37:17.9735710Z 		},
2025-03-14T04:37:17.9735930Z 		{
2025-03-14T04:37:17.9736291Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9736749Z 			"size": 304,
2025-03-14T04:37:17.9737197Z 			"digest": "sha256:d6a4daeb47737eb84ee76c579d2044d59c295fab0d1a8285835e0fd121c62424"
2025-03-14T04:37:17.9737716Z 		},
2025-03-14T04:37:17.9737933Z 		{
2025-03-14T04:37:17.9738306Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9738764Z 			"size": 7631469,
2025-03-14T04:37:17.9739227Z 			"digest": "sha256:77ecae2ce8fe9f74618d83f8719ef44ed426419e9588df08a4bf7af9798b7e2d"
2025-03-14T04:37:17.9739750Z 		},
2025-03-14T04:37:17.9739975Z 		{
2025-03-14T04:37:17.9740347Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9740795Z 			"size": 108,
2025-03-14T04:37:17.9741248Z 			"digest": "sha256:3407ce9456e11a9d6cea36166aecb9118381b4f13ac20c065f3ad536499de54a"
2025-03-14T04:37:17.9741763Z 		},
2025-03-14T04:37:17.9741988Z 		{
2025-03-14T04:37:17.9742353Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9742818Z 			"size": 54145661,
2025-03-14T04:37:17.9743275Z 			"digest": "sha256:7d8b20e6946b7a50901524240c5b0a6659c269f0c0d59fc4a4f08cc0fc5a5a4e"
2025-03-14T04:37:17.9743795Z 		},
2025-03-14T04:37:17.9744019Z 		{
2025-03-14T04:37:17.9744390Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9744849Z 			"size": 496,
2025-03-14T04:37:17.9745295Z 			"digest": "sha256:d2ec17980733679708b02eacdec356f7e36b73b7288006739485a200b389cd6e"
2025-03-14T04:37:17.9745807Z 		},
2025-03-14T04:37:17.9746033Z 		{
2025-03-14T04:37:17.9746395Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9746912Z 			"size": 1374859215,
2025-03-14T04:37:17.9747376Z 			"digest": "sha256:f959ccb65669d6f58ea932a6f17c0f382887498f487537cfb47141befae1d9ee"
2025-03-14T04:37:17.9747876Z 		},
2025-03-14T04:37:17.9748097Z 		{
2025-03-14T04:37:17.9748466Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9748924Z 			"size": 106,
2025-03-14T04:37:17.9749360Z 			"digest": "sha256:07957f6422536491700ae51eb88bf4799482fbd32dfd5ba6117e2d4025306566"
2025-03-14T04:37:17.9749863Z 		},
2025-03-14T04:37:17.9750079Z 		{
2025-03-14T04:37:17.9750438Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9750900Z 			"size": 613,
2025-03-14T04:37:17.9751336Z 			"digest": "sha256:24c53c94b549936ca855400f66d4cf90f9788859051360d7a3c749a245022807"
2025-03-14T04:37:17.9751840Z 		},
2025-03-14T04:37:17.9752063Z 		{
2025-03-14T04:37:17.9752430Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9752985Z 			"size": 303706059,
2025-03-14T04:37:17.9753456Z 			"digest": "sha256:dcaf41ee394c3ccec7f22ee6027962df424bd45f3804b151ef756f2717fb89e0"
2025-03-14T04:37:17.9753980Z 		},
2025-03-14T04:37:17.9754191Z 		{
2025-03-14T04:37:17.9754554Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9755014Z 			"size": 111,
2025-03-14T04:37:17.9755455Z 			"digest": "sha256:df00249d5f5bf42269e64a543502b84eda618705e804245329b3fd4ef67c2e40"
2025-03-14T04:37:17.9755966Z 		},
2025-03-14T04:37:17.9756186Z 		{
2025-03-14T04:37:17.9756552Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9757013Z 			"size": 530,
2025-03-14T04:37:17.9757516Z 			"digest": "sha256:9446d934374a8e09cd41f1e4fe22410c9c5e68cc59809bf1018bc8f621eeda19"
2025-03-14T04:37:17.9758032Z 		},
2025-03-14T04:37:17.9758260Z 		{
2025-03-14T04:37:17.9758628Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9759098Z 			"size": 48289042,
2025-03-14T04:37:17.9759564Z 			"digest": "sha256:b450dfdf57ec026f7427498c1695ec9f973150331cfa2dc26132fba3047d0f20"
2025-03-14T04:37:17.9760081Z 		},
2025-03-14T04:37:17.9760303Z 		{
2025-03-14T04:37:17.9760653Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9761114Z 			"size": 106,
2025-03-14T04:37:17.9761558Z 			"digest": "sha256:7a4e6f20c0a757b2dcc156f6c7855606c1c248daf67c74e2716937dbb3195295"
2025-03-14T04:37:17.9762076Z 		},
2025-03-14T04:37:17.9762297Z 		{
2025-03-14T04:37:17.9762661Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9763122Z 			"size": 32,
2025-03-14T04:37:17.9763577Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2025-03-14T04:37:17.9764100Z 		},
2025-03-14T04:37:17.9764324Z 		{
2025-03-14T04:37:17.9764689Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9765146Z 			"size": 32,
2025-03-14T04:37:17.9765602Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2025-03-14T04:37:17.9766126Z 		},
2025-03-14T04:37:17.9766352Z 		{
2025-03-14T04:37:17.9766716Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9767176Z 			"size": 32,
2025-03-14T04:37:17.9767612Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2025-03-14T04:37:17.9768138Z 		},
2025-03-14T04:37:17.9768358Z 		{
2025-03-14T04:37:17.9768721Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:37:17.9769180Z 			"size": 32,
2025-03-14T04:37:17.9769634Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2025-03-14T04:37:17.9770157Z 		}
2025-03-14T04:37:17.9770381Z 	]
2025-03-14T04:37:17.9770600Z }
2025-03-14T04:37:17.9810329Z ##[group]Run set -eux
2025-03-14T04:37:17.9810664Z set -eux
2025-03-14T04:37:17.9811671Z aws secretsmanager get-secret-value --secret-id docker_hub_readonly_token | jq --raw-output '.SecretString' | jq -r .docker_hub_readonly_token | docker login --username pytorchbot --password-stdin
2025-03-14T04:37:17.9818615Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:37:17.9819071Z env:
2025-03-14T04:37:17.9819334Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:37:17.9819647Z ##[endgroup]
2025-03-14T04:37:17.9845807Z + aws secretsmanager get-secret-value --secret-id docker_hub_readonly_token
2025-03-14T04:37:17.9849214Z + jq --raw-output .SecretString
2025-03-14T04:37:17.9850287Z + jq -r .docker_hub_readonly_token
2025-03-14T04:37:17.9851480Z + docker login --username pytorchbot --password-stdin
2025-03-14T04:37:18.5343638Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json.
2025-03-14T04:37:18.5344649Z Configure a credential helper to remove this warning. See
2025-03-14T04:37:18.5345380Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store
2025-03-14T04:37:18.5345867Z 
2025-03-14T04:37:18.5346023Z Login Succeeded
2025-03-14T04:37:18.5445797Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/}
2025-03-14T04:37:18.5446240Z tag=${ECR_DOCKER_IMAGE##*/}
2025-03-14T04:37:18.5446689Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}"
2025-03-14T04:37:18.5452280Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:37:18.5452727Z env:
2025-03-14T04:37:18.5453015Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:37:18.5453994Z   ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:37:18.5455005Z ##[endgroup]
2025-03-14T04:37:18.5479089Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks-aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:37:18.5550279Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main
2025-03-14T04:37:18.5550805Z with:
2025-03-14T04:37:18.5551753Z   docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:37:18.5552882Z   docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-03-14T04:37:18.5553353Z env:
2025-03-14T04:37:18.5553638Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:37:18.5553974Z ##[endgroup]
2025-03-14T04:37:18.5578997Z ##[group]Run set -x
2025-03-14T04:37:18.5579332Z set -x
2025-03-14T04:37:18.5579629Z set +e
2025-03-14T04:37:18.5579917Z 
2025-03-14T04:37:18.5580187Z login() {
2025-03-14T04:37:18.5580740Z   aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1"
2025-03-14T04:37:18.5581336Z }
2025-03-14T04:37:18.5581594Z 
2025-03-14T04:37:18.5581926Z retry () {
2025-03-14T04:37:18.5582270Z   $*  || (sleep 1 && $*) || (sleep 2 && $*)
2025-03-14T04:37:18.5582670Z }
2025-03-14T04:37:18.5582939Z 
2025-03-14T04:37:18.5583232Z retry login "${DOCKER_REGISTRY}"
2025-03-14T04:37:18.5583611Z 
2025-03-14T04:37:18.5583855Z set -e
2025-03-14T04:37:18.5584240Z # ignore output since only exit code is used for conditional
2025-03-14T04:37:18.5584783Z # only pull docker image if it's not available locally
2025-03-14T04:37:18.5585383Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then
2025-03-14T04:37:18.5585937Z   retry docker pull "${DOCKER_IMAGE}"
2025-03-14T04:37:18.5586279Z fi
2025-03-14T04:37:18.5591053Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:37:18.5591499Z env:
2025-03-14T04:37:18.5591759Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:37:18.5592777Z   DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:37:18.5594068Z   DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-03-14T04:37:18.5594537Z ##[endgroup]
2025-03-14T04:37:18.5614335Z + set +e
2025-03-14T04:37:18.5614713Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-03-14T04:37:18.5615207Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-03-14T04:37:18.5617770Z + aws ecr get-login-password --region us-east-1
2025-03-14T04:37:18.5619091Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-03-14T04:37:19.0730070Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json.
2025-03-14T04:37:19.0730886Z Configure a credential helper to remove this warning. See
2025-03-14T04:37:19.0731626Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store
2025-03-14T04:37:19.0732165Z 
2025-03-14T04:37:19.0732307Z Login Succeeded
2025-03-14T04:37:19.0745531Z + set -e
2025-03-14T04:37:19.0746731Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:37:19.0882021Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:37:19.0883843Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:37:19.3249157Z aa89d6e739080d90fa18625d57297c6734465849: Pulling from pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks
2025-03-14T04:37:19.3251611Z 80888bc6716f: Pulling fs layer
2025-03-14T04:37:19.3252571Z fbcd35dc5bc3: Pulling fs layer
2025-03-14T04:37:19.3254016Z c7232af9ae05: Pulling fs layer
2025-03-14T04:37:19.3254952Z db6cdef1932a: Pulling fs layer
2025-03-14T04:37:19.3255834Z 56dc85502937: Pulling fs layer
2025-03-14T04:37:19.3256257Z 30c0ea6140d0: Pulling fs layer
2025-03-14T04:37:19.3256658Z 71bdb1a72c2d: Pulling fs layer
2025-03-14T04:37:19.3257048Z 4829486be7c3: Pulling fs layer
2025-03-14T04:37:19.3257445Z 1f4e68d7b5e4: Pulling fs layer
2025-03-14T04:37:19.3257856Z 7c373e2d9b7e: Pulling fs layer
2025-03-14T04:37:19.3258252Z 622381141745: Pulling fs layer
2025-03-14T04:37:19.3258640Z f190bb8f8a61: Pulling fs layer
2025-03-14T04:37:19.3259031Z 7e229290af34: Pulling fs layer
2025-03-14T04:37:19.3259432Z abf1e0eaf8f4: Pulling fs layer
2025-03-14T04:37:19.3259829Z db6cdef1932a: Waiting
2025-03-14T04:37:19.3260192Z 4545fec1915c: Pulling fs layer
2025-03-14T04:37:19.3260588Z 40ee82023c66: Pulling fs layer
2025-03-14T04:37:19.3260981Z 279be7c466cf: Pulling fs layer
2025-03-14T04:37:19.3261375Z b5cb697686fa: Pulling fs layer
2025-03-14T04:37:19.3261779Z 07ed5400009b: Pulling fs layer
2025-03-14T04:37:19.3262125Z 71bdb1a72c2d: Waiting
2025-03-14T04:37:19.3262489Z f895057e0aef: Pulling fs layer
2025-03-14T04:37:19.3262879Z 4829486be7c3: Waiting
2025-03-14T04:37:19.3263241Z 12b934c44e19: Pulling fs layer
2025-03-14T04:37:19.3263624Z 30c0ea6140d0: Waiting
2025-03-14T04:37:19.3263982Z 13e6ba8d75a8: Pulling fs layer
2025-03-14T04:37:19.3264353Z f190bb8f8a61: Waiting
2025-03-14T04:37:19.3264686Z 56dc85502937: Waiting
2025-03-14T04:37:19.3265035Z 0cddd2dd1bb9: Pulling fs layer
2025-03-14T04:37:19.3265415Z 19d8410fa3bc: Pulling fs layer
2025-03-14T04:37:19.3265790Z e305964abe7d: Pulling fs layer
2025-03-14T04:37:19.3266175Z 48470c6711d9: Pulling fs layer
2025-03-14T04:37:19.3266535Z 622381141745: Waiting
2025-03-14T04:37:19.3266867Z 7c373e2d9b7e: Waiting
2025-03-14T04:37:19.3267181Z 102ff554f680: Pulling fs layer
2025-03-14T04:37:19.3267552Z 4545fec1915c: Waiting
2025-03-14T04:37:19.3267888Z 40ee82023c66: Waiting
2025-03-14T04:37:19.3268231Z 862f9a6d0194: Pulling fs layer
2025-03-14T04:37:19.3268606Z abf1e0eaf8f4: Waiting
2025-03-14T04:37:19.3268955Z 13e6ba8d75a8: Waiting
2025-03-14T04:37:19.3269283Z b5cb697686fa: Waiting
2025-03-14T04:37:19.3269792Z 496d42fc1121: Pulling fs layer
2025-03-14T04:37:19.3270168Z e305964abe7d: Waiting
2025-03-14T04:37:19.3270545Z 07ed5400009b: Waiting
2025-03-14T04:37:19.3270899Z 766900d8eb9c: Pulling fs layer
2025-03-14T04:37:19.3271282Z 0cddd2dd1bb9: Waiting
2025-03-14T04:37:19.3271629Z 19d8410fa3bc: Waiting
2025-03-14T04:37:19.3271994Z 2bf5096e9bc9: Pulling fs layer
2025-03-14T04:37:19.3272382Z 17fc7d1c8735: Pulling fs layer
2025-03-14T04:37:19.3272766Z 73b57ddfc7ff: Pulling fs layer
2025-03-14T04:37:19.3273151Z 246047bfca40: Pulling fs layer
2025-03-14T04:37:19.3273535Z 5314adad3d92: Pulling fs layer
2025-03-14T04:37:19.3273906Z 279be7c466cf: Waiting
2025-03-14T04:37:19.3274223Z b095bb710082: Pulling fs layer
2025-03-14T04:37:19.3274599Z a9b21d6249e6: Pulling fs layer
2025-03-14T04:37:19.3274965Z 2bf5096e9bc9: Waiting
2025-03-14T04:37:19.3275363Z 32b89bc8b340: Pulling fs layer
2025-03-14T04:37:19.3275729Z 862f9a6d0194: Waiting
2025-03-14T04:37:19.3276059Z 17fc7d1c8735: Waiting
2025-03-14T04:37:19.3276416Z 0f2b30b2c08b: Pulling fs layer
2025-03-14T04:37:19.3276884Z 48470c6711d9: Waiting
2025-03-14T04:37:19.3277266Z 7082c2fbfc18: Pulling fs layer
2025-03-14T04:37:19.3277734Z 6c73e7aaf542: Pulling fs layer
2025-03-14T04:37:19.3278205Z a9b21d6249e6: Waiting
2025-03-14T04:37:19.3278645Z 32b89bc8b340: Waiting
2025-03-14T04:37:19.3279059Z f78c26edc26c: Pulling fs layer
2025-03-14T04:37:19.3279443Z 246047bfca40: Waiting
2025-03-14T04:37:19.3279771Z 102ff554f680: Waiting
2025-03-14T04:37:19.3280103Z 496d42fc1121: Waiting
2025-03-14T04:37:19.3280457Z dcd44da9f83c: Pulling fs layer
2025-03-14T04:37:19.3280881Z 5314adad3d92: Waiting
2025-03-14T04:37:19.3281354Z 6379c2fe5672: Pulling fs layer
2025-03-14T04:37:19.3281849Z b88092e39da5: Pulling fs layer
2025-03-14T04:37:19.3282219Z 73b57ddfc7ff: Waiting
2025-03-14T04:37:19.3282560Z 0f2b30b2c08b: Waiting
2025-03-14T04:37:19.3283012Z 4ee03c8b3be2: Pulling fs layer
2025-03-14T04:37:19.3283388Z 7082c2fbfc18: Waiting
2025-03-14T04:37:19.3283736Z f82587c7a86d: Pulling fs layer
2025-03-14T04:37:19.3284132Z 15ec2b791b0b: Pulling fs layer
2025-03-14T04:37:19.3284588Z b095bb710082: Waiting
2025-03-14T04:37:19.3284976Z 4ee03c8b3be2: Waiting
2025-03-14T04:37:19.3285391Z 7b9e114c1cb7: Pulling fs layer
2025-03-14T04:37:19.3285852Z 6379c2fe5672: Waiting
2025-03-14T04:37:19.3286306Z f82587c7a86d: Waiting
2025-03-14T04:37:19.3286649Z 7b9e114c1cb7: Waiting
2025-03-14T04:37:19.3286983Z dcd44da9f83c: Waiting
2025-03-14T04:37:19.3287336Z 429a8d451044: Pulling fs layer
2025-03-14T04:37:19.3287724Z 44156619fef1: Pulling fs layer
2025-03-14T04:37:19.3288110Z 15ec2b791b0b: Waiting
2025-03-14T04:37:19.3288478Z 20ff8019c66b: Pulling fs layer
2025-03-14T04:37:19.3288878Z 7809aa0d99c0: Pulling fs layer
2025-03-14T04:37:19.3289270Z 204d74e677e1: Pulling fs layer
2025-03-14T04:37:19.3289643Z b88092e39da5: Waiting
2025-03-14T04:37:19.3290132Z fe9fb5e11a5a: Pulling fs layer
2025-03-14T04:37:19.3290515Z 7809aa0d99c0: Waiting
2025-03-14T04:37:19.3290868Z 4f4fb700ef54: Pulling fs layer
2025-03-14T04:37:19.3291275Z 0184b2f1dce7: Pulling fs layer
2025-03-14T04:37:19.3291661Z a6029d37d2e9: Pulling fs layer
2025-03-14T04:37:19.3292038Z 20ff8019c66b: Waiting
2025-03-14T04:37:19.3292373Z 0184b2f1dce7: Waiting
2025-03-14T04:37:19.3292725Z f47a1a28158b: Pulling fs layer
2025-03-14T04:37:19.3293100Z 4f4fb700ef54: Waiting
2025-03-14T04:37:19.3293454Z 70cfac12558a: Pulling fs layer
2025-03-14T04:37:19.3293838Z 4c209765e93b: Pulling fs layer
2025-03-14T04:37:19.3294213Z a6029d37d2e9: Waiting
2025-03-14T04:37:19.3294559Z 46344bfd438c: Pulling fs layer
2025-03-14T04:37:19.3294950Z d6a4daeb4773: Pulling fs layer
2025-03-14T04:37:19.3295333Z 77ecae2ce8fe: Pulling fs layer
2025-03-14T04:37:19.3295679Z f47a1a28158b: Waiting
2025-03-14T04:37:19.3296012Z 44156619fef1: Waiting
2025-03-14T04:37:19.3296336Z 70cfac12558a: Waiting
2025-03-14T04:37:19.3296688Z 3407ce9456e1: Pulling fs layer
2025-03-14T04:37:19.3297068Z 7d8b20e6946b: Pulling fs layer
2025-03-14T04:37:19.3297434Z d6a4daeb4773: Waiting
2025-03-14T04:37:19.3297894Z 3407ce9456e1: Waiting
2025-03-14T04:37:19.3298240Z d2ec17980733: Pulling fs layer
2025-03-14T04:37:19.3298620Z f959ccb65669: Pulling fs layer
2025-03-14T04:37:19.3298988Z 7d8b20e6946b: Waiting
2025-03-14T04:37:19.3299333Z 07957f642253: Pulling fs layer
2025-03-14T04:37:19.3299704Z fe9fb5e11a5a: Waiting
2025-03-14T04:37:19.3300038Z d2ec17980733: Waiting
2025-03-14T04:37:19.3300370Z f959ccb65669: Waiting
2025-03-14T04:37:19.3300715Z 24c53c94b549: Pulling fs layer
2025-03-14T04:37:19.3301055Z 46344bfd438c: Waiting
2025-03-14T04:37:19.3301406Z dcaf41ee394c: Pulling fs layer
2025-03-14T04:37:19.3301788Z 24c53c94b549: Waiting
2025-03-14T04:37:19.3302139Z df00249d5f5b: Pulling fs layer
2025-03-14T04:37:19.3302523Z dcaf41ee394c: Waiting
2025-03-14T04:37:19.3302871Z 9446d934374a: Pulling fs layer
2025-03-14T04:37:19.3303235Z 07957f642253: Waiting
2025-03-14T04:37:19.3303585Z b450dfdf57ec: Pulling fs layer
2025-03-14T04:37:19.3303961Z df00249d5f5b: Waiting
2025-03-14T04:37:19.3304291Z 9446d934374a: Waiting
2025-03-14T04:37:19.3304640Z 7a4e6f20c0a7: Pulling fs layer
2025-03-14T04:37:19.3305006Z 204d74e677e1: Waiting
2025-03-14T04:37:19.3305337Z b450dfdf57ec: Waiting
2025-03-14T04:37:19.3305667Z 7a4e6f20c0a7: Waiting
2025-03-14T04:37:19.7094369Z fbcd35dc5bc3: Verifying Checksum
2025-03-14T04:37:19.7094886Z fbcd35dc5bc3: Download complete
2025-03-14T04:37:19.8052049Z db6cdef1932a: Download complete
2025-03-14T04:37:19.9406613Z 56dc85502937: Download complete
2025-03-14T04:37:20.0279688Z 80888bc6716f: Verifying Checksum
2025-03-14T04:37:20.0280590Z 80888bc6716f: Download complete
2025-03-14T04:37:20.1179840Z 71bdb1a72c2d: Download complete
2025-03-14T04:37:20.2138591Z 4829486be7c3: Verifying Checksum
2025-03-14T04:37:20.2139052Z 4829486be7c3: Download complete
2025-03-14T04:37:20.3114330Z 1f4e68d7b5e4: Verifying Checksum
2025-03-14T04:37:20.3115130Z 1f4e68d7b5e4: Download complete
2025-03-14T04:37:20.6335059Z c7232af9ae05: Download complete
2025-03-14T04:37:20.7569281Z 80888bc6716f: Pull complete
2025-03-14T04:37:20.8118960Z 622381141745: Download complete
2025-03-14T04:37:20.8886531Z f190bb8f8a61: Verifying Checksum
2025-03-14T04:37:20.8887381Z f190bb8f8a61: Download complete
2025-03-14T04:37:20.9764099Z fbcd35dc5bc3: Pull complete
2025-03-14T04:37:21.7282278Z c7232af9ae05: Pull complete
2025-03-14T04:37:21.7511842Z db6cdef1932a: Pull complete
2025-03-14T04:37:21.7730099Z 56dc85502937: Pull complete
2025-03-14T04:37:33.1076889Z 7e229290af34: Verifying Checksum
2025-03-14T04:37:33.1077944Z 7e229290af34: Download complete
2025-03-14T04:37:33.1929384Z abf1e0eaf8f4: Verifying Checksum
2025-03-14T04:37:33.1930131Z abf1e0eaf8f4: Download complete
2025-03-14T04:37:33.2884396Z 4545fec1915c: Verifying Checksum
2025-03-14T04:37:33.2884869Z 4545fec1915c: Download complete
2025-03-14T04:37:33.3851781Z 40ee82023c66: Download complete
2025-03-14T04:37:35.3573415Z 279be7c466cf: Verifying Checksum
2025-03-14T04:37:35.3573876Z 279be7c466cf: Download complete
2025-03-14T04:37:35.4585802Z b5cb697686fa: Verifying Checksum
2025-03-14T04:37:35.4586502Z b5cb697686fa: Download complete
2025-03-14T04:37:35.5770275Z 07ed5400009b: Verifying Checksum
2025-03-14T04:37:35.5771340Z 07ed5400009b: Download complete
2025-03-14T04:37:35.7001987Z f895057e0aef: Download complete
2025-03-14T04:37:35.8347135Z 12b934c44e19: Download complete
2025-03-14T04:37:45.8103186Z 30c0ea6140d0: Verifying Checksum
2025-03-14T04:37:45.8104253Z 30c0ea6140d0: Download complete
2025-03-14T04:37:45.9524426Z 0cddd2dd1bb9: Download complete
2025-03-14T04:37:46.1338572Z 19d8410fa3bc: Download complete
2025-03-14T04:37:46.2174938Z e305964abe7d: Download complete
2025-03-14T04:37:46.3177536Z 48470c6711d9: Download complete
2025-03-14T04:37:46.4521140Z 102ff554f680: Verifying Checksum
2025-03-14T04:37:46.4521647Z 102ff554f680: Download complete
2025-03-14T04:37:48.5787745Z 862f9a6d0194: Verifying Checksum
2025-03-14T04:37:48.5788332Z 862f9a6d0194: Download complete
2025-03-14T04:37:48.6959305Z 496d42fc1121: Verifying Checksum
2025-03-14T04:37:48.6959785Z 496d42fc1121: Download complete
2025-03-14T04:37:48.7864933Z 766900d8eb9c: Verifying Checksum
2025-03-14T04:37:48.7865996Z 766900d8eb9c: Download complete
2025-03-14T04:37:48.8889439Z 2bf5096e9bc9: Verifying Checksum
2025-03-14T04:37:48.8890038Z 2bf5096e9bc9: Download complete
2025-03-14T04:37:48.9898725Z 17fc7d1c8735: Download complete
2025-03-14T04:37:49.1006551Z 73b57ddfc7ff: Verifying Checksum
2025-03-14T04:37:49.1007593Z 73b57ddfc7ff: Download complete
2025-03-14T04:37:56.4665655Z 246047bfca40: Verifying Checksum
2025-03-14T04:37:56.4666181Z 246047bfca40: Download complete
2025-03-14T04:37:56.5795775Z 5314adad3d92: Verifying Checksum
2025-03-14T04:37:56.5796252Z 5314adad3d92: Download complete
2025-03-14T04:37:56.6699483Z b095bb710082: Verifying Checksum
2025-03-14T04:37:56.6699942Z b095bb710082: Download complete
2025-03-14T04:37:57.2773557Z a9b21d6249e6: Verifying Checksum
2025-03-14T04:37:57.2774630Z a9b21d6249e6: Download complete
2025-03-14T04:37:57.3857486Z 32b89bc8b340: Download complete
2025-03-14T04:37:57.5155611Z 0f2b30b2c08b: Verifying Checksum
2025-03-14T04:37:57.5156725Z 0f2b30b2c08b: Download complete
2025-03-14T04:37:57.8529850Z 7082c2fbfc18: Verifying Checksum
2025-03-14T04:37:57.8530866Z 7082c2fbfc18: Download complete
2025-03-14T04:37:57.9836992Z 6c73e7aaf542: Download complete
2025-03-14T04:37:58.0780503Z f78c26edc26c: Download complete
2025-03-14T04:37:58.1704960Z dcd44da9f83c: Download complete
2025-03-14T04:37:58.2176135Z 30c0ea6140d0: Pull complete
2025-03-14T04:37:58.5654604Z 71bdb1a72c2d: Pull complete
2025-03-14T04:37:58.9356553Z 4829486be7c3: Pull complete
2025-03-14T04:37:59.3677396Z 1f4e68d7b5e4: Pull complete
2025-03-14T04:38:12.1687086Z 7c373e2d9b7e: Verifying Checksum
2025-03-14T04:38:12.1687650Z 7c373e2d9b7e: Download complete
2025-03-14T04:38:12.2832087Z b88092e39da5: Verifying Checksum
2025-03-14T04:38:12.2832575Z b88092e39da5: Download complete
2025-03-14T04:38:12.3723052Z 4ee03c8b3be2: Verifying Checksum
2025-03-14T04:38:12.3723620Z 4ee03c8b3be2: Download complete
2025-03-14T04:38:13.3534112Z f82587c7a86d: Verifying Checksum
2025-03-14T04:38:13.3535267Z f82587c7a86d: Download complete
2025-03-14T04:38:13.5176653Z 15ec2b791b0b: Verifying Checksum
2025-03-14T04:38:13.6233665Z 15ec2b791b0b: Download complete
2025-03-14T04:38:13.6234149Z 7b9e114c1cb7: Verifying Checksum
2025-03-14T04:38:13.6234568Z 7b9e114c1cb7: Download complete
2025-03-14T04:38:13.7156991Z 429a8d451044: Verifying Checksum
2025-03-14T04:38:13.7157442Z 429a8d451044: Download complete
2025-03-14T04:38:13.8294294Z 44156619fef1: Download complete
2025-03-14T04:38:28.5968973Z 13e6ba8d75a8: Verifying Checksum
2025-03-14T04:38:28.5969491Z 13e6ba8d75a8: Download complete
2025-03-14T04:38:28.6860132Z 7809aa0d99c0: Verifying Checksum
2025-03-14T04:38:28.6861193Z 7809aa0d99c0: Download complete
2025-03-14T04:38:28.8302661Z 204d74e677e1: Verifying Checksum
2025-03-14T04:38:28.8303792Z 204d74e677e1: Download complete
2025-03-14T04:38:28.9649253Z fe9fb5e11a5a: Verifying Checksum
2025-03-14T04:38:28.9650306Z fe9fb5e11a5a: Download complete
2025-03-14T04:38:28.9708721Z 4f4fb700ef54: Verifying Checksum
2025-03-14T04:38:28.9709197Z 4f4fb700ef54: Download complete
2025-03-14T04:38:29.1012937Z 0184b2f1dce7: Verifying Checksum
2025-03-14T04:38:29.1014013Z 0184b2f1dce7: Download complete
2025-03-14T04:38:29.2611464Z a6029d37d2e9: Download complete
2025-03-14T04:38:32.9143118Z f47a1a28158b: Verifying Checksum
2025-03-14T04:38:32.9143643Z f47a1a28158b: Download complete
2025-03-14T04:38:32.9917587Z 70cfac12558a: Verifying Checksum
2025-03-14T04:38:32.9918705Z 70cfac12558a: Download complete
2025-03-14T04:38:33.0939674Z 4c209765e93b: Verifying Checksum
2025-03-14T04:38:33.0940740Z 4c209765e93b: Download complete
2025-03-14T04:38:33.2184366Z 46344bfd438c: Download complete
2025-03-14T04:38:33.3265870Z d6a4daeb4773: Download complete
2025-03-14T04:38:33.6871770Z 77ecae2ce8fe: Verifying Checksum
2025-03-14T04:38:33.6872325Z 77ecae2ce8fe: Download complete
2025-03-14T04:38:33.8196659Z 3407ce9456e1: Verifying Checksum
2025-03-14T04:38:33.8197723Z 3407ce9456e1: Download complete
2025-03-14T04:38:34.9597300Z 7d8b20e6946b: Verifying Checksum
2025-03-14T04:38:34.9598369Z 7d8b20e6946b: Download complete
2025-03-14T04:38:35.0985298Z d2ec17980733: Verifying Checksum
2025-03-14T04:38:35.0985933Z d2ec17980733: Download complete
2025-03-14T04:38:39.7950321Z 7c373e2d9b7e: Pull complete
2025-03-14T04:38:40.2493180Z 622381141745: Pull complete
2025-03-14T04:38:40.6997490Z f190bb8f8a61: Pull complete
2025-03-14T04:38:49.5854962Z 7e229290af34: Pull complete
2025-03-14T04:38:50.0094001Z abf1e0eaf8f4: Pull complete
2025-03-14T04:38:50.4452982Z 4545fec1915c: Pull complete
2025-03-14T04:38:50.8663752Z 40ee82023c66: Pull complete
2025-03-14T04:38:53.1835000Z 279be7c466cf: Pull complete
2025-03-14T04:38:53.5890136Z b5cb697686fa: Pull complete
2025-03-14T04:38:54.0314043Z 07ed5400009b: Pull complete
2025-03-14T04:38:54.4297148Z f895057e0aef: Pull complete
2025-03-14T04:38:54.8737629Z 12b934c44e19: Pull complete
2025-03-14T04:39:00.2819196Z f959ccb65669: Verifying Checksum
2025-03-14T04:39:00.2819707Z f959ccb65669: Download complete
2025-03-14T04:39:00.4285881Z 07957f642253: Verifying Checksum
2025-03-14T04:39:00.4286469Z 07957f642253: Download complete
2025-03-14T04:39:00.5206306Z 24c53c94b549: Verifying Checksum
2025-03-14T04:39:00.5206882Z 24c53c94b549: Download complete
2025-03-14T04:39:06.4640612Z dcaf41ee394c: Verifying Checksum
2025-03-14T04:39:06.4641130Z dcaf41ee394c: Download complete
2025-03-14T04:39:06.5685616Z df00249d5f5b: Verifying Checksum
2025-03-14T04:39:06.5686683Z df00249d5f5b: Download complete
2025-03-14T04:39:06.6722269Z 9446d934374a: Verifying Checksum
2025-03-14T04:39:06.6722716Z 9446d934374a: Download complete
2025-03-14T04:39:07.8449143Z b450dfdf57ec: Verifying Checksum
2025-03-14T04:39:07.8450173Z b450dfdf57ec: Download complete
2025-03-14T04:39:07.9444591Z 7a4e6f20c0a7: Verifying Checksum
2025-03-14T04:39:07.9446211Z 7a4e6f20c0a7: Download complete
2025-03-14T04:39:32.5524017Z 6379c2fe5672: Verifying Checksum
2025-03-14T04:39:32.5526531Z 6379c2fe5672: Download complete
2025-03-14T04:39:35.9394000Z 13e6ba8d75a8: Pull complete
2025-03-14T04:39:36.4122009Z 0cddd2dd1bb9: Pull complete
2025-03-14T04:39:36.9262975Z 19d8410fa3bc: Pull complete
2025-03-14T04:39:37.4070689Z e305964abe7d: Pull complete
2025-03-14T04:39:37.8984387Z 48470c6711d9: Pull complete
2025-03-14T04:39:38.3790893Z 102ff554f680: Pull complete
2025-03-14T04:39:41.2110116Z 862f9a6d0194: Pull complete
2025-03-14T04:39:41.3358629Z 496d42fc1121: Pull complete
2025-03-14T04:39:41.6845793Z 766900d8eb9c: Pull complete
2025-03-14T04:39:42.1904381Z 2bf5096e9bc9: Pull complete
2025-03-14T04:39:42.6569802Z 17fc7d1c8735: Pull complete
2025-03-14T04:39:43.1237556Z 73b57ddfc7ff: Pull complete
2025-03-14T04:39:49.1117366Z 246047bfca40: Pull complete
2025-03-14T04:39:49.1372164Z 5314adad3d92: Pull complete
2025-03-14T04:39:49.3014044Z b095bb710082: Pull complete
2025-03-14T04:39:50.0357216Z a9b21d6249e6: Pull complete
2025-03-14T04:39:50.0569214Z 32b89bc8b340: Pull complete
2025-03-14T04:39:50.0803299Z 0f2b30b2c08b: Pull complete
2025-03-14T04:39:50.2976909Z 7082c2fbfc18: Pull complete
2025-03-14T04:39:50.3221794Z 6c73e7aaf542: Pull complete
2025-03-14T04:39:50.3719994Z f78c26edc26c: Pull complete
2025-03-14T04:39:50.3946272Z dcd44da9f83c: Pull complete
2025-03-14T04:40:16.6668958Z 20ff8019c66b: Verifying Checksum
2025-03-14T04:40:16.6670054Z 20ff8019c66b: Download complete
2025-03-14T04:40:23.7472079Z 6379c2fe5672: Pull complete
2025-03-14T04:40:24.2076458Z b88092e39da5: Pull complete
2025-03-14T04:40:24.6365646Z 4ee03c8b3be2: Pull complete
2025-03-14T04:40:26.6896712Z f82587c7a86d: Pull complete
2025-03-14T04:40:27.1731797Z 15ec2b791b0b: Pull complete
2025-03-14T04:40:27.6861740Z 7b9e114c1cb7: Pull complete
2025-03-14T04:40:28.4994433Z 429a8d451044: Pull complete
2025-03-14T04:40:28.8946702Z 44156619fef1: Pull complete
2025-03-14T04:42:06.0803333Z 20ff8019c66b: Pull complete
2025-03-14T04:42:06.5160640Z 7809aa0d99c0: Pull complete
2025-03-14T04:42:06.9190435Z 204d74e677e1: Pull complete
2025-03-14T04:42:07.7725635Z fe9fb5e11a5a: Pull complete
2025-03-14T04:42:08.2452990Z 4f4fb700ef54: Pull complete
2025-03-14T04:42:08.6943550Z 0184b2f1dce7: Pull complete
2025-03-14T04:42:09.1514014Z a6029d37d2e9: Pull complete
2025-03-14T04:42:14.7695733Z f47a1a28158b: Pull complete
2025-03-14T04:42:15.1925446Z 70cfac12558a: Pull complete
2025-03-14T04:42:15.6378470Z 4c209765e93b: Pull complete
2025-03-14T04:42:16.0887604Z 46344bfd438c: Pull complete
2025-03-14T04:42:16.5246107Z d6a4daeb4773: Pull complete
2025-03-14T04:42:17.7468773Z 77ecae2ce8fe: Pull complete
2025-03-14T04:42:18.2279846Z 3407ce9456e1: Pull complete
2025-03-14T04:42:20.3408650Z 7d8b20e6946b: Pull complete
2025-03-14T04:42:20.8129653Z d2ec17980733: Pull complete
2025-03-14T04:42:33.3032271Z f959ccb65669: Pull complete
2025-03-14T04:42:33.7796765Z 07957f642253: Pull complete
2025-03-14T04:42:34.2294365Z 24c53c94b549: Pull complete
2025-03-14T04:42:37.2286830Z dcaf41ee394c: Pull complete
2025-03-14T04:42:37.6318083Z df00249d5f5b: Pull complete
2025-03-14T04:42:38.0838109Z 9446d934374a: Pull complete
2025-03-14T04:42:38.9503840Z b450dfdf57ec: Pull complete
2025-03-14T04:42:39.4438826Z 7a4e6f20c0a7: Pull complete
2025-03-14T04:42:41.5531493Z Digest: sha256:679ab340c4f18c247440dc36ea7b4cea489f03bffd640ad0dbf06f3e027a1eaf
2025-03-14T04:42:41.6253577Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:42:41.6635522Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:42:41.6696636Z ##[group]Run echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT"
2025-03-14T04:42:41.6697731Z echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT"
2025-03-14T04:42:41.6705928Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:42:41.6706427Z env:
2025-03-14T04:42:41.6706729Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:42:41.6707105Z ##[endgroup]
2025-03-14T04:42:41.6962251Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main
2025-03-14T04:42:41.6962803Z with:
2025-03-14T04:42:41.6963132Z   driver-version: 550.54.15
2025-03-14T04:42:41.6963512Z env:
2025-03-14T04:42:41.6963837Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:42:41.6964212Z ##[endgroup]
2025-03-14T04:42:41.7161823Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482
2025-03-14T04:42:41.7162388Z with:
2025-03-14T04:42:41.7162697Z   timeout_minutes: 10
2025-03-14T04:42:41.7163063Z   max_attempts: 3
2025-03-14T04:42:41.7192309Z   command: # Is it disgusting to have a full shell script here in this github action? Sure
# But is it the best way to make it so that this action relies on nothing else? Absolutely
set -eou pipefail

DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID)
DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run"

install_nvidia_docker2_amzn2() {
    (
        set -x
        # Needed for yum-config-manager
        sudo yum install -y yum-utils
        if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then
          YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo"
        else
          # Amazon Linux 2
          YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo"
        fi

        sudo yum-config-manager --add-repo "${YUM_REPO_URL}"
        sudo yum install -y nvidia-docker2 nvidia-container-toolkit-1.16.2
        sudo systemctl restart docker
    )
}

install_nvidia_docker2_ubuntu20() {
    (
        set -x
        # Install nvidia-driver package if not installed
        status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)"
        if [ ! $? = 0 ] || [ ! "$status" = installed ]; then
          sudo apt-get install -y nvidia-docker2 nvidia-container-toolkit-1.16.2
          sudo systemctl restart docker
        fi
    )
}

pre_install_nvidia_driver_amzn2() {
    (
        # Purge any nvidia driver installed from RHEL repo
        sudo yum remove -y nvidia-driver-latest-dkms
    )
}

install_nvidia_driver_common() {
    (
        # Try to gather more information about the runner and its existing NVIDIA driver if any
        echo "Before installing NVIDIA driver"
        lspci
        lsmod
        modinfo nvidia || true

        HAS_NVIDIA_DRIVER=0
        # Check if NVIDIA driver has already been installed
        if [ -x "$(command -v nvidia-smi)" ]; then
            set +e
            # The driver exists, check its version next. Also check only the first GPU if there are more than one of them
            # so that the same driver version is not print over multiple lines
            INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0)
            NVIDIA_SMI_STATUS=$?

            if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then
                echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing"
            elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then
                echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing"
            else
                HAS_NVIDIA_DRIVER=1
                echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation"
            fi
            set -e
        fi

        if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then
            # CAUTION: this may need to be updated in future
            if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then
                  sudo yum groupinstall -y "Development Tools"
                  # ensure our kernel install is the same as our underlying kernel,
                  # groupinstall "Development Tools" has a habit of mismatching kernel headers
                  sudo yum install -y "kernel-devel-uname-r == $(uname -r)"
                  sudo modprobe backlight
            fi
            sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN"

            set +e
            sudo /bin/bash /tmp/nvidia_driver -s --no-drm
            NVIDIA_INSTALLATION_STATUS=$?

            RESET_GPU=0
            if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then
                sudo cat /var/log/nvidia-installer.log
                # Fail to install NVIDIA driver, try to reset the GPU
                RESET_GPU=1
            elif [ -x "$(command -v nvidia-smi)" ]; then
                # Check again if nvidia-smi works even if the driver installation completes successfully
                INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0)
                NVIDIA_SMI_STATUS=$?

                if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then
                    RESET_GPU=1
                fi
            fi

            if [ "$RESET_GPU" -eq 1 ]; then
                NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1)
                # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this
                # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388
                for PCI_ID in $NVIDIA_DEVICES; do
                    DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable)

                    echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)"
                    # This requires sudo permission of course
                    echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset
                    sleep 1
                done
            fi

            sudo rm -fv /tmp/nvidia_driver
            set -e
        fi
    )
}

post_install_nvidia_driver_common() {
    (
        sudo modprobe nvidia || true
        echo "After installing NVIDIA driver"
        lspci
        lsmod
        modinfo nvidia || true

        (
            set +e

            nvidia-smi
            # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in
            # the case where the driver has already crashed as it still can get the driver version
            # and some basic information like the bus ID.  However, the rest of the information
            # would be missing (ERR!), for example:
            #
            # +-----------------------------------------------------------------------------+
            # | NVIDIA-SMI 525.89.02    Driver Version: 525.89.02    CUDA Version: 12.0     |
            # |-------------------------------+----------------------+----------------------+
            # | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
            # | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
            # |                               |                      |               MIG M. |
            # |===============================+======================+======================|
            # |   0  ERR!                Off  | 00000000:00:1E.0 Off |                 ERR! |
            # |ERR!  ERR! ERR!    ERR! / ERR! |   4184MiB / 23028MiB |    ERR!      Default |
            # |                               |                      |                 ERR! |
            # +-------------------------------+----------------------+----------------------+
            #
            # +-----------------------------------------------------------------------------+
            # | Processes:                                                                  |
            # |  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
            # |        ID   ID                                                   Usage      |
            # |=============================================================================|
            # +-----------------------------------------------------------------------------+
            #
            # This should be reported as a failure instead as it will guarantee to fail when
            # Docker tries to run with --gpus all
            #
            # So, the correct check here is to query one of the missing piece of info like
            # GPU name, so that the command can fail accordingly
            nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0
            NVIDIA_SMI_STATUS=$?

            # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285
            if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then
                echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}"
            else
                echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}"
                exit ${NVIDIA_SMI_STATUS}
            fi
            set -e
        )
    )
}

install_nvidia_driver_amzn2() {
    (
        set -x
        pre_install_nvidia_driver_amzn2
        install_nvidia_driver_common
        post_install_nvidia_driver_common
    )
}

install_nvidia_driver_ubuntu20() {
    (
        set -x
        install_nvidia_driver_common
        post_install_nvidia_driver_common
    )
}

echo "== Installing nvidia driver ${DRIVER_FN} =="
case "${DISTRIBUTION}" in
    amzn*)
        install_nvidia_driver_amzn2
        ;;
    ubuntu20.04)
        install_nvidia_driver_ubuntu20
        ;;
    *)
        echo "ERROR: Unknown distribution ${DISTRIBUTION}"
        exit 1
        ;;
esac

# Install container toolkit based on distribution
echo "== Installing nvidia container toolkit for ${DISTRIBUTION} =="
case "${DISTRIBUTION}" in
    amzn*)
        install_nvidia_docker2_amzn2
        ;;
    ubuntu20.04)
        install_nvidia_docker2_ubuntu20
        ;;
    *)
        echo "ERROR: Unknown distribution ${DISTRIBUTION}"
        exit 1
        ;;
esac
echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}"

# Fix https://github.com/NVIDIA/nvidia-docker/issues/1648 on runners with
# more than one GPUs. This just needs to be run once. The command fails
# on subsequent runs and complains that the mode is already on, but that's
# ok
sudo nvidia-persistenced || true
# This should show persistence mode ON
nvidia-smi

2025-03-14T04:42:41.7222294Z   retry_wait_seconds: 10
2025-03-14T04:42:41.7222653Z   polling_interval_seconds: 1
2025-03-14T04:42:41.7222984Z   warning_on_retry: true
2025-03-14T04:42:41.7223295Z   continue_on_error: false
2025-03-14T04:42:41.7223614Z env:
2025-03-14T04:42:41.7223892Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:42:41.7224218Z   DRIVER_VERSION: 550.54.15
2025-03-14T04:42:41.7224558Z ##[endgroup]
2025-03-14T04:42:41.8720978Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run ==
2025-03-14T04:42:41.8723006Z + pre_install_nvidia_driver_amzn2
2025-03-14T04:42:41.8729428Z + sudo yum remove -y nvidia-driver-latest-dkms
2025-03-14T04:42:42.1637302Z No match for argument: nvidia-driver-latest-dkms
2025-03-14T04:42:42.1637823Z No packages marked for removal.
2025-03-14T04:42:42.1698004Z Dependencies resolved.
2025-03-14T04:42:42.1706593Z Nothing to do.
2025-03-14T04:42:42.1707231Z Complete!
2025-03-14T04:42:42.3097883Z + install_nvidia_driver_common
2025-03-14T04:42:42.3100911Z + echo 'Before installing NVIDIA driver'
2025-03-14T04:42:42.3101319Z + lspci
2025-03-14T04:42:42.3102278Z Before installing NVIDIA driver
2025-03-14T04:42:42.3341938Z 00:00.0 Host bridge: Intel Corporation Sky Lake-E DMI3 Registers (rev 07)
2025-03-14T04:42:42.3342732Z 00:04.0 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.3343496Z 00:04.1 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.3344243Z 00:04.2 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.3344986Z 00:04.3 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.3345728Z 00:04.4 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.3346493Z 00:04.5 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.3347492Z 00:04.6 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.3348251Z 00:04.7 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.3349074Z 00:05.0 System peripheral: Intel Corporation Sky Lake-E MM/Vt-d Configuration Registers (rev 07)
2025-03-14T04:42:42.3349879Z 00:05.2 System peripheral: Intel Corporation Sky Lake-E RAS (rev 07)
2025-03-14T04:42:42.3350451Z 00:05.4 PIC: Intel Corporation Sky Lake-E IOAPIC (rev 07)
2025-03-14T04:42:42.3351088Z 00:08.0 System peripheral: Intel Corporation Sky Lake-E Ubox Registers (rev 07)
2025-03-14T04:42:42.3351847Z 00:08.1 Performance counters: Intel Corporation Sky Lake-E Ubox Registers (rev 07)
2025-03-14T04:42:42.3352603Z 00:08.2 System peripheral: Intel Corporation Sky Lake-E Ubox Registers (rev 07)
2025-03-14T04:42:42.3353389Z 00:11.0 Unassigned class [ff00]: Intel Corporation C620 Series Chipset Family MROM 0 (rev 09)
2025-03-14T04:42:42.3354230Z 00:11.1 Unassigned class [ff00]: Intel Corporation C620 Series Chipset Family MROM 1 (rev 09)
2025-03-14T04:42:42.3355145Z 00:11.5 SATA controller: Intel Corporation C620 Series Chipset Family SSATA Controller [AHCI mode] (rev 09)
2025-03-14T04:42:42.3356209Z 00:14.0 USB controller: Intel Corporation C620 Series Chipset Family USB 3.0 xHCI Controller (rev 09)
2025-03-14T04:42:42.3357178Z 00:14.2 Signal processing controller: Intel Corporation C620 Series Chipset Family Thermal Subsystem (rev 09)
2025-03-14T04:42:42.3358191Z 00:16.0 Communication controller: Intel Corporation C620 Series Chipset Family MEI Controller #1 (rev 09)
2025-03-14T04:42:42.3359178Z 00:16.4 Communication controller: Intel Corporation C620 Series Chipset Family MEI Controller #3 (rev 09)
2025-03-14T04:42:42.3360249Z 00:17.0 SATA controller: Intel Corporation C620 Series Chipset Family SATA Controller [AHCI mode] (rev 09)
2025-03-14T04:42:42.3361198Z 00:1f.0 ISA bridge: Intel Corporation C621 Series Chipset LPC/eSPI Controller (rev 09)
2025-03-14T04:42:42.3362149Z 00:1f.2 Memory controller: Intel Corporation C620 Series Chipset Family Power Management Controller (rev 09)
2025-03-14T04:42:42.3362930Z 00:1f.4 SMBus: Intel Corporation C620 Series Chipset Family SMBus (rev 09)
2025-03-14T04:42:42.3363655Z 00:1f.5 Serial bus controller: Intel Corporation C620 Series Chipset Family SPI Controller (rev 09)
2025-03-14T04:42:42.3364418Z 17:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07)
2025-03-14T04:42:42.3365098Z 17:02.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port C (rev 07)
2025-03-14T04:42:42.3365733Z 17:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07)
2025-03-14T04:42:42.3366429Z 17:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07)
2025-03-14T04:42:42.3367175Z 17:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07)
2025-03-14T04:42:42.3367854Z 17:08.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3368530Z 17:08.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3369185Z 17:08.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3369872Z 17:08.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3370550Z 17:08.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3371226Z 17:08.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3371899Z 17:08.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3372567Z 17:08.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3373241Z 17:09.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3373914Z 17:09.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3374596Z 17:09.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3375318Z 17:09.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3375989Z 17:09.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3376664Z 17:09.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3377344Z 17:09.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3378017Z 17:09.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3378692Z 17:0a.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3379374Z 17:0a.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3380056Z 17:0a.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3380735Z 17:0a.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3381412Z 17:0a.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3382088Z 17:0a.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3382754Z 17:0a.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3383429Z 17:0a.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3384345Z 17:0b.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3385328Z 17:0b.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3386195Z 17:0b.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3386914Z 17:0b.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3387629Z 17:0e.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3388399Z 17:0e.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3389122Z 17:0e.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3389944Z 17:0e.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3390673Z 17:0e.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3391371Z 17:0e.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3392094Z 17:0e.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3392823Z 17:0e.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3393550Z 17:0f.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3394273Z 17:0f.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3395007Z 17:0f.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3395735Z 17:0f.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3396454Z 17:0f.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3397183Z 17:0f.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3397900Z 17:0f.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3398629Z 17:0f.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3399351Z 17:10.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3400079Z 17:10.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3400800Z 17:10.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3401527Z 17:10.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3402252Z 17:10.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3403061Z 17:10.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3403796Z 17:10.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3404520Z 17:10.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3405246Z 17:11.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3405973Z 17:11.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3406693Z 17:11.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3407410Z 17:11.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3408126Z 17:1d.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3408856Z 17:1d.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3409575Z 17:1d.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3410291Z 17:1d.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3411014Z 17:1e.0 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.3411741Z 17:1e.1 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.3412459Z 17:1e.2 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.3413179Z 17:1e.3 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.3413901Z 17:1e.4 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.3414620Z 17:1e.5 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.3415345Z 17:1e.6 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.3416073Z 18:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1)
2025-03-14T04:42:42.3416701Z 19:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1)
2025-03-14T04:42:42.3417379Z 34:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07)
2025-03-14T04:42:42.3418120Z 34:02.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port C (rev 07)
2025-03-14T04:42:42.3418819Z 34:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07)
2025-03-14T04:42:42.3419662Z 34:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07)
2025-03-14T04:42:42.3420448Z 34:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07)
2025-03-14T04:42:42.3421230Z 34:08.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3422073Z 34:09.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3422998Z 34:0a.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3423802Z 34:0a.1 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3424605Z 34:0a.2 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3425405Z 34:0a.3 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3426211Z 34:0a.4 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3426938Z 34:0a.5 System peripheral: Intel Corporation Sky Lake-E LM Channel 1 (rev 07)
2025-03-14T04:42:42.3427600Z 34:0a.6 System peripheral: Intel Corporation Sky Lake-E LMS Channel 1 (rev 07)
2025-03-14T04:42:42.3428273Z 34:0a.7 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 1 (rev 07)
2025-03-14T04:42:42.3428949Z 34:0b.0 System peripheral: Intel Corporation Sky Lake-E DECS Channel 2 (rev 07)
2025-03-14T04:42:42.3429617Z 34:0b.1 System peripheral: Intel Corporation Sky Lake-E LM Channel 2 (rev 07)
2025-03-14T04:42:42.3430313Z 34:0b.2 System peripheral: Intel Corporation Sky Lake-E LMS Channel 2 (rev 07)
2025-03-14T04:42:42.3430987Z 34:0b.3 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 2 (rev 07)
2025-03-14T04:42:42.3431723Z 34:0c.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3432527Z 34:0c.1 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3433337Z 34:0c.2 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3434147Z 34:0c.3 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3434959Z 34:0c.4 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3435693Z 34:0c.5 System peripheral: Intel Corporation Sky Lake-E LM Channel 1 (rev 07)
2025-03-14T04:42:42.3436376Z 34:0c.6 System peripheral: Intel Corporation Sky Lake-E LMS Channel 1 (rev 07)
2025-03-14T04:42:42.3437061Z 34:0c.7 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 1 (rev 07)
2025-03-14T04:42:42.3437748Z 34:0d.0 System peripheral: Intel Corporation Sky Lake-E DECS Channel 2 (rev 07)
2025-03-14T04:42:42.3438421Z 34:0d.1 System peripheral: Intel Corporation Sky Lake-E LM Channel 2 (rev 07)
2025-03-14T04:42:42.3439101Z 34:0d.2 System peripheral: Intel Corporation Sky Lake-E LMS Channel 2 (rev 07)
2025-03-14T04:42:42.3439785Z 34:0d.3 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 2 (rev 07)
2025-03-14T04:42:42.3440423Z 35:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1)
2025-03-14T04:42:42.3441009Z 36:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1)
2025-03-14T04:42:42.3441649Z 7a:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07)
2025-03-14T04:42:42.3442380Z 7a:01.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port B (rev 07)
2025-03-14T04:42:42.3443073Z 7a:02.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port C (rev 07)
2025-03-14T04:42:42.3443700Z 7a:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07)
2025-03-14T04:42:42.3444405Z 7a:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07)
2025-03-14T04:42:42.3445160Z 7a:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07)
2025-03-14T04:42:42.3445838Z 7a:0e.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07)
2025-03-14T04:42:42.3446501Z 7a:0e.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07)
2025-03-14T04:42:42.3447148Z 7a:0f.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07)
2025-03-14T04:42:42.3447797Z 7a:0f.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07)
2025-03-14T04:42:42.3448446Z 7a:10.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07)
2025-03-14T04:42:42.3449101Z 7a:10.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07)
2025-03-14T04:42:42.3449793Z 7a:12.0 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07)
2025-03-14T04:42:42.3450512Z 7a:12.1 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07)
2025-03-14T04:42:42.3451215Z 7a:12.2 System peripheral: Intel Corporation Sky Lake-E M3KTI Registers (rev 07)
2025-03-14T04:42:42.3451912Z 7a:12.4 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07)
2025-03-14T04:42:42.3452633Z 7a:12.5 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07)
2025-03-14T04:42:42.3453335Z 7a:15.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07)
2025-03-14T04:42:42.3454039Z 7a:15.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07)
2025-03-14T04:42:42.3454739Z 7a:16.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07)
2025-03-14T04:42:42.3455449Z 7a:16.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07)
2025-03-14T04:42:42.3456186Z 7a:16.4 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07)
2025-03-14T04:42:42.3456886Z 7a:16.5 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07)
2025-03-14T04:42:42.3457595Z 7a:17.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07)
2025-03-14T04:42:42.3458295Z 7a:17.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07)
2025-03-14T04:42:42.3458886Z 7b:00.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3459332Z 7c:00.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3459779Z 7c:01.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3460218Z 7c:02.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3460657Z 7c:03.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3461096Z 7c:04.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3461540Z 7c:05.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3461959Z 7c:06.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3462397Z 7c:07.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3462844Z 7c:08.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3463283Z 7c:09.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3463718Z 7c:0a.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3464154Z 7c:0b.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3464590Z 7c:0c.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3465027Z 7c:0d.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3465469Z 7c:0e.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3465900Z 7c:0f.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3466335Z 7c:10.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3466773Z 7c:11.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3467250Z 7c:12.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3467688Z 7c:13.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3468122Z 7c:14.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3468566Z 7c:15.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3469006Z 7c:16.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3469440Z 7c:17.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3469873Z 7c:18.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3470283Z 7c:19.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3470722Z 7c:1a.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3471165Z 7c:1b.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3471603Z 7c:1c.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3472040Z 7c:1d.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3472475Z 7c:1e.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3472920Z 7c:1f.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3473360Z 9d:00.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3473801Z 9e:00.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3474236Z 9e:01.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3474675Z 9e:02.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3475109Z 9e:03.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3475540Z 9e:04.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3475985Z 9e:05.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3476609Z 9e:06.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3477081Z 9e:07.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3477535Z 9e:08.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3477998Z 9e:09.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3478455Z 9e:0a.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3478898Z 9e:0b.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3479361Z 9e:0c.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3479874Z 9e:0d.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3480338Z 9e:0e.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3480801Z 9e:0f.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3481264Z 9e:10.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3481729Z 9e:11.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3482194Z 9e:12.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3482659Z 9e:13.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3483125Z 9e:14.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3483591Z 9e:15.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3484057Z 9e:16.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3484519Z 9e:17.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3484988Z 9e:18.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3485453Z 9e:19.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3485922Z 9e:1a.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3486390Z 9e:1b.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3486855Z 9e:1c.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3487320Z 9e:1d.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3487869Z 9e:1e.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3488321Z 9e:1f.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3488831Z 9f:00.0 Serial controller: Amazon.com, Inc. Device 8250
2025-03-14T04:42:42.3489569Z a0:00.0 Non-Volatile memory controller: Amazon.com, Inc. Device 0061
2025-03-14T04:42:42.3490422Z bf:00.0 Non-Volatile memory controller: Amazon.com, Inc. Device cd00
2025-03-14T04:42:42.3491061Z bf:00.1 Non-Volatile memory controller: Amazon.com, Inc. Device cd00
2025-03-14T04:42:42.3491856Z c0:04.0 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.3492599Z c0:04.1 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.3493345Z c0:04.2 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.3494079Z c0:04.3 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.3494820Z c0:04.4 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.3495560Z c0:04.5 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.3496299Z c0:04.6 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.3497044Z c0:04.7 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.3497866Z c0:05.0 System peripheral: Intel Corporation Sky Lake-E MM/Vt-d Configuration Registers (rev 07)
2025-03-14T04:42:42.3498627Z c0:05.2 System peripheral: Intel Corporation Sky Lake-E RAS (rev 07)
2025-03-14T04:42:42.3499215Z c0:05.4 PIC: Intel Corporation Sky Lake-E IOAPIC (rev 07)
2025-03-14T04:42:42.3499854Z c0:08.0 System peripheral: Intel Corporation Sky Lake-E Ubox Registers (rev 07)
2025-03-14T04:42:42.3500611Z c0:08.1 Performance counters: Intel Corporation Sky Lake-E Ubox Registers (rev 07)
2025-03-14T04:42:42.3501360Z c0:08.2 System peripheral: Intel Corporation Sky Lake-E Ubox Registers (rev 07)
2025-03-14T04:42:42.3502302Z c2:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07)
2025-03-14T04:42:42.3502940Z c2:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07)
2025-03-14T04:42:42.3503631Z c2:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07)
2025-03-14T04:42:42.3504370Z c2:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07)
2025-03-14T04:42:42.3505047Z c2:08.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3505720Z c2:08.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3506453Z c2:08.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3507126Z c2:08.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3507809Z c2:08.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3508480Z c2:08.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3509147Z c2:08.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3509786Z c2:08.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3510455Z c2:09.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3511114Z c2:09.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3511777Z c2:09.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3512441Z c2:09.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3513107Z c2:09.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3513764Z c2:09.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3514423Z c2:09.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3515086Z c2:09.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3515746Z c2:0a.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3516415Z c2:0a.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3517075Z c2:0a.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3517750Z c2:0a.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3518454Z c2:0a.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3519113Z c2:0a.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3519784Z c2:0a.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3520444Z c2:0a.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3521106Z c2:0b.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3521782Z c2:0b.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3522442Z c2:0b.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3523115Z c2:0b.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3523774Z c2:0e.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3524445Z c2:0e.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3525117Z c2:0e.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3525781Z c2:0e.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3526453Z c2:0e.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3527119Z c2:0e.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3527788Z c2:0e.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3528456Z c2:0e.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3529123Z c2:0f.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3529793Z c2:0f.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3530456Z c2:0f.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3531133Z c2:0f.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3532378Z c2:0f.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3533053Z c2:0f.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3533699Z c2:0f.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3534364Z c2:0f.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3535034Z c2:10.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3535708Z c2:10.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3536381Z c2:10.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3537051Z c2:10.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3537728Z c2:10.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3538417Z c2:10.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3539086Z c2:10.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3539755Z c2:10.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3540430Z c2:11.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3541106Z c2:11.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3541785Z c2:11.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3542459Z c2:11.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3543129Z c2:1d.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3543804Z c2:1d.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3544519Z c2:1d.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3545198Z c2:1d.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.3545883Z c2:1e.0 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.3546560Z c2:1e.1 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.3547240Z c2:1e.2 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.3547929Z c2:1e.3 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.3548608Z c2:1e.4 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.3549284Z c2:1e.5 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.3549963Z c2:1e.6 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.3550515Z c3:00.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3550957Z c4:00.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3551397Z c4:01.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3551828Z c4:02.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3552260Z c4:03.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3552673Z c4:04.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3553101Z c4:05.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3553534Z c4:06.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3553964Z c4:07.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3554401Z c4:08.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3554835Z c4:09.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3555264Z c4:0a.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3555706Z c4:0b.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3556304Z c4:0c.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3556802Z c4:0d.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3557310Z c4:0e.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3557768Z c4:0f.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.3558338Z c6:00.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA)
2025-03-14T04:42:42.3559054Z e6:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07)
2025-03-14T04:42:42.3559780Z e6:02.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port C (rev 07)
2025-03-14T04:42:42.3560466Z e6:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07)
2025-03-14T04:42:42.3561207Z e6:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07)
2025-03-14T04:42:42.3561994Z e6:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07)
2025-03-14T04:42:42.3562780Z e6:08.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3563628Z e6:09.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3564475Z e6:0a.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3565322Z e6:0a.1 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3566170Z e6:0a.2 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3567012Z e6:0a.3 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3567857Z e6:0a.4 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3568632Z e6:0a.5 System peripheral: Intel Corporation Sky Lake-E LM Channel 1 (rev 07)
2025-03-14T04:42:42.3569331Z e6:0a.6 System peripheral: Intel Corporation Sky Lake-E LMS Channel 1 (rev 07)
2025-03-14T04:42:42.3570017Z e6:0a.7 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 1 (rev 07)
2025-03-14T04:42:42.3570768Z e6:0b.0 System peripheral: Intel Corporation Sky Lake-E DECS Channel 2 (rev 07)
2025-03-14T04:42:42.3571472Z e6:0b.1 System peripheral: Intel Corporation Sky Lake-E LM Channel 2 (rev 07)
2025-03-14T04:42:42.3572166Z e6:0b.2 System peripheral: Intel Corporation Sky Lake-E LMS Channel 2 (rev 07)
2025-03-14T04:42:42.3572873Z e6:0b.3 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 2 (rev 07)
2025-03-14T04:42:42.3573655Z e6:0c.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3574503Z e6:0c.1 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3575348Z e6:0c.2 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3576193Z e6:0c.3 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3577041Z e6:0c.4 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.3577824Z e6:0c.5 System peripheral: Intel Corporation Sky Lake-E LM Channel 1 (rev 07)
2025-03-14T04:42:42.3578529Z e6:0c.6 System peripheral: Intel Corporation Sky Lake-E LMS Channel 1 (rev 07)
2025-03-14T04:42:42.3579235Z e6:0c.7 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 1 (rev 07)
2025-03-14T04:42:42.3579947Z e6:0d.0 System peripheral: Intel Corporation Sky Lake-E DECS Channel 2 (rev 07)
2025-03-14T04:42:42.3580643Z e6:0d.1 System peripheral: Intel Corporation Sky Lake-E LM Channel 2 (rev 07)
2025-03-14T04:42:42.3581463Z e6:0d.2 System peripheral: Intel Corporation Sky Lake-E LMS Channel 2 (rev 07)
2025-03-14T04:42:42.3582131Z e6:0d.3 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 2 (rev 07)
2025-03-14T04:42:42.3582745Z e7:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1)
2025-03-14T04:42:42.3583314Z e8:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1)
2025-03-14T04:42:42.3583952Z f3:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07)
2025-03-14T04:42:42.3584677Z f3:02.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port C (rev 07)
2025-03-14T04:42:42.3585323Z f3:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07)
2025-03-14T04:42:42.3586006Z f3:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07)
2025-03-14T04:42:42.3586743Z f3:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07)
2025-03-14T04:42:42.3587406Z f3:0e.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07)
2025-03-14T04:42:42.3588059Z f3:0e.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07)
2025-03-14T04:42:42.3588711Z f3:0f.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07)
2025-03-14T04:42:42.3589360Z f3:0f.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07)
2025-03-14T04:42:42.3590236Z f3:10.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07)
2025-03-14T04:42:42.3590946Z f3:10.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07)
2025-03-14T04:42:42.3591695Z f3:12.0 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07)
2025-03-14T04:42:42.3592485Z f3:12.1 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07)
2025-03-14T04:42:42.3593261Z f3:12.2 System peripheral: Intel Corporation Sky Lake-E M3KTI Registers (rev 07)
2025-03-14T04:42:42.3594028Z f3:12.4 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07)
2025-03-14T04:42:42.3594811Z f3:12.5 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07)
2025-03-14T04:42:42.3595580Z f3:15.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07)
2025-03-14T04:42:42.3596327Z f3:15.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07)
2025-03-14T04:42:42.3597099Z f3:16.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07)
2025-03-14T04:42:42.3597951Z f3:16.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07)
2025-03-14T04:42:42.3598727Z f3:16.4 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07)
2025-03-14T04:42:42.3599492Z f3:16.5 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07)
2025-03-14T04:42:42.3600264Z f3:17.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07)
2025-03-14T04:42:42.3601035Z f3:17.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07)
2025-03-14T04:42:42.3601870Z f4:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1)
2025-03-14T04:42:42.3602453Z f5:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1)
2025-03-14T04:42:42.3602927Z + lsmod
2025-03-14T04:42:42.3603239Z Module                  Size  Used by
2025-03-14T04:42:42.3603604Z veth                   36864  0
2025-03-14T04:42:42.3603953Z nvidia_modeset       1351680  0
2025-03-14T04:42:42.3604323Z video                  65536  1 nvidia_modeset
2025-03-14T04:42:42.3604720Z wmi                    36864  1 video
2025-03-14T04:42:42.3605081Z nvidia_uvm           4706304  0
2025-03-14T04:42:42.3605484Z nvidia              54071296  35 nvidia_uvm,nvidia_modeset
2025-03-14T04:42:42.3605893Z drm                   602112  1 nvidia
2025-03-14T04:42:42.3606277Z drm_panel_orientation_quirks    32768  1 drm
2025-03-14T04:42:42.3606725Z backlight              24576  3 video,drm,nvidia_modeset
2025-03-14T04:42:42.3607159Z i2c_core              110592  2 nvidia,drm
2025-03-14T04:42:42.3607534Z xt_conntrack           16384  1
2025-03-14T04:42:42.3607871Z nft_chain_nat          16384  3
2025-03-14T04:42:42.3608201Z xt_MASQUERADE          20480  1
2025-03-14T04:42:42.3608584Z nf_nat                 57344  2 nft_chain_nat,xt_MASQUERADE
2025-03-14T04:42:42.3609074Z nf_conntrack_netlink    57344  0
2025-03-14T04:42:42.3609573Z nf_conntrack          184320  4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE
2025-03-14T04:42:42.3610102Z nf_defrag_ipv6         24576  1 nf_conntrack
2025-03-14T04:42:42.3610560Z nf_defrag_ipv4         16384  1 nf_conntrack
2025-03-14T04:42:42.3610941Z xfrm_user              57344  1
2025-03-14T04:42:42.3611268Z xfrm_algo              16384  1 xfrm_user
2025-03-14T04:42:42.3611636Z xt_addrtype            16384  2
2025-03-14T04:42:42.3611971Z nft_compat             20480  4
2025-03-14T04:42:42.3612340Z nf_tables             311296  57 nft_compat,nft_chain_nat
2025-03-14T04:42:42.3612828Z nfnetlink              20480  4 nft_compat,nf_conntrack_netlink,nf_tables
2025-03-14T04:42:42.3613288Z br_netfilter           36864  0
2025-03-14T04:42:42.3613625Z bridge                323584  1 br_netfilter
2025-03-14T04:42:42.3613983Z stp                    16384  1 bridge
2025-03-14T04:42:42.3614345Z llc                    16384  2 bridge,stp
2025-03-14T04:42:42.3614709Z overlay               167936  0
2025-03-14T04:42:42.3615040Z tls                   135168  0
2025-03-14T04:42:42.3615368Z nls_ascii              16384  1
2025-03-14T04:42:42.3615681Z nls_cp437              20480  1
2025-03-14T04:42:42.3616000Z vfat                   24576  1
2025-03-14T04:42:42.3616299Z fat                    86016  1 vfat
2025-03-14T04:42:42.3616617Z skx_edac               20480  0
2025-03-14T04:42:42.3616934Z skx_edac_common        28672  1 skx_edac
2025-03-14T04:42:42.3617283Z x86_pkg_temp_thermal    20480  0
2025-03-14T04:42:42.3617605Z kvm_intel             393216  0
2025-03-14T04:42:42.3617922Z sunrpc                696320  1
2025-03-14T04:42:42.3618237Z kvm                  1155072  1 kvm_intel
2025-03-14T04:42:42.3618593Z irqbypass              16384  1 kvm
2025-03-14T04:42:42.3618913Z ena                   180224  0
2025-03-14T04:42:42.3619214Z ghash_clmulni_intel    16384  0
2025-03-14T04:42:42.3619525Z lpc_ich                28672  0
2025-03-14T04:42:42.3619848Z mfd_core               20480  1 lpc_ich
2025-03-14T04:42:42.3620207Z ioatdma                69632  0
2025-03-14T04:42:42.3620558Z dca                    16384  1 ioatdma
2025-03-14T04:42:42.3620908Z button                 24576  0
2025-03-14T04:42:42.3621244Z sch_fq_codel           20480  33
2025-03-14T04:42:42.3621548Z dm_mod                188416  0
2025-03-14T04:42:42.3621856Z fuse                  163840  1
2025-03-14T04:42:42.3622182Z configfs               57344  1
2025-03-14T04:42:42.3622508Z loop                   36864  0
2025-03-14T04:42:42.3622817Z dax                    45056  1 dm_mod
2025-03-14T04:42:42.3623147Z dmi_sysfs              20480  0
2025-03-14T04:42:42.3623454Z crc32_pclmul           16384  0
2025-03-14T04:42:42.3623757Z crc32c_intel           24576  0
2025-03-14T04:42:42.3624062Z + modinfo nvidia
2025-03-14T04:42:42.3624517Z filename:       /lib/modules/6.1.129-138.220.amzn2023.x86_64/kernel/drivers/video/nvidia.ko
2025-03-14T04:42:42.3625067Z alias:          char-major-195-*
2025-03-14T04:42:42.3625408Z version:        550.54.15
2025-03-14T04:42:42.3625729Z supported:      external
2025-03-14T04:42:42.3626043Z license:        NVIDIA
2025-03-14T04:42:42.3626360Z firmware:       nvidia/550.54.15/gsp_tu10x.bin
2025-03-14T04:42:42.3626779Z firmware:       nvidia/550.54.15/gsp_ga10x.bin
2025-03-14T04:42:42.3627164Z srcversion:     833721318DA517F0C2FEC97
2025-03-14T04:42:42.3651408Z alias:          pci:v000010DEd*sv*sd*bc06sc80i00*
2025-03-14T04:42:42.3651918Z alias:          pci:v000010DEd*sv*sd*bc03sc02i00*
2025-03-14T04:42:42.3652391Z alias:          pci:v000010DEd*sv*sd*bc03sc00i00*
2025-03-14T04:42:42.3652830Z depends:        i2c-core,drm
2025-03-14T04:42:42.3653203Z retpoline:      Y
2025-03-14T04:42:42.3653525Z name:           nvidia
2025-03-14T04:42:42.3654022Z vermagic:       6.1.129-138.220.amzn2023.x86_64 SMP preempt mod_unload modversions 
2025-03-14T04:42:42.3654663Z parm:           NvSwitchRegDwords:NvSwitch regkey (charp)
2025-03-14T04:42:42.3655269Z parm:           NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp)
2025-03-14T04:42:42.3655869Z parm:           NVreg_ResmanDebugLevel:int
2025-03-14T04:42:42.3656325Z parm:           NVreg_RmLogonRC:int
2025-03-14T04:42:42.3656748Z parm:           NVreg_ModifyDeviceFiles:int
2025-03-14T04:42:42.3657317Z parm:           NVreg_DeviceFileUID:int
2025-03-14T04:42:42.3657754Z parm:           NVreg_DeviceFileGID:int
2025-03-14T04:42:42.3658177Z parm:           NVreg_DeviceFileMode:int
2025-03-14T04:42:42.3658669Z parm:           NVreg_InitializeSystemMemoryAllocations:int
2025-03-14T04:42:42.3659194Z parm:           NVreg_UsePageAttributeTable:int
2025-03-14T04:42:42.3659663Z parm:           NVreg_EnablePCIeGen3:int
2025-03-14T04:42:42.3660084Z parm:           NVreg_EnableMSI:int
2025-03-14T04:42:42.3660497Z parm:           NVreg_TCEBypassMode:int
2025-03-14T04:42:42.3661052Z parm:           NVreg_EnableStreamMemOPs:int
2025-03-14T04:42:42.3661537Z parm:           NVreg_RestrictProfilingToAdminUsers:int
2025-03-14T04:42:42.3662067Z parm:           NVreg_PreserveVideoMemoryAllocations:int
2025-03-14T04:42:42.3662579Z parm:           NVreg_EnableS0ixPowerManagement:int
2025-03-14T04:42:42.3663130Z parm:           NVreg_S0ixPowerManagementVideoMemoryThreshold:int
2025-03-14T04:42:42.3663677Z parm:           NVreg_DynamicPowerManagement:int
2025-03-14T04:42:42.3664223Z parm:           NVreg_DynamicPowerManagementVideoMemoryThreshold:int
2025-03-14T04:42:42.3664771Z parm:           NVreg_EnableGpuFirmware:int
2025-03-14T04:42:42.3665226Z parm:           NVreg_EnableGpuFirmwareLogs:int
2025-03-14T04:42:42.3665713Z parm:           NVreg_OpenRmEnableUnsupportedGpus:int
2025-03-14T04:42:42.3666212Z parm:           NVreg_EnableUserNUMAManagement:int
2025-03-14T04:42:42.3666673Z parm:           NVreg_MemoryPoolSize:int
2025-03-14T04:42:42.3667116Z parm:           NVreg_KMallocHeapMaxSize:int
2025-03-14T04:42:42.3667543Z parm:           NVreg_VMallocHeapMaxSize:int
2025-03-14T04:42:42.3667991Z parm:           NVreg_IgnoreMMIOCheck:int
2025-03-14T04:42:42.3668427Z parm:           NVreg_NvLinkDisable:int
2025-03-14T04:42:42.3668990Z parm:           NVreg_EnablePCIERelaxedOrderingMode:int
2025-03-14T04:42:42.3669504Z parm:           NVreg_RegisterPCIDriver:int
2025-03-14T04:42:42.3669934Z parm:           NVreg_EnableResizableBar:int
2025-03-14T04:42:42.3670382Z parm:           NVreg_EnableDbgBreakpoint:int
2025-03-14T04:42:42.3670819Z parm:           NVreg_EnableNonblockingOpen:int
2025-03-14T04:42:42.3671254Z parm:           NVreg_RegistryDwords:charp
2025-03-14T04:42:42.3671710Z parm:           NVreg_RegistryDwordsPerDevice:charp
2025-03-14T04:42:42.3672135Z parm:           NVreg_RmMsg:charp
2025-03-14T04:42:42.3672517Z parm:           NVreg_GpuBlacklist:charp
2025-03-14T04:42:42.3673004Z parm:           NVreg_TemporaryFilePath:charp
2025-03-14T04:42:42.3673425Z parm:           NVreg_ExcludedGpus:charp
2025-03-14T04:42:42.3694571Z parm:           NVreg_DmaRemapPeerMmio:int
2025-03-14T04:42:42.3695039Z parm:           NVreg_RmNvlinkBandwidth:charp
2025-03-14T04:42:42.3695461Z parm:           NVreg_ImexChannelCount:int
2025-03-14T04:42:42.3695848Z parm:           rm_firmware_active:charp
2025-03-14T04:42:42.3696228Z + HAS_NVIDIA_DRIVER=0
2025-03-14T04:42:42.3696524Z ++ command -v nvidia-smi
2025-03-14T04:42:42.3696845Z + '[' -x /usr/bin/nvidia-smi ']'
2025-03-14T04:42:42.3697160Z + set +e
2025-03-14T04:42:42.3697542Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0
2025-03-14T04:42:42.3698076Z + INSTALLED_DRIVER_VERSION=550.54.15
2025-03-14T04:42:42.3698425Z + NVIDIA_SMI_STATUS=0
2025-03-14T04:42:42.3698698Z + '[' 0 -ne 0 ']'
2025-03-14T04:42:42.3698962Z + '[' 550.54.15 '!=' 550.54.15 ']'
2025-03-14T04:42:42.3699277Z + HAS_NVIDIA_DRIVER=1
2025-03-14T04:42:42.3699804Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation'
2025-03-14T04:42:42.3700416Z + set -e
2025-03-14T04:42:42.3700651Z + '[' 1 -eq 0 ']'
2025-03-14T04:42:42.3701119Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation
2025-03-14T04:42:42.3701695Z + post_install_nvidia_driver_common
2025-03-14T04:42:42.3702049Z + sudo modprobe nvidia
2025-03-14T04:42:42.5062927Z + echo 'After installing NVIDIA driver'
2025-03-14T04:42:42.5063988Z + lspci
2025-03-14T04:42:42.5065121Z After installing NVIDIA driver
2025-03-14T04:42:42.5313387Z 00:00.0 Host bridge: Intel Corporation Sky Lake-E DMI3 Registers (rev 07)
2025-03-14T04:42:42.5314262Z 00:04.0 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.5315170Z 00:04.1 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.5315940Z 00:04.2 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.5316696Z 00:04.3 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.5317445Z 00:04.4 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.5318197Z 00:04.5 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.5318942Z 00:04.6 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.5319712Z 00:04.7 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.5320537Z 00:05.0 System peripheral: Intel Corporation Sky Lake-E MM/Vt-d Configuration Registers (rev 07)
2025-03-14T04:42:42.5321310Z 00:05.2 System peripheral: Intel Corporation Sky Lake-E RAS (rev 07)
2025-03-14T04:42:42.5321916Z 00:05.4 PIC: Intel Corporation Sky Lake-E IOAPIC (rev 07)
2025-03-14T04:42:42.5322553Z 00:08.0 System peripheral: Intel Corporation Sky Lake-E Ubox Registers (rev 07)
2025-03-14T04:42:42.5323324Z 00:08.1 Performance counters: Intel Corporation Sky Lake-E Ubox Registers (rev 07)
2025-03-14T04:42:42.5324094Z 00:08.2 System peripheral: Intel Corporation Sky Lake-E Ubox Registers (rev 07)
2025-03-14T04:42:42.5324892Z 00:11.0 Unassigned class [ff00]: Intel Corporation C620 Series Chipset Family MROM 0 (rev 09)
2025-03-14T04:42:42.5325736Z 00:11.1 Unassigned class [ff00]: Intel Corporation C620 Series Chipset Family MROM 1 (rev 09)
2025-03-14T04:42:42.5328362Z 00:11.5 SATA controller: Intel Corporation C620 Series Chipset Family SSATA Controller [AHCI mode] (rev 09)
2025-03-14T04:42:42.5329335Z 00:14.0 USB controller: Intel Corporation C620 Series Chipset Family USB 3.0 xHCI Controller (rev 09)
2025-03-14T04:42:42.5330318Z 00:14.2 Signal processing controller: Intel Corporation C620 Series Chipset Family Thermal Subsystem (rev 09)
2025-03-14T04:42:42.5331334Z 00:16.0 Communication controller: Intel Corporation C620 Series Chipset Family MEI Controller #1 (rev 09)
2025-03-14T04:42:42.5332439Z 00:16.4 Communication controller: Intel Corporation C620 Series Chipset Family MEI Controller #3 (rev 09)
2025-03-14T04:42:42.5333398Z 00:17.0 SATA controller: Intel Corporation C620 Series Chipset Family SATA Controller [AHCI mode] (rev 09)
2025-03-14T04:42:42.5334268Z 00:1f.0 ISA bridge: Intel Corporation C621 Series Chipset LPC/eSPI Controller (rev 09)
2025-03-14T04:42:42.5335149Z 00:1f.2 Memory controller: Intel Corporation C620 Series Chipset Family Power Management Controller (rev 09)
2025-03-14T04:42:42.5335990Z 00:1f.4 SMBus: Intel Corporation C620 Series Chipset Family SMBus (rev 09)
2025-03-14T04:42:42.5336784Z 00:1f.5 Serial bus controller: Intel Corporation C620 Series Chipset Family SPI Controller (rev 09)
2025-03-14T04:42:42.5337600Z 17:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07)
2025-03-14T04:42:42.5338330Z 17:02.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port C (rev 07)
2025-03-14T04:42:42.5339010Z 17:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07)
2025-03-14T04:42:42.5339760Z 17:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07)
2025-03-14T04:42:42.5340552Z 17:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07)
2025-03-14T04:42:42.5341385Z 17:08.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5342067Z 17:08.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5342750Z 17:08.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5343534Z 17:08.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5344387Z 17:08.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5345076Z 17:08.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5345794Z 17:08.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5346505Z 17:08.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5347216Z 17:09.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5347926Z 17:09.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5348627Z 17:09.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5349342Z 17:09.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5350063Z 17:09.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5350756Z 17:09.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5351443Z 17:09.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5352137Z 17:09.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5352825Z 17:0a.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5353516Z 17:0a.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5354210Z 17:0a.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5354910Z 17:0a.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5355600Z 17:0a.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5356445Z 17:0a.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5357090Z 17:0a.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5357749Z 17:0a.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5358419Z 17:0b.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5359271Z 17:0b.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5359960Z 17:0b.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5360649Z 17:0b.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5361343Z 17:0e.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5362032Z 17:0e.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5362755Z 17:0e.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5363445Z 17:0e.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5364129Z 17:0e.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5364812Z 17:0e.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5365492Z 17:0e.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5366169Z 17:0e.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5366857Z 17:0f.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5367532Z 17:0f.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5368212Z 17:0f.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5368871Z 17:0f.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5369551Z 17:0f.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5370340Z 17:0f.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5371045Z 17:0f.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5371745Z 17:0f.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5372446Z 17:10.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5373150Z 17:10.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5373959Z 17:10.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5374635Z 17:10.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5375297Z 17:10.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5375950Z 17:10.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5376602Z 17:10.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5377377Z 17:10.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5378043Z 17:11.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5378709Z 17:11.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5379372Z 17:11.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5380026Z 17:11.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5380677Z 17:1d.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5381345Z 17:1d.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5382009Z 17:1d.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5382721Z 17:1d.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5383387Z 17:1e.0 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.5384052Z 17:1e.1 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.5384721Z 17:1e.2 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.5385389Z 17:1e.3 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.5386069Z 17:1e.4 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.5386715Z 17:1e.5 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.5387384Z 17:1e.6 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.5388012Z 18:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1)
2025-03-14T04:42:42.5388591Z 19:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1)
2025-03-14T04:42:42.5389217Z 34:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07)
2025-03-14T04:42:42.5390373Z 34:02.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port C (rev 07)
2025-03-14T04:42:42.5391033Z 34:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07)
2025-03-14T04:42:42.5391749Z 34:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07)
2025-03-14T04:42:42.5392523Z 34:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07)
2025-03-14T04:42:42.5393297Z 34:08.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5394297Z 34:09.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5395144Z 34:0a.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5396003Z 34:0a.1 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5396923Z 34:0a.2 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5397771Z 34:0a.3 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5398618Z 34:0a.4 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5399398Z 34:0a.5 System peripheral: Intel Corporation Sky Lake-E LM Channel 1 (rev 07)
2025-03-14T04:42:42.5400108Z 34:0a.6 System peripheral: Intel Corporation Sky Lake-E LMS Channel 1 (rev 07)
2025-03-14T04:42:42.5400833Z 34:0a.7 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 1 (rev 07)
2025-03-14T04:42:42.5401556Z 34:0b.0 System peripheral: Intel Corporation Sky Lake-E DECS Channel 2 (rev 07)
2025-03-14T04:42:42.5402382Z 34:0b.1 System peripheral: Intel Corporation Sky Lake-E LM Channel 2 (rev 07)
2025-03-14T04:42:42.5403227Z 34:0b.2 System peripheral: Intel Corporation Sky Lake-E LMS Channel 2 (rev 07)
2025-03-14T04:42:42.5403926Z 34:0b.3 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 2 (rev 07)
2025-03-14T04:42:42.5404698Z 34:0c.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5405531Z 34:0c.1 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5406461Z 34:0c.2 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5407245Z 34:0c.3 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5408239Z 34:0c.4 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5409161Z 34:0c.5 System peripheral: Intel Corporation Sky Lake-E LM Channel 1 (rev 07)
2025-03-14T04:42:42.5409867Z 34:0c.6 System peripheral: Intel Corporation Sky Lake-E LMS Channel 1 (rev 07)
2025-03-14T04:42:42.5410692Z 34:0c.7 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 1 (rev 07)
2025-03-14T04:42:42.5411412Z 34:0d.0 System peripheral: Intel Corporation Sky Lake-E DECS Channel 2 (rev 07)
2025-03-14T04:42:42.5412122Z 34:0d.1 System peripheral: Intel Corporation Sky Lake-E LM Channel 2 (rev 07)
2025-03-14T04:42:42.5412837Z 34:0d.2 System peripheral: Intel Corporation Sky Lake-E LMS Channel 2 (rev 07)
2025-03-14T04:42:42.5413561Z 34:0d.3 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 2 (rev 07)
2025-03-14T04:42:42.5414238Z 35:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1)
2025-03-14T04:42:42.5414851Z 36:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1)
2025-03-14T04:42:42.5415517Z 7a:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07)
2025-03-14T04:42:42.5416224Z 7a:01.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port B (rev 07)
2025-03-14T04:42:42.5416944Z 7a:02.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port C (rev 07)
2025-03-14T04:42:42.5417619Z 7a:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07)
2025-03-14T04:42:42.5418359Z 7a:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07)
2025-03-14T04:42:42.5419151Z 7a:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07)
2025-03-14T04:42:42.5419962Z 7a:0e.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07)
2025-03-14T04:42:42.5420634Z 7a:0e.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07)
2025-03-14T04:42:42.5421304Z 7a:0f.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07)
2025-03-14T04:42:42.5422081Z 7a:0f.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07)
2025-03-14T04:42:42.5422716Z 7a:10.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07)
2025-03-14T04:42:42.5423352Z 7a:10.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07)
2025-03-14T04:42:42.5424040Z 7a:12.0 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07)
2025-03-14T04:42:42.5424802Z 7a:12.1 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07)
2025-03-14T04:42:42.5425483Z 7a:12.2 System peripheral: Intel Corporation Sky Lake-E M3KTI Registers (rev 07)
2025-03-14T04:42:42.5426157Z 7a:12.4 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07)
2025-03-14T04:42:42.5426869Z 7a:12.5 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07)
2025-03-14T04:42:42.5427554Z 7a:15.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07)
2025-03-14T04:42:42.5428243Z 7a:15.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07)
2025-03-14T04:42:42.5428939Z 7a:16.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07)
2025-03-14T04:42:42.5429623Z 7a:16.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07)
2025-03-14T04:42:42.5430324Z 7a:16.4 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07)
2025-03-14T04:42:42.5431010Z 7a:16.5 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07)
2025-03-14T04:42:42.5431703Z 7a:17.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07)
2025-03-14T04:42:42.5432375Z 7a:17.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07)
2025-03-14T04:42:42.5432947Z 7b:00.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5433379Z 7c:00.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5433798Z 7c:01.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5434223Z 7c:02.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5434635Z 7c:03.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5435048Z 7c:04.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5435453Z 7c:05.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5435869Z 7c:06.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5436339Z 7c:07.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5436760Z 7c:08.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5437183Z 7c:09.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5437605Z 7c:0a.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5438032Z 7c:0b.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5438461Z 7c:0c.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5438888Z 7c:0d.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5439317Z 7c:0e.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5439753Z 7c:0f.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5440180Z 7c:10.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5440605Z 7c:11.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5441035Z 7c:12.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5441465Z 7c:13.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5441894Z 7c:14.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5442324Z 7c:15.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5442753Z 7c:16.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5443177Z 7c:17.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5443583Z 7c:18.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5443999Z 7c:19.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5444415Z 7c:1a.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5444849Z 7c:1b.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5445277Z 7c:1c.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5445714Z 7c:1d.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5446147Z 7c:1e.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5446564Z 7c:1f.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5446982Z 9d:00.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5447406Z 9e:00.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5447880Z 9e:01.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5448308Z 9e:02.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5448725Z 9e:03.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5449143Z 9e:04.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5449564Z 9e:05.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5449990Z 9e:06.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5450416Z 9e:07.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5450839Z 9e:08.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5451273Z 9e:09.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5451681Z 9e:0a.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5452102Z 9e:0b.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5452523Z 9e:0c.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5452947Z 9e:0d.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5453369Z 9e:0e.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5453803Z 9e:0f.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5454230Z 9e:10.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5454655Z 9e:11.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5455074Z 9e:12.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5455507Z 9e:13.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5455930Z 9e:14.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5456352Z 9e:15.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5456777Z 9e:16.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5457198Z 9e:17.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5457622Z 9e:18.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5458037Z 9e:19.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5458507Z 9e:1a.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5458938Z 9e:1b.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5459368Z 9e:1c.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5459762Z 9e:1d.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5460183Z 9e:1e.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5460609Z 9e:1f.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5461066Z 9f:00.0 Serial controller: Amazon.com, Inc. Device 8250
2025-03-14T04:42:42.5461595Z a0:00.0 Non-Volatile memory controller: Amazon.com, Inc. Device 0061
2025-03-14T04:42:42.5462172Z bf:00.0 Non-Volatile memory controller: Amazon.com, Inc. Device cd00
2025-03-14T04:42:42.5462747Z bf:00.1 Non-Volatile memory controller: Amazon.com, Inc. Device cd00
2025-03-14T04:42:42.5463363Z c0:04.0 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.5464040Z c0:04.1 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.5464711Z c0:04.2 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.5465383Z c0:04.3 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.5466048Z c0:04.4 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.5466719Z c0:04.5 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.5467386Z c0:04.6 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.5468074Z c0:04.7 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 07)
2025-03-14T04:42:42.5468799Z c0:05.0 System peripheral: Intel Corporation Sky Lake-E MM/Vt-d Configuration Registers (rev 07)
2025-03-14T04:42:42.5469489Z c0:05.2 System peripheral: Intel Corporation Sky Lake-E RAS (rev 07)
2025-03-14T04:42:42.5470009Z c0:05.4 PIC: Intel Corporation Sky Lake-E IOAPIC (rev 07)
2025-03-14T04:42:42.5470563Z c0:08.0 System peripheral: Intel Corporation Sky Lake-E Ubox Registers (rev 07)
2025-03-14T04:42:42.5471290Z c0:08.1 Performance counters: Intel Corporation Sky Lake-E Ubox Registers (rev 07)
2025-03-14T04:42:42.5471967Z c0:08.2 System peripheral: Intel Corporation Sky Lake-E Ubox Registers (rev 07)
2025-03-14T04:42:42.5472627Z c2:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07)
2025-03-14T04:42:42.5473255Z c2:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07)
2025-03-14T04:42:42.5473932Z c2:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07)
2025-03-14T04:42:42.5474866Z c2:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07)
2025-03-14T04:42:42.5475574Z c2:08.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5476267Z c2:08.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5476961Z c2:08.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5477649Z c2:08.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5478337Z c2:08.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5478996Z c2:08.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5479676Z c2:08.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5480367Z c2:08.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5481061Z c2:09.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5481764Z c2:09.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5482462Z c2:09.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5483159Z c2:09.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5483953Z c2:09.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5484662Z c2:09.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5485359Z c2:09.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5486055Z c2:09.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5486818Z c2:0a.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5487520Z c2:0a.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5488216Z c2:0a.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5488919Z c2:0a.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5489611Z c2:0a.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5490551Z c2:0a.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5491280Z c2:0a.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5491992Z c2:0a.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5492712Z c2:0b.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5493430Z c2:0b.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5494156Z c2:0b.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5494875Z c2:0b.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5495599Z c2:0e.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5496314Z c2:0e.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5497017Z c2:0e.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5497726Z c2:0e.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5498511Z c2:0e.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5499218Z c2:0e.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5500035Z c2:0e.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5500725Z c2:0e.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5501415Z c2:0f.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5502212Z c2:0f.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5502855Z c2:0f.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5503511Z c2:0f.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5504301Z c2:0f.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5504954Z c2:0f.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5505607Z c2:0f.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5506266Z c2:0f.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5506915Z c2:10.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5507565Z c2:10.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5508218Z c2:10.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5508882Z c2:10.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5509533Z c2:10.4 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5510188Z c2:10.5 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5510927Z c2:10.6 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5511592Z c2:10.7 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5512254Z c2:11.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5512915Z c2:11.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5513572Z c2:11.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5514223Z c2:11.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5514876Z c2:1d.0 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5515525Z c2:1d.1 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5516175Z c2:1d.2 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5516832Z c2:1d.3 System peripheral: Intel Corporation Sky Lake-E CHA Registers (rev 07)
2025-03-14T04:42:42.5517494Z c2:1e.0 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.5518158Z c2:1e.1 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.5518816Z c2:1e.2 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.5519474Z c2:1e.3 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.5520126Z c2:1e.4 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.5520779Z c2:1e.5 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.5521434Z c2:1e.6 System peripheral: Intel Corporation Sky Lake-E PCU Registers (rev 07)
2025-03-14T04:42:42.5521981Z c3:00.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5522413Z c4:00.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5522835Z c4:01.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5523266Z c4:02.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5523708Z c4:03.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5524129Z c4:04.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5524549Z c4:05.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5524968Z c4:06.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5525377Z c4:07.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5525799Z c4:08.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5526220Z c4:09.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5526641Z c4:0a.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5527066Z c4:0b.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5527495Z c4:0c.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5527914Z c4:0d.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5528345Z c4:0e.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5528774Z c4:0f.0 PCI bridge: Amazon.com, Inc. Device bec2
2025-03-14T04:42:42.5529306Z c6:00.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA)
2025-03-14T04:42:42.5529954Z e6:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07)
2025-03-14T04:42:42.5530618Z e6:02.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port C (rev 07)
2025-03-14T04:42:42.5531244Z e6:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07)
2025-03-14T04:42:42.5531929Z e6:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07)
2025-03-14T04:42:42.5532659Z e6:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07)
2025-03-14T04:42:42.5533389Z e6:08.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5534178Z e6:09.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5535005Z e6:0a.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5535796Z e6:0a.1 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5536583Z e6:0a.2 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5537376Z e6:0a.3 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5538173Z e6:0a.4 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5538881Z e6:0a.5 System peripheral: Intel Corporation Sky Lake-E LM Channel 1 (rev 07)
2025-03-14T04:42:42.5539540Z e6:0a.6 System peripheral: Intel Corporation Sky Lake-E LMS Channel 1 (rev 07)
2025-03-14T04:42:42.5540210Z e6:0a.7 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 1 (rev 07)
2025-03-14T04:42:42.5540886Z e6:0b.0 System peripheral: Intel Corporation Sky Lake-E DECS Channel 2 (rev 07)
2025-03-14T04:42:42.5541555Z e6:0b.1 System peripheral: Intel Corporation Sky Lake-E LM Channel 2 (rev 07)
2025-03-14T04:42:42.5542217Z e6:0b.2 System peripheral: Intel Corporation Sky Lake-E LMS Channel 2 (rev 07)
2025-03-14T04:42:42.5542888Z e6:0b.3 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 2 (rev 07)
2025-03-14T04:42:42.5543627Z e6:0c.0 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5544427Z e6:0c.1 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5545221Z e6:0c.2 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5546016Z e6:0c.3 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5546806Z e6:0c.4 System peripheral: Intel Corporation Sky Lake-E Integrated Memory Controller (rev 07)
2025-03-14T04:42:42.5547528Z e6:0c.5 System peripheral: Intel Corporation Sky Lake-E LM Channel 1 (rev 07)
2025-03-14T04:42:42.5548184Z e6:0c.6 System peripheral: Intel Corporation Sky Lake-E LMS Channel 1 (rev 07)
2025-03-14T04:42:42.5548892Z e6:0c.7 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 1 (rev 07)
2025-03-14T04:42:42.5549558Z e6:0d.0 System peripheral: Intel Corporation Sky Lake-E DECS Channel 2 (rev 07)
2025-03-14T04:42:42.5550217Z e6:0d.1 System peripheral: Intel Corporation Sky Lake-E LM Channel 2 (rev 07)
2025-03-14T04:42:42.5550856Z e6:0d.2 System peripheral: Intel Corporation Sky Lake-E LMS Channel 2 (rev 07)
2025-03-14T04:42:42.5551516Z e6:0d.3 System peripheral: Intel Corporation Sky Lake-E LMDP Channel 2 (rev 07)
2025-03-14T04:42:42.5552352Z e7:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1)
2025-03-14T04:42:42.5552951Z e8:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1)
2025-03-14T04:42:42.5553598Z f3:00.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port A (rev 07)
2025-03-14T04:42:42.5554298Z f3:02.0 PCI bridge: Intel Corporation Sky Lake-E PCI Express Root Port C (rev 07)
2025-03-14T04:42:42.5554963Z f3:05.0 System peripheral: Intel Corporation Sky Lake-E VT-d (rev 07)
2025-03-14T04:42:42.5555680Z f3:05.2 System peripheral: Intel Corporation Sky Lake-E RAS Configuration Registers (rev 07)
2025-03-14T04:42:42.5556457Z f3:05.4 PIC: Intel Corporation Sky Lake-E IOxAPIC Configuration Registers (rev 07)
2025-03-14T04:42:42.5557154Z f3:0e.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07)
2025-03-14T04:42:42.5557831Z f3:0e.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07)
2025-03-14T04:42:42.5558500Z f3:0f.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07)
2025-03-14T04:42:42.5559170Z f3:0f.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07)
2025-03-14T04:42:42.5559851Z f3:10.0 Performance counters: Intel Corporation Sky Lake-E KTI 0 (rev 07)
2025-03-14T04:42:42.5560514Z f3:10.1 System peripheral: Intel Corporation Sky Lake-E UPI Registers (rev 07)
2025-03-14T04:42:42.5561279Z f3:12.0 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07)
2025-03-14T04:42:42.5562022Z f3:12.1 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07)
2025-03-14T04:42:42.5562751Z f3:12.2 System peripheral: Intel Corporation Sky Lake-E M3KTI Registers (rev 07)
2025-03-14T04:42:42.5563480Z f3:12.4 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07)
2025-03-14T04:42:42.5564211Z f3:12.5 Performance counters: Intel Corporation Sky Lake-E M3KTI Registers (rev 07)
2025-03-14T04:42:42.5565112Z f3:15.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07)
2025-03-14T04:42:42.5565868Z f3:15.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07)
2025-03-14T04:42:42.5566623Z f3:16.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07)
2025-03-14T04:42:42.5567376Z f3:16.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07)
2025-03-14T04:42:42.5568135Z f3:16.4 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07)
2025-03-14T04:42:42.5568887Z f3:16.5 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07)
2025-03-14T04:42:42.5569640Z f3:17.0 System peripheral: Intel Corporation Sky Lake-E M2PCI Registers (rev 07)
2025-03-14T04:42:42.5570384Z f3:17.1 Performance counters: Intel Corporation Sky Lake-E DDRIO Registers (rev 07)
2025-03-14T04:42:42.5571073Z f4:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1)
2025-03-14T04:42:42.5571685Z f5:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1)
2025-03-14T04:42:42.5572174Z + lsmod
2025-03-14T04:42:42.5572469Z Module                  Size  Used by
2025-03-14T04:42:42.5572847Z veth                   36864  0
2025-03-14T04:42:42.5573199Z nvidia_modeset       1351680  0
2025-03-14T04:42:42.5573574Z video                  65536  1 nvidia_modeset
2025-03-14T04:42:42.5573985Z wmi                    36864  1 video
2025-03-14T04:42:42.5574359Z nvidia_uvm           4706304  0
2025-03-14T04:42:42.5574771Z nvidia              54071296  35 nvidia_uvm,nvidia_modeset
2025-03-14T04:42:42.5575266Z drm                   602112  1 nvidia
2025-03-14T04:42:42.5575682Z drm_panel_orientation_quirks    32768  1 drm
2025-03-14T04:42:42.5576165Z backlight              24576  3 video,drm,nvidia_modeset
2025-03-14T04:42:42.5576615Z i2c_core              110592  2 nvidia,drm
2025-03-14T04:42:42.5577101Z xt_conntrack           16384  1
2025-03-14T04:42:42.5577454Z nft_chain_nat          16384  3
2025-03-14T04:42:42.5577798Z xt_MASQUERADE          20480  1
2025-03-14T04:42:42.5578308Z nf_nat                 57344  2 nft_chain_nat,xt_MASQUERADE
2025-03-14T04:42:42.5578707Z nf_conntrack_netlink    57344  0
2025-03-14T04:42:42.5579192Z nf_conntrack          184320  4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE
2025-03-14T04:42:42.5579730Z nf_defrag_ipv6         24576  1 nf_conntrack
2025-03-14T04:42:42.5580126Z nf_defrag_ipv4         16384  1 nf_conntrack
2025-03-14T04:42:42.5580498Z xfrm_user              57344  1
2025-03-14T04:42:42.5580840Z xfrm_algo              16384  1 xfrm_user
2025-03-14T04:42:42.5581199Z xt_addrtype            16384  2
2025-03-14T04:42:42.5581523Z nft_compat             20480  4
2025-03-14T04:42:42.5581898Z nf_tables             311296  57 nft_compat,nft_chain_nat
2025-03-14T04:42:42.5582403Z nfnetlink              20480  4 nft_compat,nf_conntrack_netlink,nf_tables
2025-03-14T04:42:42.5582866Z br_netfilter           36864  0
2025-03-14T04:42:42.5583205Z bridge                323584  1 br_netfilter
2025-03-14T04:42:42.5583574Z stp                    16384  1 bridge
2025-03-14T04:42:42.5583924Z llc                    16384  2 bridge,stp
2025-03-14T04:42:42.5584259Z overlay               167936  0
2025-03-14T04:42:42.5584578Z tls                   135168  0
2025-03-14T04:42:42.5584884Z nls_ascii              16384  1
2025-03-14T04:42:42.5585201Z nls_cp437              20480  1
2025-03-14T04:42:42.5585512Z vfat                   24576  1
2025-03-14T04:42:42.5585886Z fat                    86016  1 vfat
2025-03-14T04:42:42.5586214Z skx_edac               20480  0
2025-03-14T04:42:42.5586557Z skx_edac_common        28672  1 skx_edac
2025-03-14T04:42:42.5586943Z x86_pkg_temp_thermal    20480  0
2025-03-14T04:42:42.5587285Z kvm_intel             393216  0
2025-03-14T04:42:42.5587595Z sunrpc                696320  1
2025-03-14T04:42:42.5587909Z kvm                  1155072  1 kvm_intel
2025-03-14T04:42:42.5588276Z irqbypass              16384  1 kvm
2025-03-14T04:42:42.5588600Z ena                   180224  0
2025-03-14T04:42:42.5588909Z ghash_clmulni_intel    16384  0
2025-03-14T04:42:42.5589240Z lpc_ich                28672  0
2025-03-14T04:42:42.5589556Z mfd_core               20480  1 lpc_ich
2025-03-14T04:42:42.5589957Z ioatdma                69632  0
2025-03-14T04:42:42.5590470Z dca                    16384  1 ioatdma
2025-03-14T04:42:42.5590816Z button                 24576  0
2025-03-14T04:42:42.5591146Z sch_fq_codel           20480  33
2025-03-14T04:42:42.5591496Z dm_mod                188416  0
2025-03-14T04:42:42.5591833Z fuse                  163840  1
2025-03-14T04:42:42.5592171Z configfs               57344  1
2025-03-14T04:42:42.5592510Z loop                   36864  0
2025-03-14T04:42:42.5592848Z dax                    45056  1 dm_mod
2025-03-14T04:42:42.5593212Z dmi_sysfs              20480  0
2025-03-14T04:42:42.5593536Z crc32_pclmul           16384  0
2025-03-14T04:42:42.5593858Z crc32c_intel           24576  0
2025-03-14T04:42:42.5594178Z + modinfo nvidia
2025-03-14T04:42:42.5594673Z filename:       /lib/modules/6.1.129-138.220.amzn2023.x86_64/kernel/drivers/video/nvidia.ko
2025-03-14T04:42:42.5595247Z alias:          char-major-195-*
2025-03-14T04:42:42.5595579Z version:        550.54.15
2025-03-14T04:42:42.5595888Z supported:      external
2025-03-14T04:42:42.5596189Z license:        NVIDIA
2025-03-14T04:42:42.5596530Z firmware:       nvidia/550.54.15/gsp_tu10x.bin
2025-03-14T04:42:42.5596973Z firmware:       nvidia/550.54.15/gsp_ga10x.bin
2025-03-14T04:42:42.5597377Z srcversion:     833721318DA517F0C2FEC97
2025-03-14T04:42:42.5597879Z alias:          pci:v000010DEd*sv*sd*bc06sc80i00*
2025-03-14T04:42:42.5598313Z alias:          pci:v000010DEd*sv*sd*bc03sc02i00*
2025-03-14T04:42:42.5598732Z alias:          pci:v000010DEd*sv*sd*bc03sc00i00*
2025-03-14T04:42:42.5599126Z depends:        i2c-core,drm
2025-03-14T04:42:42.5599474Z retpoline:      Y
2025-03-14T04:42:42.5599780Z name:           nvidia
2025-03-14T04:42:42.5600237Z vermagic:       6.1.129-138.220.amzn2023.x86_64 SMP preempt mod_unload modversions 
2025-03-14T04:42:42.5600838Z parm:           NvSwitchRegDwords:NvSwitch regkey (charp)
2025-03-14T04:42:42.5601549Z parm:           NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp)
2025-03-14T04:42:42.5602200Z parm:           NVreg_ResmanDebugLevel:int
2025-03-14T04:42:42.5602582Z parm:           NVreg_RmLogonRC:int
2025-03-14T04:42:42.5602958Z parm:           NVreg_ModifyDeviceFiles:int
2025-03-14T04:42:42.5603339Z parm:           NVreg_DeviceFileUID:int
2025-03-14T04:42:42.5603707Z parm:           NVreg_DeviceFileGID:int
2025-03-14T04:42:42.5604082Z parm:           NVreg_DeviceFileMode:int
2025-03-14T04:42:42.5604524Z parm:           NVreg_InitializeSystemMemoryAllocations:int
2025-03-14T04:42:42.5605006Z parm:           NVreg_UsePageAttributeTable:int
2025-03-14T04:42:42.5605416Z parm:           NVreg_EnablePCIeGen3:int
2025-03-14T04:42:42.5605777Z parm:           NVreg_EnableMSI:int
2025-03-14T04:42:42.5606134Z parm:           NVreg_TCEBypassMode:int
2025-03-14T04:42:42.5606504Z parm:           NVreg_EnableStreamMemOPs:int
2025-03-14T04:42:42.5606926Z parm:           NVreg_RestrictProfilingToAdminUsers:int
2025-03-14T04:42:42.5607383Z parm:           NVreg_PreserveVideoMemoryAllocations:int
2025-03-14T04:42:42.5607822Z parm:           NVreg_EnableS0ixPowerManagement:int
2025-03-14T04:42:42.5608295Z parm:           NVreg_S0ixPowerManagementVideoMemoryThreshold:int
2025-03-14T04:42:42.5608772Z parm:           NVreg_DynamicPowerManagement:int
2025-03-14T04:42:42.5610300Z parm:           NVreg_DynamicPowerManagementVideoMemoryThreshold:int
2025-03-14T04:42:42.5610796Z parm:           NVreg_EnableGpuFirmware:int
2025-03-14T04:42:42.5610948Z parm:           NVreg_EnableGpuFirmwareLogs:int
2025-03-14T04:42:42.5611136Z parm:           NVreg_OpenRmEnableUnsupportedGpus:int
2025-03-14T04:42:42.5611292Z parm:           NVreg_EnableUserNUMAManagement:int
2025-03-14T04:42:42.5611442Z parm:           NVreg_MemoryPoolSize:int
2025-03-14T04:42:42.5611586Z parm:           NVreg_KMallocHeapMaxSize:int
2025-03-14T04:42:42.5611748Z parm:           NVreg_VMallocHeapMaxSize:int
2025-03-14T04:42:42.5611885Z parm:           NVreg_IgnoreMMIOCheck:int
2025-03-14T04:42:42.5612038Z parm:           NVreg_NvLinkDisable:int
2025-03-14T04:42:42.5612208Z parm:           NVreg_EnablePCIERelaxedOrderingMode:int
2025-03-14T04:42:42.5612365Z parm:           NVreg_RegisterPCIDriver:int
2025-03-14T04:42:42.5612501Z parm:           NVreg_EnableResizableBar:int
2025-03-14T04:42:42.5612674Z parm:           NVreg_EnableDbgBreakpoint:int
2025-03-14T04:42:42.5612827Z parm:           NVreg_EnableNonblockingOpen:int
2025-03-14T04:42:42.5612991Z parm:           NVreg_RegistryDwords:charp
2025-03-14T04:42:42.5613151Z parm:           NVreg_RegistryDwordsPerDevice:charp
2025-03-14T04:42:42.5613264Z parm:           NVreg_RmMsg:charp
2025-03-14T04:42:42.5613407Z parm:           NVreg_GpuBlacklist:charp
2025-03-14T04:42:42.5613560Z parm:           NVreg_TemporaryFilePath:charp
2025-03-14T04:42:42.5613719Z parm:           NVreg_ExcludedGpus:charp
2025-03-14T04:42:42.5613858Z parm:           NVreg_DmaRemapPeerMmio:int
2025-03-14T04:42:42.5614021Z parm:           NVreg_RmNvlinkBandwidth:charp
2025-03-14T04:42:42.5614156Z parm:           NVreg_ImexChannelCount:int
2025-03-14T04:42:42.5614302Z parm:           rm_firmware_active:charp
2025-03-14T04:42:42.5614394Z + set +e
2025-03-14T04:42:42.5614519Z + nvidia-smi
2025-03-14T04:42:42.6187305Z Fri Mar 14 04:42:42 2025       
2025-03-14T04:42:42.6187690Z +-----------------------------------------------------------------------------------------+
2025-03-14T04:42:42.6188324Z | NVIDIA-SMI 550.54.15              Driver Version: 550.54.15      CUDA Version: 12.4     |
2025-03-14T04:42:42.6188582Z |-----------------------------------------+------------------------+----------------------+
2025-03-14T04:42:42.6188868Z | GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
2025-03-14T04:42:42.6189167Z | Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
2025-03-14T04:42:42.6189334Z |                                         |                        |               MIG M. |
2025-03-14T04:42:42.6189479Z |=========================================+========================+======================|
2025-03-14T04:42:42.6369570Z |   0  Tesla T4                       On  |   00000000:18:00.0 Off |                    0 |
2025-03-14T04:42:42.6369949Z | N/A   22C    P8              9W /   70W |       0MiB /  15360MiB |      0%      Default |
2025-03-14T04:42:42.6370146Z |                                         |                        |                  N/A |
2025-03-14T04:42:42.6370386Z +-----------------------------------------+------------------------+----------------------+
2025-03-14T04:42:42.6512861Z |   1  Tesla T4                       On  |   00000000:19:00.0 Off |                    0 |
2025-03-14T04:42:42.6513456Z | N/A   22C    P8              9W /   70W |       0MiB /  15360MiB |      0%      Default |
2025-03-14T04:42:42.6513970Z |                                         |                        |                  N/A |
2025-03-14T04:42:42.6514512Z +-----------------------------------------+------------------------+----------------------+
2025-03-14T04:42:42.6655746Z |   2  Tesla T4                       On  |   00000000:35:00.0 Off |                    0 |
2025-03-14T04:42:42.6657282Z | N/A   22C    P8              9W /   70W |       0MiB /  15360MiB |      0%      Default |
2025-03-14T04:42:42.6658932Z |                                         |                        |                  N/A |
2025-03-14T04:42:42.6660308Z +-----------------------------------------+------------------------+----------------------+
2025-03-14T04:42:42.6807565Z |   3  Tesla T4                       On  |   00000000:36:00.0 Off |                    0 |
2025-03-14T04:42:42.6809071Z | N/A   21C    P8              9W /   70W |       0MiB /  15360MiB |      0%      Default |
2025-03-14T04:42:42.6810375Z |                                         |                        |                  N/A |
2025-03-14T04:42:42.6811743Z +-----------------------------------------+------------------------+----------------------+
2025-03-14T04:42:42.6969911Z |   4  Tesla T4                       On  |   00000000:E7:00.0 Off |                    0 |
2025-03-14T04:42:42.6971459Z | N/A   22C    P8             11W /   70W |       0MiB /  15360MiB |      0%      Default |
2025-03-14T04:42:42.6972795Z |                                         |                        |                  N/A |
2025-03-14T04:42:42.6974303Z +-----------------------------------------+------------------------+----------------------+
2025-03-14T04:42:42.7120868Z |   5  Tesla T4                       On  |   00000000:E8:00.0 Off |                    0 |
2025-03-14T04:42:42.7122400Z | N/A   22C    P8              9W /   70W |       0MiB /  15360MiB |      0%      Default |
2025-03-14T04:42:42.7123717Z |                                         |                        |                  N/A |
2025-03-14T04:42:42.7125004Z +-----------------------------------------+------------------------+----------------------+
2025-03-14T04:42:42.7261493Z |   6  Tesla T4                       On  |   00000000:F4:00.0 Off |                    0 |
2025-03-14T04:42:42.7263037Z | N/A   21C    P8              9W /   70W |       0MiB /  15360MiB |      0%      Default |
2025-03-14T04:42:42.7264349Z |                                         |                        |                  N/A |
2025-03-14T04:42:42.7265392Z +-----------------------------------------+------------------------+----------------------+
2025-03-14T04:42:42.7418319Z |   7  Tesla T4                       On  |   00000000:F5:00.0 Off |                    0 |
2025-03-14T04:42:42.7419881Z | N/A   21C    P8              9W /   70W |       0MiB /  15360MiB |      0%      Default |
2025-03-14T04:42:42.7421181Z |                                         |                        |                  N/A |
2025-03-14T04:42:42.7422541Z +-----------------------------------------+------------------------+----------------------+
2025-03-14T04:42:42.7465581Z                                                                                          
2025-03-14T04:42:42.7467054Z +-----------------------------------------------------------------------------------------+
2025-03-14T04:42:42.7468555Z | Processes:                                                                              |
2025-03-14T04:42:42.7470238Z |  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
2025-03-14T04:42:42.7471724Z |        ID   ID                                                               Usage      |
2025-03-14T04:42:42.7472890Z |=========================================================================================|
2025-03-14T04:42:42.7568735Z |  No running processes found                                                             |
2025-03-14T04:42:42.7570442Z +-----------------------------------------------------------------------------------------+
2025-03-14T04:42:43.8938594Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0
2025-03-14T04:42:43.9112638Z Tesla T4
2025-03-14T04:42:43.9169102Z + NVIDIA_SMI_STATUS=0
2025-03-14T04:42:43.9169985Z + '[' 0 -eq 0 ']'
2025-03-14T04:42:43.9170810Z + echo 'INFO: Ignoring allowed status 0'
2025-03-14T04:42:43.9171778Z + set -e
2025-03-14T04:42:43.9172082Z INFO: Ignoring allowed status 0
2025-03-14T04:42:43.9179587Z == Installing nvidia container toolkit for amzn2023 ==
2025-03-14T04:42:43.9184648Z + sudo yum install -y yum-utils
2025-03-14T04:42:44.2989547Z Last metadata expiration check: 0:41:06 ago on Fri Mar 14 04:01:38 2025.
2025-03-14T04:42:44.3224887Z Package dnf-utils-4.3.0-13.amzn2023.0.5.noarch is already installed.
2025-03-14T04:42:44.3648082Z Dependencies resolved.
2025-03-14T04:42:44.3876885Z Nothing to do.
2025-03-14T04:42:44.3877667Z Complete!
2025-03-14T04:42:44.4242940Z + [[ amzn2023 == \a\m\z\n\2\0\2\3 ]]
2025-03-14T04:42:44.4243790Z + YUM_REPO_URL=https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo
2025-03-14T04:42:44.4244950Z + sudo yum-config-manager --add-repo https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo
2025-03-14T04:42:44.7252651Z Adding repo from: https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo
2025-03-14T04:42:44.7763707Z + sudo yum install -y nvidia-docker2 nvidia-container-toolkit-1.16.2
2025-03-14T04:42:45.2586207Z nvidia-container-toolkit                         17 kB/s | 833  B     00:00    
2025-03-14T04:42:45.2823613Z Package nvidia-docker2-2.14.0-1.noarch is already installed.
2025-03-14T04:42:45.2827276Z Package nvidia-container-toolkit-1.16.2-1.x86_64 is already installed.
2025-03-14T04:42:45.3246909Z Dependencies resolved.
2025-03-14T04:42:45.3475184Z Nothing to do.
2025-03-14T04:42:45.3476201Z Complete!
2025-03-14T04:42:45.3836367Z + sudo systemctl restart docker
2025-03-14T04:45:19.8639032Z nvidia-persistenced failed to initialize. Check syslog for more details.
2025-03-14T04:45:19.9492066Z Fri Mar 14 04:45:19 2025       
2025-03-14T04:45:19.9492902Z +-----------------------------------------------------------------------------------------+
2025-03-14T04:45:19.9494135Z | NVIDIA-SMI 550.54.15              Driver Version: 550.54.15      CUDA Version: 12.4     |
2025-03-14T04:45:19.9494973Z |-----------------------------------------+------------------------+----------------------+
2025-03-14T04:45:19.9496221Z | GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
2025-03-14T04:45:19.9497740Z | Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
2025-03-14T04:45:19.9498753Z |                                         |                        |               MIG M. |
2025-03-14T04:45:19.9499558Z |=========================================+========================+======================|
2025-03-14T04:45:19.9677149Z |   0  Tesla T4                       On  |   00000000:18:00.0 Off |                    0 |
2025-03-14T04:45:19.9678087Z | N/A   22C    P8              9W /   70W |       0MiB /  15360MiB |      0%      Default |
2025-03-14T04:45:19.9678678Z |                                         |                        |                  N/A |
2025-03-14T04:45:19.9679238Z +-----------------------------------------+------------------------+----------------------+
2025-03-14T04:45:19.9823106Z |   1  Tesla T4                       On  |   00000000:19:00.0 Off |                    0 |
2025-03-14T04:45:19.9823838Z | N/A   22C    P8              9W /   70W |       0MiB /  15360MiB |      0%      Default |
2025-03-14T04:45:19.9824378Z |                                         |                        |                  N/A |
2025-03-14T04:45:19.9824928Z +-----------------------------------------+------------------------+----------------------+
2025-03-14T04:45:19.9961289Z |   2  Tesla T4                       On  |   00000000:35:00.0 Off |                    0 |
2025-03-14T04:45:19.9961905Z | N/A   22C    P8             10W /   70W |       0MiB /  15360MiB |      0%      Default |
2025-03-14T04:45:19.9962413Z |                                         |                        |                  N/A |
2025-03-14T04:45:19.9973922Z +-----------------------------------------+------------------------+----------------------+
2025-03-14T04:45:20.0104377Z |   3  Tesla T4                       On  |   00000000:36:00.0 Off |                    0 |
2025-03-14T04:45:20.0105281Z | N/A   21C    P8              9W /   70W |       0MiB /  15360MiB |      0%      Default |
2025-03-14T04:45:20.0105823Z |                                         |                        |                  N/A |
2025-03-14T04:45:20.0106378Z +-----------------------------------------+------------------------+----------------------+
2025-03-14T04:45:20.0244672Z |   4  Tesla T4                       On  |   00000000:E7:00.0 Off |                    0 |
2025-03-14T04:45:20.0245325Z | N/A   22C    P8              9W /   70W |       0MiB /  15360MiB |      0%      Default |
2025-03-14T04:45:20.0245870Z |                                         |                        |                  N/A |
2025-03-14T04:45:20.0246433Z +-----------------------------------------+------------------------+----------------------+
2025-03-14T04:45:20.0383810Z |   5  Tesla T4                       On  |   00000000:E8:00.0 Off |                    0 |
2025-03-14T04:45:20.0384438Z | N/A   22C    P8              9W /   70W |       0MiB /  15360MiB |      0%      Default |
2025-03-14T04:45:20.0384994Z |                                         |                        |                  N/A |
2025-03-14T04:45:20.0385614Z +-----------------------------------------+------------------------+----------------------+
2025-03-14T04:45:20.0522084Z |   6  Tesla T4                       On  |   00000000:F4:00.0 Off |                    0 |
2025-03-14T04:45:20.0522744Z | N/A   21C    P8              9W /   70W |       0MiB /  15360MiB |      0%      Default |
2025-03-14T04:45:20.0523278Z |                                         |                        |                  N/A |
2025-03-14T04:45:20.0523833Z +-----------------------------------------+------------------------+----------------------+
2025-03-14T04:45:20.0666417Z |   7  Tesla T4                       On  |   00000000:F5:00.0 Off |                    0 |
2025-03-14T04:45:20.0667662Z | N/A   21C    P8              9W /   70W |       0MiB /  15360MiB |      0%      Default |
2025-03-14T04:45:20.0669864Z |                                         |                        |                  N/A |
2025-03-14T04:45:20.0670591Z +-----------------------------------------+------------------------+----------------------+
2025-03-14T04:45:20.0698687Z                                                                                          
2025-03-14T04:45:20.0699558Z +-----------------------------------------------------------------------------------------+
2025-03-14T04:45:20.0700181Z | Processes:                                                                              |
2025-03-14T04:45:20.0700812Z |  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
2025-03-14T04:45:20.0701405Z |        ID   ID                                                               Usage      |
2025-03-14T04:45:20.0701891Z |=========================================================================================|
2025-03-14T04:45:20.0807138Z |  No running processes found                                                             |
2025-03-14T04:45:20.0807928Z +-----------------------------------------------------------------------------------------+
2025-03-14T04:45:22.0425492Z Command completed after 1 attempt(s).
2025-03-14T04:45:22.0529649Z Prepare all required actions
2025-03-14T04:45:22.0564443Z ##[group]Run ./.github/actions/get-workflow-job-id
2025-03-14T04:45:22.0564897Z with:
2025-03-14T04:45:22.0565470Z   github-token: ***
2025-03-14T04:45:22.0565796Z env:
2025-03-14T04:45:22.0566113Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:45:22.0566543Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:45:22.0567001Z ##[endgroup]
2025-03-14T04:45:22.0593465Z ##[group]Run set -eux
2025-03-14T04:45:22.0593870Z set -eux
2025-03-14T04:45:22.0594448Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}"
2025-03-14T04:45:22.0605623Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:45:22.0606263Z env:
2025-03-14T04:45:22.0606541Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:45:22.0606973Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:45:22.0607653Z   GITHUB_TOKEN: ***
2025-03-14T04:45:22.0607927Z ##[endgroup]
2025-03-14T04:45:22.0637388Z + python3 .github/scripts/get_workflow_job_id.py 13849515290 i-0d8b0d116e8f11da9
2025-03-14T04:45:24.2336851Z setting job-id=38755508671
2025-03-14T04:45:24.2337631Z setting job-name=cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu)
2025-03-14T04:45:24.2477119Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 dataclasses_json==0.6.7
2025-03-14T04:45:24.2477995Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 dataclasses_json==0.6.7
2025-03-14T04:45:24.2478696Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 &
2025-03-14T04:45:24.2479289Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}"
2025-03-14T04:45:24.2484989Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:45:24.2485497Z env:
2025-03-14T04:45:24.2485803Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:45:24.2486231Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:45:24.2486700Z   JOB_ID: 38755508671
2025-03-14T04:45:24.2487273Z   JOB_NAME: cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu)
2025-03-14T04:45:24.2487916Z   WORKFLOW_NAME: pull
2025-03-14T04:45:24.2488275Z   WORKFLOW_RUN_ID: 13849515290
2025-03-14T04:45:24.2488714Z ##[endgroup]
2025-03-14T04:45:24.4831381Z Defaulting to user installation because normal site-packages is not writeable
2025-03-14T04:45:24.4989606Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.9/site-packages (5.9.1)
2025-03-14T04:45:24.4992833Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.9/site-packages (11.525.84)
2025-03-14T04:45:24.4997334Z Requirement already satisfied: dataclasses_json==0.6.7 in /home/ec2-user/.local/lib/python3.9/site-packages (0.6.7)
2025-03-14T04:45:24.5099827Z Requirement already satisfied: typing-inspect<1,>=0.4.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from dataclasses_json==0.6.7) (0.9.0)
2025-03-14T04:45:24.5104098Z Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from dataclasses_json==0.6.7) (3.26.1)
2025-03-14T04:45:24.5193491Z Requirement already satisfied: packaging>=17.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from marshmallow<4.0.0,>=3.18.0->dataclasses_json==0.6.7) (24.2)
2025-03-14T04:45:24.5224700Z Requirement already satisfied: typing-extensions>=3.7.4 in /home/ec2-user/.local/lib/python3.9/site-packages (from typing-inspect<1,>=0.4.0->dataclasses_json==0.6.7) (4.12.2)
2025-03-14T04:45:24.5227499Z Requirement already satisfied: mypy-extensions>=0.3.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from typing-inspect<1,>=0.4.0->dataclasses_json==0.6.7) (1.0.0)
2025-03-14T04:45:24.7078241Z Prepare all required actions
2025-03-14T04:45:24.7078763Z Getting action download info
2025-03-14T04:45:24.8371644Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6)
2025-03-14T04:45:25.0286380Z Download action repository 'actions/download-artifact@v4' (SHA:cc203385981b70ca67e1cc392babf9cc229d5806)
2025-03-14T04:45:25.2581620Z ##[group]Run ./.github/actions/download-build-artifacts
2025-03-14T04:45:25.2582096Z with:
2025-03-14T04:45:25.2582452Z   name: linux-focal-cuda12.4-py3.10-gcc9-sm75
2025-03-14T04:45:25.2582893Z   s3-bucket: gha-artifacts
2025-03-14T04:45:25.2583243Z env:
2025-03-14T04:45:25.2583537Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:45:25.2583950Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:45:25.2584412Z ##[endgroup]
2025-03-14T04:45:25.2620104Z ##[group]Run seemethere/download-artifact-s3@v4
2025-03-14T04:45:25.2620561Z with:
2025-03-14T04:45:25.2621013Z   name: linux-focal-cuda12.4-py3.10-gcc9-sm75
2025-03-14T04:45:25.2621631Z   s3-bucket: gha-artifacts
2025-03-14T04:45:25.2621998Z   region: us-east-1
2025-03-14T04:45:25.2622313Z env:
2025-03-14T04:45:25.2622588Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:45:25.2623029Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:45:25.2623495Z ##[endgroup]
2025-03-14T04:45:25.7407357Z (node:85980) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.
2025-03-14T04:45:25.7407972Z 
2025-03-14T04:45:25.7408220Z Please migrate your code to use AWS SDK for JavaScript (v3).
2025-03-14T04:45:25.7408879Z For more information, check the migration guide at https://a.co/7PzMCcy
2025-03-14T04:45:25.7409560Z (Use `node --trace-warnings ...` to show where the warning was created)
2025-03-14T04:45:25.8505722Z Found 1 objects with prefix pytorch/pytorch/13849515290/linux-focal-cuda12.4-py3.10-gcc9-sm75/
2025-03-14T04:45:25.8506714Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip
2025-03-14T04:45:42.9890997Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip
2025-03-14T04:45:42.9896581Z Artifact download has finished successfully
2025-03-14T04:45:43.0155373Z ##[group]Run unzip -o artifacts.zip
2025-03-14T04:45:43.0155826Z unzip -o artifacts.zip
2025-03-14T04:45:43.0164025Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:45:43.0164543Z env:
2025-03-14T04:45:43.0164832Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:45:43.0165284Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:45:43.0165756Z ##[endgroup]
2025-03-14T04:45:43.0210359Z Archive:  artifacts.zip
2025-03-14T04:45:43.0210783Z    creating: dist/
2025-03-14T04:45:45.0834675Z   inflating: dist/torch-2.8.0a0+gitaed0b7a-cp310-cp310-linux_x86_64.whl  
2025-03-14T04:45:45.0959887Z   inflating: dist/.ninja_log         
2025-03-14T04:45:45.0960949Z    creating: build/custom_test_artifacts/
2025-03-14T04:45:45.0961662Z    creating: build/custom_test_artifacts/custom-op-build/
2025-03-14T04:45:45.0962312Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/
2025-03-14T04:45:45.0963055Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/
2025-03-14T04:45:45.0967816Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml  
2025-03-14T04:45:45.0968665Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/
2025-03-14T04:45:45.0969504Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CMakeSystem.cmake  
2025-03-14T04:45:45.0970388Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdC/
2025-03-14T04:45:45.0971258Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdC/tmp/
2025-03-14T04:45:45.0972240Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdC/CMakeCCompilerId.c  
2025-03-14T04:45:45.0973561Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdC/a.out  
2025-03-14T04:45:45.0974518Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CMakeCCompiler.cmake  
2025-03-14T04:45:45.0975430Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCXX/
2025-03-14T04:45:45.0976306Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCXX/tmp/
2025-03-14T04:45:45.0977338Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCXX/CMakeCXXCompilerId.cpp  
2025-03-14T04:45:45.0978381Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCXX/a.out  
2025-03-14T04:45:45.0979341Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CMakeCXXCompiler.cmake  
2025-03-14T04:45:45.0980370Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_C.bin  
2025-03-14T04:45:45.0982039Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_CXX.bin  
2025-03-14T04:45:45.0983038Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/
2025-03-14T04:45:45.0983934Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/
2025-03-14T04:45:45.1025576Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii  
2025-03-14T04:45:45.1064796Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp  
2025-03-14T04:45:45.1066290Z  extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id  
2025-03-14T04:45:45.1112783Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii  
2025-03-14T04:45:45.1114115Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c  
2025-03-14T04:45:45.1115408Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu  
2025-03-14T04:45:45.1116722Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c  
2025-03-14T04:45:45.1117992Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx  
2025-03-14T04:45:45.1119244Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin  
2025-03-14T04:45:45.1120509Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin  
2025-03-14T04:45:45.1121748Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c  
2025-03-14T04:45:45.1122966Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o  
2025-03-14T04:45:45.1124124Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin  
2025-03-14T04:45:45.1125243Z  extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.reg.c  
2025-03-14T04:45:45.1126354Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.fatbin  
2025-03-14T04:45:45.1127458Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.fatbin.c  
2025-03-14T04:45:45.1128528Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.o  
2025-03-14T04:45:45.1129613Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/CMakeCUDACompilerId.cu  
2025-03-14T04:45:45.1200273Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/a.out  
2025-03-14T04:45:45.1201298Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CMakeCUDACompiler.cmake  
2025-03-14T04:45:45.1273967Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_CUDA.bin  
2025-03-14T04:45:45.1275010Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/
2025-03-14T04:45:45.1275763Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/
2025-03-14T04:45:45.1276578Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache  
2025-03-14T04:45:45.1277401Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/
2025-03-14T04:45:45.1278319Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts  
2025-03-14T04:45:45.1279370Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make  
2025-03-14T04:45:45.1280530Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make  
2025-03-14T04:45:45.1281473Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt  
2025-03-14T04:45:45.1282410Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake  
2025-03-14T04:45:45.1283356Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make  
2025-03-14T04:45:45.1284306Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake  
2025-03-14T04:45:45.1285253Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make  
2025-03-14T04:45:45.1286195Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make  
2025-03-14T04:45:45.1298811Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d  
2025-03-14T04:45:45.1500376Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o  
2025-03-14T04:45:45.1501287Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/
2025-03-14T04:45:45.1502258Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts  
2025-03-14T04:45:45.1503345Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make  
2025-03-14T04:45:45.1504376Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make  
2025-03-14T04:45:45.1505339Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt  
2025-03-14T04:45:45.1506348Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake  
2025-03-14T04:45:45.1507391Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make  
2025-03-14T04:45:45.1508442Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake  
2025-03-14T04:45:45.1509438Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make  
2025-03-14T04:45:45.1510408Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make  
2025-03-14T04:45:45.1525424Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d  
2025-03-14T04:45:45.1607002Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o  
2025-03-14T04:45:45.1608292Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake  
2025-03-14T04:45:45.1609244Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt  
2025-03-14T04:45:45.1610278Z  extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks  
2025-03-14T04:45:45.1611082Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2  
2025-03-14T04:45:45.1611883Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake  
2025-03-14T04:45:45.1612670Z   inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc  
2025-03-14T04:45:45.1613409Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt  
2025-03-14T04:45:45.1614096Z   inflating: build/custom_test_artifacts/custom-op-build/Makefile  
2025-03-14T04:45:45.1614799Z   inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake  
2025-03-14T04:45:45.1772762Z   inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so  
2025-03-14T04:45:45.1834675Z   inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops  
2025-03-14T04:45:45.1835360Z    creating: build/custom_test_artifacts/jit-hook-build/
2025-03-14T04:45:45.1836298Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/
2025-03-14T04:45:45.1837014Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/
2025-03-14T04:45:45.1841792Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml  
2025-03-14T04:45:45.1842594Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/
2025-03-14T04:45:45.1843414Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CMakeSystem.cmake  
2025-03-14T04:45:45.1844285Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdC/
2025-03-14T04:45:45.1845138Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdC/tmp/
2025-03-14T04:45:45.1846114Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdC/CMakeCCompilerId.c  
2025-03-14T04:45:45.1847112Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdC/a.out  
2025-03-14T04:45:45.1848049Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CMakeCCompiler.cmake  
2025-03-14T04:45:45.1848951Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCXX/
2025-03-14T04:45:45.1849829Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCXX/tmp/
2025-03-14T04:45:45.1850862Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCXX/CMakeCXXCompilerId.cpp  
2025-03-14T04:45:45.1852197Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCXX/a.out  
2025-03-14T04:45:45.1853159Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CMakeCXXCompiler.cmake  
2025-03-14T04:45:45.1854662Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_C.bin  
2025-03-14T04:45:45.1856271Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_CXX.bin  
2025-03-14T04:45:45.1857264Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/
2025-03-14T04:45:45.1858156Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/
2025-03-14T04:45:45.1899024Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii  
2025-03-14T04:45:45.1938294Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp  
2025-03-14T04:45:45.1939771Z  extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id  
2025-03-14T04:45:45.1984179Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii  
2025-03-14T04:45:45.1985812Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c  
2025-03-14T04:45:45.1987112Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu  
2025-03-14T04:45:45.1988426Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c  
2025-03-14T04:45:45.1989681Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx  
2025-03-14T04:45:45.1990984Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin  
2025-03-14T04:45:45.1992233Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin  
2025-03-14T04:45:45.1993470Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c  
2025-03-14T04:45:45.1994675Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o  
2025-03-14T04:45:45.1995902Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin  
2025-03-14T04:45:45.1997010Z  extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.reg.c  
2025-03-14T04:45:45.1998100Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.fatbin  
2025-03-14T04:45:45.1999202Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.fatbin.c  
2025-03-14T04:45:45.2000265Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.o  
2025-03-14T04:45:45.2001358Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/CMakeCUDACompilerId.cu  
2025-03-14T04:45:45.2071111Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/a.out  
2025-03-14T04:45:45.2072121Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CMakeCUDACompiler.cmake  
2025-03-14T04:45:45.2147296Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_CUDA.bin  
2025-03-14T04:45:45.2148465Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/
2025-03-14T04:45:45.2149204Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/
2025-03-14T04:45:45.2149997Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache  
2025-03-14T04:45:45.2150838Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/
2025-03-14T04:45:45.2151782Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts  
2025-03-14T04:45:45.2152862Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make  
2025-03-14T04:45:45.2153898Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make  
2025-03-14T04:45:45.2154856Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt  
2025-03-14T04:45:45.2155850Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake  
2025-03-14T04:45:45.2156848Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make  
2025-03-14T04:45:45.2157846Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake  
2025-03-14T04:45:45.2158843Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make  
2025-03-14T04:45:45.2159817Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make  
2025-03-14T04:45:45.2174386Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d  
2025-03-14T04:45:45.2237248Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o  
2025-03-14T04:45:45.2238331Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake  
2025-03-14T04:45:45.2239298Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt  
2025-03-14T04:45:45.2240171Z  extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks  
2025-03-14T04:45:45.2240971Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2  
2025-03-14T04:45:45.2241762Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake  
2025-03-14T04:45:45.2242557Z   inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc  
2025-03-14T04:45:45.2243303Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt  
2025-03-14T04:45:45.2243999Z   inflating: build/custom_test_artifacts/jit-hook-build/Makefile  
2025-03-14T04:45:45.2244872Z   inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake  
2025-03-14T04:45:45.2293365Z   inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks  
2025-03-14T04:45:45.2294078Z    creating: build/custom_test_artifacts/custom-backend-build/
2025-03-14T04:45:45.2294750Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/
2025-03-14T04:45:45.2295531Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/
2025-03-14T04:45:45.2300913Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml  
2025-03-14T04:45:45.2301805Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/
2025-03-14T04:45:45.2302681Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CMakeSystem.cmake  
2025-03-14T04:45:45.2303633Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdC/
2025-03-14T04:45:45.2304558Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdC/tmp/
2025-03-14T04:45:45.2305597Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdC/CMakeCCompilerId.c  
2025-03-14T04:45:45.2306655Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdC/a.out  
2025-03-14T04:45:45.2307648Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CMakeCCompiler.cmake  
2025-03-14T04:45:45.2308619Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCXX/
2025-03-14T04:45:45.2309554Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCXX/tmp/
2025-03-14T04:45:45.2310640Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCXX/CMakeCXXCompilerId.cpp  
2025-03-14T04:45:45.2311751Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCXX/a.out  
2025-03-14T04:45:45.2312776Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CMakeCXXCompiler.cmake  
2025-03-14T04:45:45.2313871Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_C.bin  
2025-03-14T04:45:45.2315030Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_CXX.bin  
2025-03-14T04:45:45.2316083Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/
2025-03-14T04:45:45.2317035Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/
2025-03-14T04:45:45.2357679Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii  
2025-03-14T04:45:45.2397730Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp  
2025-03-14T04:45:45.2399148Z  extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id  
2025-03-14T04:45:45.2443320Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii  
2025-03-14T04:45:45.2444886Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c  
2025-03-14T04:45:45.2446253Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu  
2025-03-14T04:45:45.2447645Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c  
2025-03-14T04:45:45.2448992Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx  
2025-03-14T04:45:45.2450321Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin  
2025-03-14T04:45:45.2451824Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin  
2025-03-14T04:45:45.2453149Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c  
2025-03-14T04:45:45.2454441Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o  
2025-03-14T04:45:45.2455663Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin  
2025-03-14T04:45:45.2456862Z  extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.reg.c  
2025-03-14T04:45:45.2458041Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.fatbin  
2025-03-14T04:45:45.2459227Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.fatbin.c  
2025-03-14T04:45:45.2460385Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.o  
2025-03-14T04:45:45.2461554Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/CMakeCUDACompilerId.cu  
2025-03-14T04:45:45.2531154Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/a.out  
2025-03-14T04:45:45.2532220Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CMakeCUDACompiler.cmake  
2025-03-14T04:45:45.2606321Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_CUDA.bin  
2025-03-14T04:45:45.2607388Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/
2025-03-14T04:45:45.2608229Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/
2025-03-14T04:45:45.2609093Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache  
2025-03-14T04:45:45.2609992Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/
2025-03-14T04:45:45.2611010Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts  
2025-03-14T04:45:45.2612159Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make  
2025-03-14T04:45:45.2613261Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make  
2025-03-14T04:45:45.2614292Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt  
2025-03-14T04:45:45.2615367Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake  
2025-03-14T04:45:45.2616632Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make  
2025-03-14T04:45:45.2617721Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake  
2025-03-14T04:45:45.2618805Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make  
2025-03-14T04:45:45.2619859Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make  
2025-03-14T04:45:45.2620994Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d  
2025-03-14T04:45:45.2740370Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o  
2025-03-14T04:45:45.2741482Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/
2025-03-14T04:45:45.2742568Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts  
2025-03-14T04:45:45.2743929Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make  
2025-03-14T04:45:45.2745084Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make  
2025-03-14T04:45:45.2746173Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt  
2025-03-14T04:45:45.2747289Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake  
2025-03-14T04:45:45.2748424Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make  
2025-03-14T04:45:45.2749546Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake  
2025-03-14T04:45:45.2750682Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make  
2025-03-14T04:45:45.2751796Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make  
2025-03-14T04:45:45.2764666Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d  
2025-03-14T04:45:45.2820739Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o  
2025-03-14T04:45:45.2822129Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake  
2025-03-14T04:45:45.2823174Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt  
2025-03-14T04:45:45.2824118Z  extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks  
2025-03-14T04:45:45.2824982Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2  
2025-03-14T04:45:45.2825846Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake  
2025-03-14T04:45:45.2826704Z   inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc  
2025-03-14T04:45:45.2827519Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt  
2025-03-14T04:45:45.2828256Z   inflating: build/custom_test_artifacts/custom-backend-build/Makefile  
2025-03-14T04:45:45.2829007Z   inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake  
2025-03-14T04:45:45.2927679Z   inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so  
2025-03-14T04:45:45.2969979Z   inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend  
2025-03-14T04:45:45.2970777Z    creating: build/lib/
2025-03-14T04:45:45.3058766Z   inflating: build/lib/libprotobuf-lite.a  
2025-03-14T04:45:45.3504086Z   inflating: build/lib/libprotobuf.a  
2025-03-14T04:45:45.3998511Z   inflating: build/lib/libprotoc.a   
2025-03-14T04:45:45.4007760Z   inflating: build/lib/libpthreadpool.a  
2025-03-14T04:45:45.4015633Z   inflating: build/lib/libcpuinfo.a  
2025-03-14T04:45:45.4023498Z   inflating: build/lib/libcpuinfo_internals.a  
2025-03-14T04:45:45.4024312Z   inflating: build/lib/libclog.a     
2025-03-14T04:45:45.4046145Z   inflating: build/lib/libpytorch_qnnpack.a  
2025-03-14T04:45:45.4046891Z   inflating: build/lib/libnnpack_reference_layers.a  
2025-03-14T04:45:45.4066596Z   inflating: build/lib/libnnpack.a   
2025-03-14T04:45:45.4249549Z   inflating: build/lib/libmicrokernels-prod.a  
2025-03-14T04:45:45.5026568Z   inflating: build/lib/libmicrokernels-all.a  
2025-03-14T04:45:45.5094504Z   inflating: build/lib/libgtest.a    
2025-03-14T04:45:45.5112752Z   inflating: build/lib/libgmock.a    
2025-03-14T04:45:45.5113729Z   inflating: build/lib/libgtest_main.a  
2025-03-14T04:45:45.5114232Z   inflating: build/lib/libgmock_main.a  
2025-03-14T04:45:45.5204179Z   inflating: build/lib/libXNNPACK.a  
2025-03-14T04:45:45.5277470Z   inflating: build/lib/libbenchmark.a  
2025-03-14T04:45:45.5278146Z   inflating: build/lib/libbenchmark_main.a  
2025-03-14T04:45:45.5284622Z   inflating: build/lib/libittnotify.a  
2025-03-14T04:45:45.5349298Z   inflating: build/lib/libasmjit.a   
2025-03-14T04:45:45.6544692Z   inflating: build/lib/libfbgemm.a   
2025-03-14T04:45:45.6573316Z   inflating: build/lib/libtensorpipe_uv.a  
2025-03-14T04:45:45.7123035Z   inflating: build/lib/libtensorpipe.a  
2025-03-14T04:45:45.7374544Z   inflating: build/lib/libtensorpipe_cuda.a  
2025-03-14T04:45:45.7500226Z   inflating: build/lib/libgloo.a     
2025-03-14T04:45:45.7541748Z   inflating: build/lib/libonnx_proto.a  
2025-03-14T04:45:45.8249028Z   inflating: build/lib/libonnx.a     
2025-03-14T04:45:45.8268462Z   inflating: build/lib/libfmt.a      
2025-03-14T04:45:46.8312995Z   inflating: build/lib/libdnnl.a     
2025-03-14T04:45:46.8767656Z   inflating: build/lib/libkineto.a   
2025-03-14T04:45:46.8881094Z   inflating: build/lib/libc10.so     
2025-03-14T04:45:46.8940505Z   inflating: build/lib/libc10_cuda.so  
2025-03-14T04:45:46.8941520Z   inflating: build/lib/libtorch_global_deps.so  
2025-03-14T04:45:46.8942141Z   inflating: build/lib/libcaffe2_nvrtc.so  
2025-03-14T04:45:49.3617717Z   inflating: build/lib/libtorch_cpu.so  
2025-03-14T04:45:49.3621035Z   inflating: build/lib/libunbox_lib.a  
2025-03-14T04:45:49.3625011Z   inflating: build/lib/libshm.so     
2025-03-14T04:45:49.3990653Z   inflating: build/lib/libgloo_cuda.a  
2025-03-14T04:45:51.4245473Z   inflating: build/lib/libtorch_cuda.so  
2025-03-14T04:45:51.4246074Z   inflating: build/lib/libtorch.so   
2025-03-14T04:45:52.2948788Z   inflating: build/lib/libtorch_cuda_linalg.so  
2025-03-14T04:45:52.3016495Z   inflating: build/lib/libtorchbind_test.so  
2025-03-14T04:45:52.3036842Z   inflating: build/lib/libjitbackend_test.so  
2025-03-14T04:45:52.3061917Z   inflating: build/lib/libbackend_with_compiler.so  
2025-03-14T04:45:52.3086718Z   inflating: build/lib/libaoti_custom_ops.so  
2025-03-14T04:45:52.3087758Z   inflating: build/lib/libc10d_cuda_test.so  
2025-03-14T04:45:52.5094097Z   inflating: build/lib/libtorch_python.so  
2025-03-14T04:45:52.5129513Z   inflating: build/lib/libnnapi_backend.so  
2025-03-14T04:45:52.5130045Z    creating: build/bin/
2025-03-14T04:45:52.5569829Z   inflating: build/bin/protoc-3.13.0.0  
2025-03-14T04:45:52.6008487Z   inflating: build/bin/protoc        
2025-03-14T04:45:52.6061393Z   inflating: build/bin/c10_CompileTimeFunctionPointer_test  
2025-03-14T04:45:52.6116319Z   inflating: build/bin/c10_DeviceGuard_test  
2025-03-14T04:45:52.6171173Z   inflating: build/bin/c10_Device_test  
2025-03-14T04:45:52.6233884Z   inflating: build/bin/c10_DispatchKeySet_test  
2025-03-14T04:45:52.6287048Z   inflating: build/bin/c10_StreamGuard_test  
2025-03-14T04:45:52.6341084Z   inflating: build/bin/c10_SymInt_test  
2025-03-14T04:45:52.6396969Z   inflating: build/bin/c10_Scalar_test  
2025-03-14T04:45:52.6456266Z   inflating: build/bin/c10_InlineStreamGuard_test  
2025-03-14T04:45:52.6515452Z   inflating: build/bin/c10_InlineDeviceGuard_test  
2025-03-14T04:45:52.6575876Z   inflating: build/bin/c10_SizesAndStrides_test  
2025-03-14T04:45:52.6650323Z   inflating: build/bin/c10_cow_test  
2025-03-14T04:45:52.6702461Z   inflating: build/bin/c10_ArrayRef_test  
2025-03-14T04:45:52.6754366Z   inflating: build/bin/c10_ConstexprCrc_test  
2025-03-14T04:45:52.6812605Z   inflating: build/bin/c10_Bitset_test  
2025-03-14T04:45:52.6865137Z   inflating: build/bin/c10_DeadlockDetection_test  
2025-03-14T04:45:52.6919204Z   inflating: build/bin/c10_Half_test  
2025-03-14T04:45:52.6977855Z   inflating: build/bin/c10_LeftRight_test  
2025-03-14T04:45:52.7036271Z   inflating: build/bin/c10_Metaprogramming_test  
2025-03-14T04:45:52.7091216Z   inflating: build/bin/c10_NetworkFlow_test  
2025-03-14T04:45:52.7147202Z   inflating: build/bin/c10_Synchronized_test  
2025-03-14T04:45:52.7200738Z   inflating: build/bin/c10_TypeTraits_test  
2025-03-14T04:45:52.7256374Z   inflating: build/bin/c10_TypeIndex_test  
2025-03-14T04:45:52.7319315Z   inflating: build/bin/c10_ThreadLocal_test  
2025-03-14T04:45:52.7375487Z   inflating: build/bin/c10_TypeList_test  
2025-03-14T04:45:52.7432521Z   inflating: build/bin/c10_accumulate_test  
2025-03-14T04:45:52.7493703Z   inflating: build/bin/c10_bfloat16_test  
2025-03-14T04:45:52.7555097Z   inflating: build/bin/c10_complex_math_test  
2025-03-14T04:45:52.7609691Z   inflating: build/bin/c10_bit_cast_test  
2025-03-14T04:45:52.7662329Z   inflating: build/bin/c10_error_test  
2025-03-14T04:45:52.7720447Z   inflating: build/bin/c10_complex_test  
2025-03-14T04:45:52.7776148Z   inflating: build/bin/c10_exception_test  
2025-03-14T04:45:52.7831523Z   inflating: build/bin/c10_flags_test  
2025-03-14T04:45:52.7884749Z   inflating: build/bin/c10_generic_math_test  
2025-03-14T04:45:52.7939051Z   inflating: build/bin/c10_irange_test  
2025-03-14T04:45:52.8103576Z   inflating: build/bin/c10_intrusive_ptr_test  
2025-03-14T04:45:52.8161182Z   inflating: build/bin/c10_lazy_test  
2025-03-14T04:45:52.8221265Z   inflating: build/bin/c10_logging_test  
2025-03-14T04:45:52.8286805Z   inflating: build/bin/c10_ordered_preserving_dict_test  
2025-03-14T04:45:52.8366022Z   inflating: build/bin/c10_optional_test  
2025-03-14T04:45:52.8422708Z   inflating: build/bin/c10_registry_test  
2025-03-14T04:45:52.8577225Z   inflating: build/bin/c10_small_vector_test  
2025-03-14T04:45:52.8632063Z   inflating: build/bin/c10_string_util_test  
2025-03-14T04:45:52.8686499Z   inflating: build/bin/c10_ssize_test  
2025-03-14T04:45:52.8738389Z   inflating: build/bin/c10_string_view_test  
2025-03-14T04:45:52.8791330Z   inflating: build/bin/c10_tempfile_test  
2025-03-14T04:45:52.8840705Z   inflating: build/bin/c10_intrusive_ptr_benchmark  
2025-03-14T04:45:52.8900538Z   inflating: build/bin/c10_typeid_test  
2025-03-14T04:45:52.8956252Z   inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device  
2025-03-14T04:45:52.9013583Z   inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads  
2025-03-14T04:45:52.9068662Z   inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test  
2025-03-14T04:45:52.9126314Z   inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream  
2025-03-14T04:45:52.9182227Z   inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes  
2025-03-14T04:45:52.9234845Z   inflating: build/bin/c10_cuda_CUDATest  
2025-03-14T04:45:52.9290392Z   inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block  
2025-03-14T04:45:52.9347224Z   inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks  
2025-03-14T04:45:52.9742109Z   inflating: build/bin/vec_test_all_types_DEFAULT  
2025-03-14T04:45:53.0148014Z   inflating: build/bin/vec_test_all_types_AVX512  
2025-03-14T04:45:53.0566651Z   inflating: build/bin/vec_test_all_types_AVX2  
2025-03-14T04:45:53.0626168Z   inflating: build/bin/TCPStoreTest  
2025-03-14T04:45:53.0681551Z   inflating: build/bin/BackoffTest   
2025-03-14T04:45:53.0737465Z   inflating: build/bin/FileStoreTest  
2025-03-14T04:45:53.0793684Z   inflating: build/bin/test_edge_op_registration  
2025-03-14T04:45:53.0850783Z   inflating: build/bin/HashStoreTest  
2025-03-14T04:45:53.0865121Z   inflating: build/bin/ProcessGroupMPITest  
2025-03-14T04:45:53.0867801Z   inflating: build/bin/torch_shm_manager  
2025-03-14T04:45:53.0945585Z   inflating: build/bin/Dict_test     
2025-03-14T04:45:53.0999850Z   inflating: build/bin/Dimname_test  
2025-03-14T04:45:53.1067501Z   inflating: build/bin/MaybeOwned_test  
2025-03-14T04:45:53.1130298Z   inflating: build/bin/NamedTensor_test  
2025-03-14T04:45:53.1192312Z   inflating: build/bin/apply_utils_test  
2025-03-14T04:45:53.1255066Z   inflating: build/bin/atest         
2025-03-14T04:45:53.1323373Z   inflating: build/bin/basic         
2025-03-14T04:45:53.1381626Z   inflating: build/bin/broadcast_test  
2025-03-14T04:45:53.1436623Z   inflating: build/bin/cpu_allocator_test  
2025-03-14T04:45:53.1498466Z   inflating: build/bin/cpu_generator_test  
2025-03-14T04:45:53.1557923Z   inflating: build/bin/cpu_profiling_allocator_test  
2025-03-14T04:45:53.1653185Z   inflating: build/bin/cpu_rng_test  
2025-03-14T04:45:53.1706223Z   inflating: build/bin/dispatch_key_set_test  
2025-03-14T04:45:53.1760336Z   inflating: build/bin/dlconvertor_test  
2025-03-14T04:45:53.1822297Z   inflating: build/bin/extension_backend_test  
2025-03-14T04:45:53.1880924Z   inflating: build/bin/half_test     
2025-03-14T04:45:53.1981744Z   inflating: build/bin/ivalue_test   
2025-03-14T04:45:53.2035265Z   inflating: build/bin/lazy_tensor_test  
2025-03-14T04:45:53.2092247Z   inflating: build/bin/math_kernel_test  
2025-03-14T04:45:53.2150871Z   inflating: build/bin/memory_format_test  
2025-03-14T04:45:53.2208522Z   inflating: build/bin/memory_overlapping_test  
2025-03-14T04:45:53.2264784Z   inflating: build/bin/mobile_memory_cleanup  
2025-03-14T04:45:53.2327048Z   inflating: build/bin/native_test   
2025-03-14T04:45:53.2381833Z   inflating: build/bin/operator_name_test  
2025-03-14T04:45:53.2437100Z   inflating: build/bin/operators_test  
2025-03-14T04:45:53.2491981Z   inflating: build/bin/packedtensoraccessor_test  
2025-03-14T04:45:53.2561500Z   inflating: build/bin/pow_test      
2025-03-14T04:45:53.2623615Z   inflating: build/bin/quantized_test  
2025-03-14T04:45:53.2676625Z   inflating: build/bin/reduce_ops_test  
2025-03-14T04:45:53.2731581Z   inflating: build/bin/reportMemoryUsage_test  
2025-03-14T04:45:53.2791285Z   inflating: build/bin/scalar_tensor_test  
2025-03-14T04:45:53.2854948Z   inflating: build/bin/scalar_test   
2025-03-14T04:45:53.2911062Z   inflating: build/bin/StorageUtils_test  
2025-03-14T04:45:53.2967026Z   inflating: build/bin/stride_properties_test  
2025-03-14T04:45:53.3050067Z   inflating: build/bin/tensor_iterator_test  
2025-03-14T04:45:53.3108782Z   inflating: build/bin/test_parallel  
2025-03-14T04:45:53.3110503Z   inflating: build/bin/thread_init_test  
2025-03-14T04:45:53.3170695Z   inflating: build/bin/type_ptr_test  
2025-03-14T04:45:53.3234670Z   inflating: build/bin/type_test     
2025-03-14T04:45:53.3290307Z   inflating: build/bin/undefined_tensor_test  
2025-03-14T04:45:53.3290823Z   inflating: build/bin/verify_api_visibility  
2025-03-14T04:45:53.3366589Z   inflating: build/bin/legacy_vmap_test  
2025-03-14T04:45:53.3422396Z   inflating: build/bin/weakref_test  
2025-03-14T04:45:53.3477149Z   inflating: build/bin/wrapdim_test  
2025-03-14T04:45:53.3533125Z   inflating: build/bin/xla_tensor_test  
2025-03-14T04:45:53.3596468Z   inflating: build/bin/IListRef_test  
2025-03-14T04:45:53.3704415Z   inflating: build/bin/List_test     
2025-03-14T04:45:53.3773325Z   inflating: build/bin/KernelFunction_test  
2025-03-14T04:45:53.3874284Z   inflating: build/bin/kernel_function_test  
2025-03-14T04:45:53.3998555Z   inflating: build/bin/kernel_function_legacy_test  
2025-03-14T04:45:53.4130254Z   inflating: build/bin/kernel_lambda_legacy_test  
2025-03-14T04:45:53.4235475Z   inflating: build/bin/kernel_lambda_test  
2025-03-14T04:45:53.4300492Z   inflating: build/bin/kernel_stackbased_test  
2025-03-14T04:45:53.4400105Z   inflating: build/bin/make_boxed_from_unboxed_functor_test  
2025-03-14T04:45:53.4454659Z   inflating: build/bin/CppSignature_test  
2025-03-14T04:45:53.4515054Z   inflating: build/bin/backend_fallback_test  
2025-03-14T04:45:53.4567162Z   inflating: build/bin/op_allowlist_test  
2025-03-14T04:45:53.4864749Z   inflating: build/bin/op_registration_test  
2025-03-14T04:45:53.4931948Z   inflating: build/bin/inline_container_test  
2025-03-14T04:45:53.4987336Z   inflating: build/bin/cuda_allocator_test  
2025-03-14T04:45:53.5043974Z   inflating: build/bin/cuda_apply_test  
2025-03-14T04:45:53.5108257Z   inflating: build/bin/cuda_atomic_ops_test  
2025-03-14T04:45:53.5171227Z   inflating: build/bin/cuda_caching_host_allocator_test  
2025-03-14T04:45:53.5244717Z   inflating: build/bin/cuda_complex_math_test  
2025-03-14T04:45:53.5309486Z   inflating: build/bin/cuda_complex_test  
2025-03-14T04:45:53.5374213Z   inflating: build/bin/cuda_cub_test  
2025-03-14T04:45:53.5430101Z   inflating: build/bin/cuda_device_test  
2025-03-14T04:45:53.5501041Z   inflating: build/bin/cuda_distributions_test  
2025-03-14T04:45:53.5557901Z   inflating: build/bin/cuda_dlconvertor_test  
2025-03-14T04:45:53.5615956Z   inflating: build/bin/cuda_reportMemoryUsage_test  
2025-03-14T04:45:53.5669045Z   inflating: build/bin/cuda_allocatorTraceTracker_test  
2025-03-14T04:45:53.5735150Z   inflating: build/bin/cuda_stream_test  
2025-03-14T04:45:53.5790011Z   inflating: build/bin/cuda_integer_divider_test  
2025-03-14T04:45:53.5842761Z   inflating: build/bin/cuda_cudnn_test  
2025-03-14T04:45:53.5905727Z   inflating: build/bin/cuda_generator_test  
2025-03-14T04:45:53.5957896Z   inflating: build/bin/cuda_optional_test  
2025-03-14T04:45:53.6012654Z   inflating: build/bin/cuda_packedtensoraccessor_test  
2025-03-14T04:45:53.6065252Z   inflating: build/bin/cuda_half_test  
2025-03-14T04:45:53.6120885Z   inflating: build/bin/cuda_vectorized_test  
2025-03-14T04:45:53.6694307Z   inflating: build/bin/test_jit      
2025-03-14T04:45:53.6710053Z   inflating: build/bin/tutorial_tensorexpr  
2025-03-14T04:45:53.7533878Z   inflating: build/bin/test_tensorexpr  
2025-03-14T04:45:53.7535543Z   inflating: build/bin/example_allreduce  
2025-03-14T04:45:53.7594504Z   inflating: build/bin/test_dist_autograd  
2025-03-14T04:45:53.7666099Z   inflating: build/bin/test_cpp_rpc  
2025-03-14T04:45:53.7734597Z   inflating: build/bin/ProcessGroupGlooTest  
2025-03-14T04:45:53.7795231Z   inflating: build/bin/ProcessGroupGlooAsyncTest  
2025-03-14T04:45:53.7859733Z   inflating: build/bin/ProcessGroupNCCLErrorsTest  
2025-03-14T04:45:53.7926865Z   inflating: build/bin/ProcessGroupNCCLTest  
2025-03-14T04:45:53.7927990Z   inflating: build/bin/parallel_benchmark  
2025-03-14T04:45:53.9107462Z   inflating: build/bin/test_api      
2025-03-14T04:45:53.9177591Z   inflating: build/bin/test_mobile_nnc  
2025-03-14T04:45:53.9185411Z   inflating: build/bin/aot_model_compiler_test  
2025-03-14T04:45:53.9530164Z   inflating: build/bin/test_lazy     
2025-03-14T04:45:53.9531050Z    creating: .additional_ci_files/
2025-03-14T04:45:53.9635374Z   inflating: .additional_ci_files/test-times.json  
2025-03-14T04:45:54.0043323Z   inflating: .additional_ci_files/test-class-times.json  
2025-03-14T04:45:54.0081601Z ##[group]Run rm artifacts.zip
2025-03-14T04:45:54.0081989Z rm artifacts.zip
2025-03-14T04:45:54.0088262Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:45:54.0088719Z env:
2025-03-14T04:45:54.0088987Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:45:54.0089382Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:45:54.0089906Z ##[endgroup]
2025-03-14T04:45:54.0722364Z ##[group]Run df -H
2025-03-14T04:45:54.0722696Z df -H
2025-03-14T04:45:54.0727260Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:45:54.0727719Z env:
2025-03-14T04:45:54.0728005Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:45:54.0728407Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:45:54.0728996Z ##[endgroup]
2025-03-14T04:45:54.0767523Z Filesystem        Size  Used Avail Use% Mounted on
2025-03-14T04:45:54.0768213Z devtmpfs          4.2M     0  4.2M   0% /dev
2025-03-14T04:45:54.0768664Z tmpfs             203G     0  203G   0% /dev/shm
2025-03-14T04:45:54.0769108Z tmpfs              82G   11M   82G   1% /run
2025-03-14T04:45:54.0769601Z /dev/nvme0n1p1    161G   67G   95G  42% /
2025-03-14T04:45:54.0770073Z tmpfs             203G   25k  203G   1% /tmp
2025-03-14T04:45:54.0770521Z /dev/nvme0n1p128   11M  1.4M  9.2M  13% /boot/efi
2025-03-14T04:45:54.0770980Z tmpfs              41G     0   41G   0% /run/user/0
2025-03-14T04:45:54.0808624Z Prepare all required actions
2025-03-14T04:45:54.0809114Z Getting action download info
2025-03-14T04:45:54.2115031Z ##[group]Run ./.github/actions/download-td-artifacts
2025-03-14T04:45:54.2130270Z with:
2025-03-14T04:45:54.2130595Z env:
2025-03-14T04:45:54.2130866Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:45:54.2131294Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:45:54.2131937Z ##[endgroup]
2025-03-14T04:45:54.2169000Z ##[group]Run seemethere/download-artifact-s3@v4
2025-03-14T04:45:54.2169418Z with:
2025-03-14T04:45:54.2169712Z   name: td_results
2025-03-14T04:45:54.2170051Z   s3-bucket: gha-artifacts
2025-03-14T04:45:54.2170421Z   region: us-east-1
2025-03-14T04:45:54.2170739Z env:
2025-03-14T04:45:54.2171039Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:45:54.2171474Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:45:54.2171948Z ##[endgroup]
2025-03-14T04:45:54.6610799Z (node:86003) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.
2025-03-14T04:45:54.6612354Z 
2025-03-14T04:45:54.6612996Z Please migrate your code to use AWS SDK for JavaScript (v3).
2025-03-14T04:45:54.6614681Z For more information, check the migration guide at https://a.co/7PzMCcy
2025-03-14T04:45:54.6616433Z (Use `node --trace-warnings ...` to show where the warning was created)
2025-03-14T04:45:54.7582190Z Found 1 objects with prefix pytorch/pytorch/13849515290/td_results/
2025-03-14T04:45:54.7584306Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json
2025-03-14T04:45:54.8334447Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json
2025-03-14T04:45:54.8339009Z Artifact download has finished successfully
2025-03-14T04:45:54.8541372Z ##[group]Run mkdir -p .additional_ci_files
2025-03-14T04:45:54.8541803Z mkdir -p .additional_ci_files
2025-03-14T04:45:54.8542338Z mv td_results.json .additional_ci_files/td_results.json || true
2025-03-14T04:45:54.8548346Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:45:54.8548804Z env:
2025-03-14T04:45:54.8549079Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:45:54.8549477Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:45:54.8549885Z ##[endgroup]
2025-03-14T04:45:54.8628334Z ##[group]Run .github/scripts/parse_ref.py
2025-03-14T04:45:54.8628790Z .github/scripts/parse_ref.py
2025-03-14T04:45:54.8633255Z shell: /usr/bin/bash -e {0}
2025-03-14T04:45:54.8633563Z env:
2025-03-14T04:45:54.8633817Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:45:54.8634202Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:45:54.8634608Z ##[endgroup]
2025-03-14T04:45:54.8961692Z Prepare all required actions
2025-03-14T04:45:54.8962460Z Getting action download info
2025-03-14T04:45:55.2353205Z ##[group]Run ./.github/actions/filter-test-configs
2025-03-14T04:45:55.2353632Z with:
2025-03-14T04:45:55.2354157Z   github-token: ***
2025-03-14T04:45:55.2354787Z   test-matrix: {"include": [{"config": "pr_time_benchmarks", "shard": 1, "num_shards": 1, "runner": "linux.g4dn.metal.nvidia.gpu"}]}
2025-03-14T04:45:55.2355715Z   job-name: cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu)
2025-03-14T04:45:55.2356393Z env:
2025-03-14T04:45:55.2356680Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:45:55.2357113Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:45:55.2357552Z ##[endgroup]
2025-03-14T04:45:55.2399135Z ##[group]Run nick-fields/retry@v3.0.0
2025-03-14T04:45:55.2399537Z with:
2025-03-14T04:45:55.2399821Z   shell: bash
2025-03-14T04:45:55.2400127Z   timeout_minutes: 10
2025-03-14T04:45:55.2400546Z   max_attempts: 5
2025-03-14T04:45:55.2400878Z   retry_wait_seconds: 30
2025-03-14T04:45:55.2401961Z   command: set -eux
# PyYAML 6.0 doesn't work with MacOS x86 anymore
# This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2
python3 -m pip install requests==2.27.1 pyyaml==6.0.1

2025-03-14T04:45:55.2402915Z   polling_interval_seconds: 1
2025-03-14T04:45:55.2403269Z   warning_on_retry: true
2025-03-14T04:45:55.2403607Z   continue_on_error: false
2025-03-14T04:45:55.2403944Z env:
2025-03-14T04:45:55.2404205Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:45:55.2404613Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:45:55.2405412Z   GITHUB_TOKEN: ***
2025-03-14T04:45:55.2405699Z ##[endgroup]
2025-03-14T04:45:55.3331371Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1
2025-03-14T04:45:55.5630132Z Defaulting to user installation because normal site-packages is not writeable
2025-03-14T04:45:55.5777647Z Requirement already satisfied: requests==2.27.1 in /home/ec2-user/.local/lib/python3.9/site-packages (2.27.1)
2025-03-14T04:45:55.5780088Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.9/site-packages (6.0.1)
2025-03-14T04:45:55.5890884Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2025.1.31)
2025-03-14T04:45:55.5899381Z Requirement already satisfied: charset-normalizer~=2.0.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2.0.12)
2025-03-14T04:45:55.5909457Z Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (2.10)
2025-03-14T04:45:55.5914672Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (1.25.10)
2025-03-14T04:45:56.3179193Z Command completed after 1 attempt(s).
2025-03-14T04:45:56.3237330Z ##[group]Run set -x
2025-03-14T04:45:56.3237707Z set -x
2025-03-14T04:45:56.3237979Z 
2025-03-14T04:45:56.3238433Z # Use relative path here as this could be checked out anywhere, not necessarily
2025-03-14T04:45:56.3238985Z # in runner workspace
2025-03-14T04:45:56.3239430Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py"
2025-03-14T04:45:56.3244943Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:45:56.3245382Z env:
2025-03-14T04:45:56.3245641Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:45:56.3246017Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:45:56.3246425Z ##[endgroup]
2025-03-14T04:45:56.3269723Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py
2025-03-14T04:45:56.3527584Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}"
2025-03-14T04:45:56.3528064Z echo "Workflow: ${GITHUB_WORKFLOW}"
2025-03-14T04:45:56.3528491Z echo "Job name: ${JOB_NAME}"
2025-03-14T04:45:56.3528858Z 
2025-03-14T04:45:56.3529494Z # Use relative path here as this could be checked out anywhere, not necessarily
2025-03-14T04:45:56.3530075Z # in runner workspace
2025-03-14T04:45:56.3530583Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \
2025-03-14T04:45:56.3531159Z   --workflow "${GITHUB_WORKFLOW}" \
2025-03-14T04:45:56.3531580Z   --job-name "${JOB_NAME}" \
2025-03-14T04:45:56.3532318Z   --test-matrix "{"include": [{"config": "pr_time_benchmarks", "shard": 1, "num_shards": 1, "runner": "linux.g4dn.metal.nvidia.gpu"}]}" \
2025-03-14T04:45:56.3533055Z   --selected-test-configs "" \
2025-03-14T04:45:56.3533446Z   --pr-number "${PR_NUMBER}" \
2025-03-14T04:45:56.3533839Z   --tag "${TAG}" \
2025-03-14T04:45:56.3534164Z   --event-name "${EVENT_NAME}" \
2025-03-14T04:45:56.3534538Z   --schedule "${SCHEDULE}" \
2025-03-14T04:45:56.3534903Z   --branch "${HEAD_BRANCH}"
2025-03-14T04:45:56.3539444Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:45:56.3539877Z env:
2025-03-14T04:45:56.3540136Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:45:56.3540548Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:45:56.3541216Z   GITHUB_TOKEN: ***
2025-03-14T04:45:56.3541763Z   JOB_NAME: cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu)
2025-03-14T04:45:56.3542363Z   PR_NUMBER: 
2025-03-14T04:45:56.3542651Z   TAG: 
2025-03-14T04:45:56.3542917Z   EVENT_NAME: push
2025-03-14T04:45:56.3543191Z   SCHEDULE: 
2025-03-14T04:45:56.3543442Z   HEAD_BRANCH: 
2025-03-14T04:45:56.3543712Z ##[endgroup]
2025-03-14T04:45:56.3564157Z Workflow: pull
2025-03-14T04:45:56.3564790Z Job name: cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu)
2025-03-14T04:45:56.5378045Z ##[group]Run echo "Filtered matrix:"
2025-03-14T04:45:56.5378419Z echo "Filtered matrix:"
2025-03-14T04:45:56.5379081Z echo "{"include": [{"config": "pr_time_benchmarks", "shard": 1, "num_shards": 1, "runner": "linux.g4dn.metal.nvidia.gpu"}]}"
2025-03-14T04:45:56.5379725Z 
2025-03-14T04:45:56.5379967Z echo
2025-03-14T04:45:56.5380285Z echo "Is the current job unstable? False"
2025-03-14T04:45:56.5380666Z 
2025-03-14T04:45:56.5380904Z echo
2025-03-14T04:45:56.5381200Z echo "Is keep-going label set? False"
2025-03-14T04:45:56.5381562Z 
2025-03-14T04:45:56.5381796Z echo
2025-03-14T04:45:56.5382059Z echo "Renabled issues? "
2025-03-14T04:45:56.5386660Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:45:56.5387094Z env:
2025-03-14T04:45:56.5387340Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:45:56.5387715Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:45:56.5388111Z ##[endgroup]
2025-03-14T04:45:56.5408305Z Filtered matrix:
2025-03-14T04:45:56.5409171Z {include: [{config: pr_time_benchmarks, shard: 1, num_shards: 1, runner: linux.g4dn.metal.nvidia.gpu}]}
2025-03-14T04:45:56.5409860Z 
2025-03-14T04:45:56.5410004Z Is the current job unstable? False
2025-03-14T04:45:56.5410260Z 
2025-03-14T04:45:56.5410434Z Is keep-going label set? False
2025-03-14T04:45:56.5410664Z 
2025-03-14T04:45:56.5410818Z Renabled issues? 
2025-03-14T04:45:56.5456460Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}"
2025-03-14T04:45:56.5457046Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}"
2025-03-14T04:45:56.5462372Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:45:56.5462819Z env:
2025-03-14T04:45:56.5463075Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:45:56.5463454Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:45:56.5463855Z   JOB_TIMEOUT: 240
2025-03-14T04:45:56.5464122Z ##[endgroup]
2025-03-14T04:45:56.5539843Z ##[group]Run set -x
2025-03-14T04:45:56.5540238Z set -x
2025-03-14T04:45:56.5540503Z 
2025-03-14T04:45:56.5540792Z if [[ $TEST_CONFIG == 'multigpu' ]]; then
2025-03-14T04:45:56.5541234Z   TEST_COMMAND=.ci/pytorch/multigpu-test.sh
2025-03-14T04:45:56.5541680Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then
2025-03-14T04:45:56.5542090Z   TEST_COMMAND=.ci/onnx/test.sh
2025-03-14T04:45:56.5542433Z else
2025-03-14T04:45:56.5542720Z   TEST_COMMAND=.ci/pytorch/test.sh
2025-03-14T04:45:56.5543058Z fi
2025-03-14T04:45:56.5543295Z 
2025-03-14T04:45:56.5543592Z # Leaving 1GB for the runner and other things
2025-03-14T04:45:56.5544229Z TOTAL_AVAILABLE_MEMORY_IN_GB=$(awk '/MemTotal/ { printf "%.3f \n", $2/1024/1024 - 1 }' /proc/meminfo)
2025-03-14T04:45:56.5545196Z # https://docs.docker.com/engine/containers/resource_constraints/#--memory-swap-details, the 3GB swap
2025-03-14T04:45:56.5545970Z # comes from https://github.com/pytorch/test-infra/pull/6058
2025-03-14T04:45:56.5546557Z TOTAL_MEMORY_WITH_SWAP=$(("${TOTAL_AVAILABLE_MEMORY_IN_GB%.*}" + 3))
2025-03-14T04:45:56.5547021Z 
2025-03-14T04:45:56.5547322Z if [[ ${BUILD_ENVIRONMENT} == *"s390x"* ]]; then
2025-03-14T04:45:56.5547713Z   SHM_OPTS=
2025-03-14T04:45:56.5548001Z   JENKINS_USER=
2025-03-14T04:45:56.5548398Z   # ensure that docker container cleanly exits in 12 hours
2025-03-14T04:45:56.5548917Z   # if for some reason cleanup action doesn't stop container
2025-03-14T04:45:56.5549364Z   # when job is cancelled
2025-03-14T04:45:56.5549713Z   DOCKER_SHELL_CMD="sleep 12h"
2025-03-14T04:45:56.5550198Z 
2025-03-14T04:45:56.5550615Z   # since some steps are skipped on s390x, if they are necessary, run them here
2025-03-14T04:45:56.5551221Z   env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}"
2025-03-14T04:45:56.5551715Z   env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}"
2025-03-14T04:45:56.5552108Z else
2025-03-14T04:45:56.5552391Z   SHM_OPTS="--shm-size=${SHM_SIZE}"
2025-03-14T04:45:56.5552764Z   JENKINS_USER="--user jenkins"
2025-03-14T04:45:56.5553116Z   DOCKER_SHELL_CMD=
2025-03-14T04:45:56.5553408Z fi
2025-03-14T04:45:56.5553641Z 
2025-03-14T04:45:56.5554020Z # detached container should get cleaned up by teardown_ec2_linux
2025-03-14T04:45:56.5554606Z # TODO: Stop building test binaries as part of the build phase
2025-03-14T04:45:56.5555280Z # Used for GPU_FLAG, SHM_OPTS, JENKINS_USER and DOCKER_SHELL_CMD since that doesn't play nice
2025-03-14T04:45:56.5555873Z # shellcheck disable=SC2086,SC2090
2025-03-14T04:45:56.5556251Z container_name=$(docker run \
2025-03-14T04:45:56.5556611Z   ${GPU_FLAG:-} \
2025-03-14T04:45:56.5556955Z   ${SCCACHE_SERVER_PORT_DOCKER_FLAG:-} \
2025-03-14T04:45:56.5557338Z   -e BUILD_ENVIRONMENT \
2025-03-14T04:45:56.5557662Z   -e PR_NUMBER \
2025-03-14T04:45:56.5557970Z   -e GITHUB_ACTIONS \
2025-03-14T04:45:56.5558287Z   -e GITHUB_REPOSITORY \
2025-03-14T04:45:56.5558618Z   -e GITHUB_WORKFLOW \
2025-03-14T04:45:56.5558941Z   -e GITHUB_JOB \
2025-03-14T04:45:56.5559243Z   -e GITHUB_RUN_ID \
2025-03-14T04:45:56.5559556Z   -e GITHUB_RUN_NUMBER \
2025-03-14T04:45:56.5559897Z   -e GITHUB_RUN_ATTEMPT \
2025-03-14T04:45:56.5560236Z   -e JOB_ID \
2025-03-14T04:45:56.5560532Z   -e JOB_NAME \
2025-03-14T04:45:56.5560833Z   -e BASE_SHA \
2025-03-14T04:45:56.5561129Z   -e BRANCH \
2025-03-14T04:45:56.5561426Z   -e SHA1 \
2025-03-14T04:45:56.5561721Z   -e AWS_DEFAULT_REGION \
2025-03-14T04:45:56.5562155Z   -e IN_WHEEL_TEST \
2025-03-14T04:45:56.5562480Z   -e SHARD_NUMBER \
2025-03-14T04:45:56.5562795Z   -e TEST_CONFIG \
2025-03-14T04:45:56.5563103Z   -e NUM_TEST_SHARDS \
2025-03-14T04:45:56.5563434Z   -e REENABLED_ISSUES \
2025-03-14T04:45:56.5563777Z   -e CONTINUE_THROUGH_ERROR \
2025-03-14T04:45:56.5564118Z   -e VERBOSE_TEST_LOGS \
2025-03-14T04:45:56.5564448Z   -e TEST_SHOWLOCALS \
2025-03-14T04:45:56.5564759Z   -e NO_TEST_TIMEOUT \
2025-03-14T04:45:56.5565066Z   -e NO_TD \
2025-03-14T04:45:56.5565343Z   -e TD_DISTRIBUTED \
2025-03-14T04:45:56.5565653Z   -e PR_LABELS \
2025-03-14T04:45:56.5565976Z   -e MAX_JOBS="$(nproc --ignore=2)" \
2025-03-14T04:45:56.5566339Z   -e SCCACHE_BUCKET \
2025-03-14T04:45:56.5566657Z   -e SCCACHE_REGION \
2025-03-14T04:45:56.5566962Z   -e XLA_CUDA \
2025-03-14T04:45:56.5567279Z   -e XLA_CLANG_CACHE_S3_BUCKET_NAME \
2025-03-14T04:45:56.5567676Z   -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \
2025-03-14T04:45:56.5568085Z   -e PYTORCH_TEST_RERUN_DISABLED_TESTS \
2025-03-14T04:45:56.5568479Z   -e SKIP_SCCACHE_INITIALIZATION=1 \
2025-03-14T04:45:56.5568853Z   -e HUGGING_FACE_HUB_TOKEN \
2025-03-14T04:45:56.5569219Z   -e SCRIBE_GRAPHQL_ACCESS_TOKEN \
2025-03-14T04:45:56.5569572Z   -e DASHBOARD_TAG \
2025-03-14T04:45:56.5569882Z   -e IS_A100_RUNNER \
2025-03-14T04:45:56.5570204Z   -e ARTIFACTS_FILE_SUFFIX \
2025-03-14T04:45:56.5570602Z   --memory="${TOTAL_AVAILABLE_MEMORY_IN_GB%.*}g" \
2025-03-14T04:45:56.5571045Z   --memory-swap="${TOTAL_MEMORY_WITH_SWAP}g" \
2025-03-14T04:45:56.5571498Z   --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \
2025-03-14T04:45:56.5572002Z   --security-opt seccomp=unconfined \
2025-03-14T04:45:56.5572379Z   --cap-add=SYS_PTRACE \
2025-03-14T04:45:56.5572707Z   --ipc=host \
2025-03-14T04:45:56.5572990Z   ${SHM_OPTS} \
2025-03-14T04:45:56.5573272Z   --tty \
2025-03-14T04:45:56.5573540Z   --detach \
2025-03-14T04:45:56.5573834Z   --name="${container_name}" \
2025-03-14T04:45:56.5574179Z   ${JENKINS_USER} \
2025-03-14T04:45:56.5574562Z   -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \
2025-03-14T04:45:56.5575001Z   -w /var/lib/jenkins/workspace \
2025-03-14T04:45:56.5575359Z   "${DOCKER_IMAGE}" \
2025-03-14T04:45:56.5575675Z   ${DOCKER_SHELL_CMD}
2025-03-14T04:45:56.5575973Z )
2025-03-14T04:45:56.5576297Z # Propagate download.pytorch.org IP to container
2025-03-14T04:45:56.5577022Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts"
2025-03-14T04:45:56.5577796Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}"
2025-03-14T04:45:56.5578252Z 
2025-03-14T04:45:56.5578550Z if [[ ${BUILD_ENVIRONMENT} == *"s390x"* ]]; then
2025-03-14T04:45:56.5579186Z   docker exec -t "${container_name}" sh -c "python3 -m pip install -r .ci/docker/requirements-ci.txt"
2025-03-14T04:45:56.5579756Z fi
2025-03-14T04:45:56.5580003Z 
2025-03-14T04:45:56.5580547Z docker exec -t "${container_name}" sh -c "python3 -m pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}"
2025-03-14T04:45:56.5585585Z shell: /usr/bin/bash -e {0}
2025-03-14T04:45:56.5585888Z env:
2025-03-14T04:45:56.5586132Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:45:56.5586500Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:45:56.5586977Z   BUILD_ENVIRONMENT: linux-focal-cuda12.4-py3.10-gcc9-sm75
2025-03-14T04:45:56.5587400Z   PR_NUMBER: 
2025-03-14T04:45:56.5587673Z   GITHUB_REPOSITORY: pytorch/pytorch
2025-03-14T04:45:56.5588019Z   GITHUB_WORKFLOW: pull
2025-03-14T04:45:56.5588300Z   GITHUB_JOB: test
2025-03-14T04:45:56.5588658Z   GITHUB_RUN_ID: 13849515290
2025-03-14T04:45:56.5588959Z   GITHUB_RUN_NUMBER: 299021
2025-03-14T04:45:56.5589258Z   GITHUB_RUN_ATTEMPT: 1
2025-03-14T04:45:56.5589537Z   JOB_ID: 38755508671
2025-03-14T04:45:56.5590280Z   JOB_NAME: cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu)
2025-03-14T04:45:56.5590898Z   BRANCH: main
2025-03-14T04:45:56.5591224Z   SHA1: aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:45:56.5591681Z   BASE_SHA: aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:45:56.5592117Z   TEST_CONFIG: pr_time_benchmarks
2025-03-14T04:45:56.5592549Z   SHARD_NUMBER: 1
2025-03-14T04:45:56.5592839Z   NUM_TEST_SHARDS: 1
2025-03-14T04:45:56.5593126Z   REENABLED_ISSUES: 
2025-03-14T04:45:56.5593443Z   CONTINUE_THROUGH_ERROR: False
2025-03-14T04:45:56.5593806Z   VERBOSE_TEST_LOGS: False
2025-03-14T04:45:56.5594139Z   TEST_SHOWLOCALS: False
2025-03-14T04:45:56.5594467Z   NO_TEST_TIMEOUT: False
2025-03-14T04:45:56.5594777Z   NO_TD: False
2025-03-14T04:45:56.5595068Z   TD_DISTRIBUTED: False
2025-03-14T04:45:56.5595448Z   SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2
2025-03-14T04:45:56.5595892Z   SCCACHE_REGION: us-east-1
2025-03-14T04:45:56.5596212Z   SHM_SIZE: 2g
2025-03-14T04:45:56.5597189Z   DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:45:56.5598250Z   XLA_CUDA: 
2025-03-14T04:45:56.5598686Z   XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla
2025-03-14T04:45:56.5599238Z   PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0
2025-03-14T04:45:56.5599637Z   PYTORCH_TEST_RERUN_DISABLED_TESTS: 0
2025-03-14T04:45:56.5600005Z   DASHBOARD_TAG: 
2025-03-14T04:45:56.5600495Z   HUGGING_FACE_HUB_TOKEN: ***
2025-03-14T04:45:56.5601151Z   SCRIBE_GRAPHQL_ACCESS_TOKEN: ***
2025-03-14T04:45:56.5601509Z   IS_A100_RUNNER: 0
2025-03-14T04:45:56.5602171Z   ARTIFACTS_FILE_SUFFIX: test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_38755508671
2025-03-14T04:45:56.5602739Z ##[endgroup]
2025-03-14T04:45:56.5624500Z + [[ pr_time_benchmarks == \m\u\l\t\i\g\p\u ]]
2025-03-14T04:45:56.5625008Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm75 == *onnx* ]]
2025-03-14T04:45:56.5625467Z + TEST_COMMAND=.ci/pytorch/test.sh
2025-03-14T04:45:56.5627445Z ++ awk '/MemTotal/ { printf "%.3f \n", $2/1024/1024 - 1 }' /proc/meminfo
2025-03-14T04:45:56.5647155Z + TOTAL_AVAILABLE_MEMORY_IN_GB='376.508 '
2025-03-14T04:45:56.5648293Z + TOTAL_MEMORY_WITH_SWAP=379
2025-03-14T04:45:56.5649452Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm75 == *\s\3\9\0\x* ]]
2025-03-14T04:45:56.5650703Z + SHM_OPTS=--shm-size=2g
2025-03-14T04:45:56.5651578Z + JENKINS_USER='--user jenkins'
2025-03-14T04:45:56.5652460Z + DOCKER_SHELL_CMD=
2025-03-14T04:45:56.5655694Z +++ nproc --ignore=2
2025-03-14T04:45:56.5670113Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=94 -e SCCACHE_BUCKET -e SCCACHE_REGION -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e SCRIBE_GRAPHQL_ACCESS_TOKEN -e DASHBOARD_TAG -e IS_A100_RUNNER -e ARTIFACTS_FILE_SUFFIX --memory=376g --memory-swap=379g --env-file=/tmp/github_env_13849515290 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:46:08.6239504Z + container_name=6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:46:08.6243146Z + grep download.pytorch.org /etc/hosts
2025-03-14T04:46:08.6244792Z + docker exec -i 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73 sudo bash -c '/bin/cat >> /etc/hosts'
2025-03-14T04:46:08.7728226Z + echo DOCKER_CONTAINER_ID=6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:46:08.7729003Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm75 == *\s\3\9\0\x* ]]
2025-03-14T04:46:08.7732120Z ++ echo dist/torch-2.8.0a0+gitaed0b7a-cp310-cp310-linux_x86_64.whl
2025-03-14T04:46:08.7737104Z + docker exec -t 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73 sh -c 'python3 -m pip install dist/torch-2.8.0a0+gitaed0b7a-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh'
2025-03-14T04:46:09.2106360Z Processing ./dist/torch-2.8.0a0+gitaed0b7a-cp310-cp310-linux_x86_64.whl (from torch==2.8.0a0+gitaed0b7a)
2025-03-14T04:46:10.1161214Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.8.0a0+gitaed0b7a->torch==2.8.0a0+gitaed0b7a) (3.16.1)
2025-03-14T04:46:10.1162877Z Requirement already satisfied: typing-extensions>=4.10.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.8.0a0+gitaed0b7a->torch==2.8.0a0+gitaed0b7a) (4.12.2)
2025-03-14T04:46:10.1666327Z Collecting sympy>=1.13.3 (from torch==2.8.0a0+gitaed0b7a->torch==2.8.0a0+gitaed0b7a)
2025-03-14T04:46:10.1680421Z   Using cached sympy-1.13.3-py3-none-any.whl.metadata (12 kB)
2025-03-14T04:46:10.1700624Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.8.0a0+gitaed0b7a->torch==2.8.0a0+gitaed0b7a) (2.8.8)
2025-03-14T04:46:10.1702600Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.8.0a0+gitaed0b7a->torch==2.8.0a0+gitaed0b7a) (3.1.6)
2025-03-14T04:46:10.1704150Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.8.0a0+gitaed0b7a->torch==2.8.0a0+gitaed0b7a) (2024.10.0)
2025-03-14T04:46:10.1725300Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.8.0a0+gitaed0b7a->torch==2.8.0a0+gitaed0b7a) (3.3.0)
2025-03-14T04:46:10.1745301Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.8.0a0+gitaed0b7a->torch==2.8.0a0+gitaed0b7a) (1.22.4)
2025-03-14T04:46:10.1750418Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy>=1.13.3->torch==2.8.0a0+gitaed0b7a->torch==2.8.0a0+gitaed0b7a) (1.3.0)
2025-03-14T04:46:10.2125219Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.8.0a0+gitaed0b7a->torch==2.8.0a0+gitaed0b7a) (3.0.2)
2025-03-14T04:46:10.2206753Z Using cached sympy-1.13.3-py3-none-any.whl (6.2 MB)
2025-03-14T04:46:10.8360676Z Installing collected packages: sympy, torch
2025-03-14T04:46:10.8361901Z   Attempting uninstall: sympy
2025-03-14T04:46:10.8373172Z     Found existing installation: sympy 1.13.1
2025-03-14T04:46:10.9649719Z     Uninstalling sympy-1.13.1:
2025-03-14T04:46:11.6259730Z       Successfully uninstalled sympy-1.13.1
2025-03-14T04:46:25.7082379Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
2025-03-14T04:46:25.7083680Z timm 1.0.14 requires torchvision, which is not installed.
2025-03-14T04:46:25.7084386Z Successfully installed sympy-1.13.3 torch-2.8.0a0+gitaed0b7a
2025-03-14T04:46:25.8098633Z + export TERM=vt100
2025-03-14T04:46:25.8099476Z + TERM=vt100
2025-03-14T04:46:25.8100907Z ++ dirname .ci/pytorch/test.sh
2025-03-14T04:46:25.8109412Z + source .ci/pytorch/common.sh
2025-03-14T04:46:25.8110386Z +++ dirname .ci/pytorch/common.sh
2025-03-14T04:46:25.8119710Z ++ source .ci/pytorch/common_utils.sh
2025-03-14T04:46:25.8120214Z +++ declare -f -t trap_add
2025-03-14T04:46:25.8129328Z ++ set -ex -o pipefail
2025-03-14T04:46:25.8129789Z ++ [[ linux-focal-cuda12.4-py3.10-gcc9-sm75 == *rocm* ]]
2025-03-14T04:46:25.8130245Z ++ BUILD_TEST_LIBTORCH=0
2025-03-14T04:46:25.8130678Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm75 != *rocm* ]]
2025-03-14T04:46:25.8131182Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm75 != *s390x* ]]
2025-03-14T04:46:25.8131666Z + [[ -d /var/lib/jenkins/workspace ]]
2025-03-14T04:46:25.8132072Z ++ stat -c %u /var/lib/jenkins/workspace
2025-03-14T04:46:25.8146974Z + WORKSPACE_ORIGINAL_OWNER_ID=1000
2025-03-14T04:46:25.8147495Z + trap_add cleanup_workspace EXIT
2025-03-14T04:46:25.8147925Z + trap_add_cmd=cleanup_workspace
2025-03-14T04:46:25.8148302Z + shift
2025-03-14T04:46:25.8148616Z + for trap_add_name in "$@"
2025-03-14T04:46:25.8152438Z +++ trap -p EXIT
2025-03-14T04:46:25.8152803Z ++ eval 'extract_trap_cmd '
2025-03-14T04:46:25.8153155Z +++ extract_trap_cmd
2025-03-14T04:46:25.8153480Z +++ printf '%s\n' ''
2025-03-14T04:46:25.8156074Z ++ printf '%s\n' cleanup_workspace
2025-03-14T04:46:25.8156474Z + trap -- '
2025-03-14T04:46:25.8156784Z cleanup_workspace' EXIT
2025-03-14T04:46:25.8157162Z + sudo chown -R jenkins /var/lib/jenkins/workspace
2025-03-14T04:46:26.2949182Z + git config --global --add safe.directory /var/lib/jenkins/workspace
2025-03-14T04:46:26.2967901Z + echo 'Environment variables:'
2025-03-14T04:46:26.2968338Z Environment variables:
2025-03-14T04:46:26.2968675Z + env
2025-03-14T04:46:26.2978086Z INSTALLED_DB=yes
2025-03-14T04:46:26.2978676Z NV_LIBCUBLAS_VERSION=12.4.5.8-1
2025-03-14T04:46:26.2979446Z NVIDIA_VISIBLE_DEVICES=all
2025-03-14T04:46:26.2979816Z NV_NVML_DEV_VERSION=12.4.127-1
2025-03-14T04:46:26.2980351Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch
2025-03-14T04:46:26.2980929Z CONTINUE_THROUGH_ERROR=False
2025-03-14T04:46:26.2981342Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.21.5-1+cuda12.4
2025-03-14T04:46:26.2981809Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.21.5-1
2025-03-14T04:46:26.2982285Z BUILD_ENVIRONMENT=linux-focal-cuda12.4-py3.10-gcc9-sm75
2025-03-14T04:46:26.2982737Z HOSTNAME=6761c124b60f
2025-03-14T04:46:26.2983443Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_afae4307-1b02-4888-936b-deab1dfa8330
2025-03-14T04:46:26.2984224Z GITHUB_ACTION=__self
2025-03-14T04:46:26.2984560Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0
2025-03-14T04:46:26.2989627Z NVIDIA_REQUIRE_CUDA=cuda>=12.4 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 brand=tesla,driver>=535,driver<536 brand=unknown,driver>=535,driver<536 brand=nvidia,driver>=535,driver<536 brand=nvidiartx,driver>=535,driver<536 brand=geforce,driver>=535,driver<536 brand=geforcertx,driver>=535,driver<536 brand=quadro,driver>=535,driver<536 brand=quadrortx,driver>=535,driver<536 brand=titan,driver>=535,driver<536 brand=titanrtx,driver>=535,driver<536
2025-03-14T04:46:26.2995360Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-4=12.4.5.8-1
2025-03-14T04:46:26.2995809Z NV_NVTX_VERSION=12.4.127-1
2025-03-14T04:46:26.2996137Z GITHUB_RUN_NUMBER=299021
2025-03-14T04:46:26.2996454Z TEST_CONFIG=pr_time_benchmarks
2025-03-14T04:46:26.2996955Z GITHUB_REPOSITORY_OWNER_ID=21003710
2025-03-14T04:46:26.2997360Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all
2025-03-14T04:46:26.2997745Z IS_A100_RUNNER=0
2025-03-14T04:46:26.2998055Z NV_CUDA_CUDART_DEV_VERSION=12.4.127-1
2025-03-14T04:46:26.2998445Z NV_LIBCUSPARSE_VERSION=12.3.1.170-1
2025-03-14T04:46:26.2999073Z SCRIBE_GRAPHQL_ACCESS_TOKEN=***
2025-03-14T04:46:26.2999435Z NV_LIBNPP_VERSION=12.2.5.30-1
2025-03-14T04:46:26.2999831Z GITHUB_TRIGGERING_ACTOR=pytorchmergebot
2025-03-14T04:46:26.3000298Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache
2025-03-14T04:46:26.3000747Z GITHUB_REF_TYPE=branch
2025-03-14T04:46:26.3001105Z TORCH_CUDA_ARCH_LIST=Maxwell
2025-03-14T04:46:26.3001473Z NCCL_VERSION=2.21.5-1
2025-03-14T04:46:26.3001855Z BASE_SHA=aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:46:26.3002392Z XLA_CUDA=
2025-03-14T04:46:26.3002806Z HUGGING_FACE_HUB_TOKEN=***
2025-03-14T04:46:26.3003354Z ***
2025-03-14T04:46:26.3003642Z CARGO_NET_GIT_FETCH_WITH_CLI=true
2025-03-14T04:46:26.3004016Z GITHUB_REPOSITORY_ID=65600975
2025-03-14T04:46:26.3004351Z GITHUB_ACTIONS=true
2025-03-14T04:46:26.3004637Z NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:46:26.3005024Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-4=12.4.127-1
2025-03-14T04:46:26.3005458Z NV_LIBNPP_PACKAGE=libnpp-12-4=12.2.5.30-1
2025-03-14T04:46:26.3005875Z SHA1=aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:46:26.3006282Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev
2025-03-14T04:46:26.3006701Z GITHUB_SHA=aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:46:26.3007314Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/heads/main
2025-03-14T04:46:26.3007851Z UCC_HOME=/usr
2025-03-14T04:46:26.3008135Z NV_LIBCUBLAS_DEV_VERSION=12.4.5.8-1
2025-03-14T04:46:26.3008490Z VERBOSE_TEST_LOGS=False
2025-03-14T04:46:26.3008912Z NVIDIA_PRODUCT_NAME=CUDA
2025-03-14T04:46:26.3009281Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-4
2025-03-14T04:46:26.3009695Z GITHUB_REF=refs/heads/main
2025-03-14T04:46:26.3010030Z NV_CUDA_CUDART_VERSION=12.4.127-1
2025-03-14T04:46:26.3010364Z SHARD_NUMBER=1
2025-03-14T04:46:26.3010643Z GITHUB_REF_PROTECTED=true
2025-03-14T04:46:26.3010965Z HOME=/var/lib/jenkins
2025-03-14T04:46:26.3011297Z GITHUB_API_URL=https://api.github.com
2025-03-14T04:46:26.3011692Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0
2025-03-14T04:46:26.3012108Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb
2025-03-14T04:46:26.3012513Z CUDA_VERSION=12.4.1
2025-03-14T04:46:26.3012834Z NV_LIBCUBLAS_PACKAGE=libcublas-12-4=12.4.5.8-1
2025-03-14T04:46:26.3013218Z NUM_TEST_SHARDS=1
2025-03-14T04:46:26.3013489Z UCX_HOME=/usr
2025-03-14T04:46:26.3013895Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-4=12.4.1-1
2025-03-14T04:46:26.3014769Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_afae4307-1b02-4888-936b-deab1dfa8330
2025-03-14T04:46:26.3015754Z JOB_NAME=cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu)
2025-03-14T04:46:26.3016713Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_afae4307-1b02-4888-936b-deab1dfa8330
2025-03-14T04:46:26.3017646Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json
2025-03-14T04:46:26.3018232Z GITHUB_EVENT_NAME=push
2025-03-14T04:46:26.3018529Z DASHBOARD_TAG=
2025-03-14T04:46:26.3018806Z GITHUB_RUN_ID=13849515290
2025-03-14T04:46:26.3019168Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-4=12.2.5.30-1
2025-03-14T04:46:26.3019636Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-4
2025-03-14T04:46:26.3020438Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_afae4307-1b02-4888-936b-deab1dfa8330
2025-03-14T04:46:26.3021241Z GITHUB_ACTOR=pytorchmergebot
2025-03-14T04:46:26.3021591Z NV_LIBNPP_DEV_VERSION=12.2.5.30-1
2025-03-14T04:46:26.3021940Z PR_NUMBER=
2025-03-14T04:46:26.3022202Z GITHUB_RUN_ATTEMPT=1
2025-03-14T04:46:26.3022524Z ANACONDA_PYTHON_VERSION=3.10
2025-03-14T04:46:26.3022988Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql
2025-03-14T04:46:26.3023402Z TERM=vt100
2025-03-14T04:46:26.3023690Z NV_LIBCUSPARSE_DEV_VERSION=12.3.1.170-1
2025-03-14T04:46:26.3024063Z INSTALLED_VISION=yes
2025-03-14T04:46:26.3024353Z BRANCH=main
2025-03-14T04:46:26.3024622Z SCCACHE_REGION=us-east-1
2025-03-14T04:46:26.3024950Z OPENSSL_ROOT_DIR=/opt/openssl
2025-03-14T04:46:26.3025309Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs
2025-03-14T04:46:26.3025686Z CUDA_PATH=/usr/local/cuda
2025-03-14T04:46:26.3026298Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux
2025-03-14T04:46:26.3026969Z GITHUB_SERVER_URL=https://github.com
2025-03-14T04:46:26.3027390Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b
2025-03-14T04:46:26.3027802Z REENABLED_ISSUES=
2025-03-14T04:46:26.3028074Z SHLVL=1
2025-03-14T04:46:26.3028320Z MAX_JOBS=94
2025-03-14T04:46:26.3028577Z NV_CUDA_LIB_VERSION=12.4.1-1
2025-03-14T04:46:26.3028891Z NVARCH=x86_64
2025-03-14T04:46:26.3029163Z GITHUB_ACTOR_ID=97764156
2025-03-14T04:46:26.3029561Z GITHUB_WORKFLOW_SHA=aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:46:26.3030011Z GITHUB_REF_NAME=main
2025-03-14T04:46:26.3030331Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-4
2025-03-14T04:46:26.3030850Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla
2025-03-14T04:46:26.3031345Z GITHUB_JOB=test
2025-03-14T04:46:26.3031650Z NV_LIBNCCL_PACKAGE=libnccl2=2.21.5-1+cuda12.4
2025-03-14T04:46:26.3032130Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64
2025-03-14T04:46:26.3032579Z NO_TEST_TIMEOUT=False
2025-03-14T04:46:26.3032868Z TD_DISTRIBUTED=False
2025-03-14T04:46:26.3033183Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.4.1-1
2025-03-14T04:46:26.3033567Z GITHUB_REPOSITORY=pytorch/pytorch
2025-03-14T04:46:26.3033925Z NV_NVPROF_VERSION=12.4.127-1
2025-03-14T04:46:26.3034252Z GITHUB_RETENTION_DAYS=90
2025-03-14T04:46:26.3034610Z OPENSSL_DIR=/opt/openssl
2025-03-14T04:46:26.3034930Z GITHUB_ACTION_REPOSITORY=
2025-03-14T04:46:26.3035826Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2025-03-14T04:46:26.3036742Z GITHUB_BASE_REF=
2025-03-14T04:46:26.3037267Z ARTIFACTS_FILE_SUFFIX=test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_38755508671
2025-03-14T04:46:26.3037889Z NV_LIBNCCL_PACKAGE_NAME=libnccl2
2025-03-14T04:46:26.3038217Z CI=true
2025-03-14T04:46:26.3038483Z NV_LIBNCCL_PACKAGE_VERSION=2.21.5-1
2025-03-14T04:46:26.3038852Z GITHUB_REPOSITORY_OWNER=pytorch
2025-03-14T04:46:26.3039179Z JOB_ID=38755508671
2025-03-14T04:46:26.3039461Z INSTALLED_PROTOBUF=yes
2025-03-14T04:46:26.3039757Z GITHUB_HEAD_REF=
2025-03-14T04:46:26.3040034Z GITHUB_ACTION_REF=
2025-03-14T04:46:26.3040382Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2
2025-03-14T04:46:26.3040900Z TEST_SHOWLOCALS=False
2025-03-14T04:46:26.3041186Z GITHUB_WORKFLOW=pull
2025-03-14T04:46:26.3041479Z DEBIAN_FRONTEND=noninteractive
2025-03-14T04:46:26.3042157Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_afae4307-1b02-4888-936b-deab1dfa8330
2025-03-14T04:46:26.3042851Z NO_TD=False
2025-03-14T04:46:26.3043113Z SKIP_SCCACHE_INITIALIZATION=1
2025-03-14T04:46:26.3043425Z _=/usr/bin/env
2025-03-14T04:46:26.3043774Z ++ python -c 'import site; print(site.getsitepackages()[0])'
2025-03-14T04:46:26.3153985Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch
2025-03-14T04:46:26.3154761Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin
2025-03-14T04:46:26.3155468Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib
2025-03-14T04:46:26.3156183Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test
2025-03-14T04:46:26.3156731Z + BUILD_DIR=build
2025-03-14T04:46:26.3157039Z + BUILD_RENAMED_DIR=build_renamed
2025-03-14T04:46:26.3157417Z + BUILD_BIN_DIR=build/bin
2025-03-14T04:46:26.3157732Z + SHARD_NUMBER=1
2025-03-14T04:46:26.3158015Z + NUM_TEST_SHARDS=1
2025-03-14T04:46:26.3158518Z + export TORCH_SERIALIZATION_DEBUG=1
2025-03-14T04:46:26.3158909Z + TORCH_SERIALIZATION_DEBUG=1
2025-03-14T04:46:26.3159252Z + export VALGRIND=ON
2025-03-14T04:46:26.3159550Z + VALGRIND=ON
2025-03-14T04:46:26.3159914Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm75 == *clang9* ]]
2025-03-14T04:46:26.3160428Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm75 == *xpu* ]]
2025-03-14T04:46:26.3160916Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm75 == *s390x* ]]
2025-03-14T04:46:26.3161335Z + [[ 0 == \1 ]]
2025-03-14T04:46:26.3161603Z + [[ False == \1 ]]
2025-03-14T04:46:26.3161991Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm75 != *bazel* ]]
2025-03-14T04:46:26.3162457Z ++ realpath build/custom_test_artifacts
2025-03-14T04:46:26.3167819Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts
2025-03-14T04:46:26.3168438Z + [[ -n '' ]]
2025-03-14T04:46:26.3168737Z + echo 'Environment variables'
2025-03-14T04:46:26.3169074Z Environment variables
2025-03-14T04:46:26.3169369Z + env
2025-03-14T04:46:26.3173439Z INSTALLED_DB=yes
2025-03-14T04:46:26.3173798Z NV_LIBCUBLAS_VERSION=12.4.5.8-1
2025-03-14T04:46:26.3174161Z NVIDIA_VISIBLE_DEVICES=all
2025-03-14T04:46:26.3174500Z NV_NVML_DEV_VERSION=12.4.127-1
2025-03-14T04:46:26.3174995Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch
2025-03-14T04:46:26.3175531Z CONTINUE_THROUGH_ERROR=False
2025-03-14T04:46:26.3175936Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.21.5-1+cuda12.4
2025-03-14T04:46:26.3176406Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.21.5-1
2025-03-14T04:46:26.3176881Z BUILD_ENVIRONMENT=linux-focal-cuda12.4-py3.10-gcc9-sm75
2025-03-14T04:46:26.3177336Z HOSTNAME=6761c124b60f
2025-03-14T04:46:26.3178028Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_afae4307-1b02-4888-936b-deab1dfa8330
2025-03-14T04:46:26.3178967Z GITHUB_ACTION=__self
2025-03-14T04:46:26.3179297Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0
2025-03-14T04:46:26.3184303Z NVIDIA_REQUIRE_CUDA=cuda>=12.4 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 brand=tesla,driver>=535,driver<536 brand=unknown,driver>=535,driver<536 brand=nvidia,driver>=535,driver<536 brand=nvidiartx,driver>=535,driver<536 brand=geforce,driver>=535,driver<536 brand=geforcertx,driver>=535,driver<536 brand=quadro,driver>=535,driver<536 brand=quadrortx,driver>=535,driver<536 brand=titan,driver>=535,driver<536 brand=titanrtx,driver>=535,driver<536
2025-03-14T04:46:26.3189614Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-4=12.4.5.8-1
2025-03-14T04:46:26.3190126Z NV_NVTX_VERSION=12.4.127-1
2025-03-14T04:46:26.3190455Z GITHUB_RUN_NUMBER=299021
2025-03-14T04:46:26.3190787Z TEST_CONFIG=pr_time_benchmarks
2025-03-14T04:46:26.3191149Z GITHUB_REPOSITORY_OWNER_ID=21003710
2025-03-14T04:46:26.3191541Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all
2025-03-14T04:46:26.3191915Z IS_A100_RUNNER=0
2025-03-14T04:46:26.3192220Z NV_CUDA_CUDART_DEV_VERSION=12.4.127-1
2025-03-14T04:46:26.3192602Z NV_LIBCUSPARSE_VERSION=12.3.1.170-1
2025-03-14T04:46:26.3193173Z SCRIBE_GRAPHQL_ACCESS_TOKEN=***
2025-03-14T04:46:26.3193540Z NV_LIBNPP_VERSION=12.2.5.30-1
2025-03-14T04:46:26.3193932Z GITHUB_TRIGGERING_ACTOR=pytorchmergebot
2025-03-14T04:46:26.3194378Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache
2025-03-14T04:46:26.3194830Z GITHUB_REF_TYPE=branch
2025-03-14T04:46:26.3195167Z TORCH_CUDA_ARCH_LIST=Maxwell
2025-03-14T04:46:26.3195690Z NCCL_VERSION=2.21.5-1
2025-03-14T04:46:26.3196069Z BASE_SHA=aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:46:26.3196492Z XLA_CUDA=
2025-03-14T04:46:26.3196904Z HUGGING_FACE_HUB_TOKEN=***
2025-03-14T04:46:26.3197308Z ***
2025-03-14T04:46:26.3197584Z CARGO_NET_GIT_FETCH_WITH_CLI=true
2025-03-14T04:46:26.3197944Z GITHUB_REPOSITORY_ID=65600975
2025-03-14T04:46:26.3198297Z GITHUB_ACTIONS=true
2025-03-14T04:46:26.3198638Z NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:46:26.3199046Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-4=12.4.127-1
2025-03-14T04:46:26.3199507Z NV_LIBNPP_PACKAGE=libnpp-12-4=12.2.5.30-1
2025-03-14T04:46:26.3199963Z SHA1=aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:46:26.3200419Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev
2025-03-14T04:46:26.3200875Z GITHUB_SHA=aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:46:26.3201499Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/heads/main
2025-03-14T04:46:26.3202066Z UCC_HOME=/usr
2025-03-14T04:46:26.3202386Z TORCH_SERIALIZATION_DEBUG=1
2025-03-14T04:46:26.3202742Z NV_LIBCUBLAS_DEV_VERSION=12.4.5.8-1
2025-03-14T04:46:26.3203133Z VERBOSE_TEST_LOGS=False
2025-03-14T04:46:26.3203479Z NVIDIA_PRODUCT_NAME=CUDA
2025-03-14T04:46:26.3203880Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-4
2025-03-14T04:46:26.3204300Z GITHUB_REF=refs/heads/main
2025-03-14T04:46:26.3204685Z NV_CUDA_CUDART_VERSION=12.4.127-1
2025-03-14T04:46:26.3205048Z SHARD_NUMBER=1
2025-03-14T04:46:26.3205336Z GITHUB_REF_PROTECTED=true
2025-03-14T04:46:26.3205657Z HOME=/var/lib/jenkins
2025-03-14T04:46:26.3206022Z GITHUB_API_URL=https://api.github.com
2025-03-14T04:46:26.3206408Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0
2025-03-14T04:46:26.3206840Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb
2025-03-14T04:46:26.3207262Z CUDA_VERSION=12.4.1
2025-03-14T04:46:26.3207715Z NV_LIBCUBLAS_PACKAGE=libcublas-12-4=12.4.5.8-1
2025-03-14T04:46:26.3208134Z NUM_TEST_SHARDS=1
2025-03-14T04:46:26.3208446Z UCX_HOME=/usr
2025-03-14T04:46:26.3208897Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-4=12.4.1-1
2025-03-14T04:46:26.3209822Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_afae4307-1b02-4888-936b-deab1dfa8330
2025-03-14T04:46:26.3210862Z JOB_NAME=cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu)
2025-03-14T04:46:26.3211837Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_afae4307-1b02-4888-936b-deab1dfa8330
2025-03-14T04:46:26.3212795Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json
2025-03-14T04:46:26.3213395Z GITHUB_EVENT_NAME=push
2025-03-14T04:46:26.3213723Z DASHBOARD_TAG=
2025-03-14T04:46:26.3214055Z GITHUB_RUN_ID=13849515290
2025-03-14T04:46:26.3214440Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-4=12.2.5.30-1
2025-03-14T04:46:26.3214900Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-4
2025-03-14T04:46:26.3215728Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_afae4307-1b02-4888-936b-deab1dfa8330
2025-03-14T04:46:26.3216547Z GITHUB_ACTOR=pytorchmergebot
2025-03-14T04:46:26.3216929Z NV_LIBNPP_DEV_VERSION=12.2.5.30-1
2025-03-14T04:46:26.3217300Z PR_NUMBER=
2025-03-14T04:46:26.3217577Z GITHUB_RUN_ATTEMPT=1
2025-03-14T04:46:26.3217898Z VALGRIND=ON
2025-03-14T04:46:26.3218212Z ANACONDA_PYTHON_VERSION=3.10
2025-03-14T04:46:26.3218656Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql
2025-03-14T04:46:26.3219116Z TERM=vt100
2025-03-14T04:46:26.3219443Z NV_LIBCUSPARSE_DEV_VERSION=12.3.1.170-1
2025-03-14T04:46:26.3219856Z INSTALLED_VISION=yes
2025-03-14T04:46:26.3220189Z BRANCH=main
2025-03-14T04:46:26.3220507Z SCCACHE_REGION=us-east-1
2025-03-14T04:46:26.3220880Z OPENSSL_ROOT_DIR=/opt/openssl
2025-03-14T04:46:26.3221287Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs
2025-03-14T04:46:26.3221718Z CUDA_PATH=/usr/local/cuda
2025-03-14T04:46:26.3222380Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux
2025-03-14T04:46:26.3223178Z GITHUB_SERVER_URL=https://github.com
2025-03-14T04:46:26.3223643Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b
2025-03-14T04:46:26.3224097Z REENABLED_ISSUES=
2025-03-14T04:46:26.3224387Z SHLVL=1
2025-03-14T04:46:26.3224675Z MAX_JOBS=94
2025-03-14T04:46:26.3224986Z NV_CUDA_LIB_VERSION=12.4.1-1
2025-03-14T04:46:26.3225350Z NVARCH=x86_64
2025-03-14T04:46:26.3225666Z GITHUB_ACTOR_ID=97764156
2025-03-14T04:46:26.3226117Z GITHUB_WORKFLOW_SHA=aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:46:26.3226610Z GITHUB_REF_NAME=main
2025-03-14T04:46:26.3226972Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-4
2025-03-14T04:46:26.3227536Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla
2025-03-14T04:46:26.3228075Z GITHUB_JOB=test
2025-03-14T04:46:26.3228482Z NV_LIBNCCL_PACKAGE=libnccl2=2.21.5-1+cuda12.4
2025-03-14T04:46:26.3244277Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64
2025-03-14T04:46:26.3244895Z NO_TEST_TIMEOUT=False
2025-03-14T04:46:26.3245203Z TD_DISTRIBUTED=False
2025-03-14T04:46:26.3245580Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.4.1-1
2025-03-14T04:46:26.3245993Z GITHUB_REPOSITORY=pytorch/pytorch
2025-03-14T04:46:26.3246374Z NV_NVPROF_VERSION=12.4.127-1
2025-03-14T04:46:26.3246719Z GITHUB_RETENTION_DAYS=90
2025-03-14T04:46:26.3247048Z OPENSSL_DIR=/opt/openssl
2025-03-14T04:46:26.3247383Z GITHUB_ACTION_REPOSITORY=
2025-03-14T04:46:26.3248316Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2025-03-14T04:46:26.3249265Z GITHUB_BASE_REF=
2025-03-14T04:46:26.3249808Z ARTIFACTS_FILE_SUFFIX=test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_38755508671
2025-03-14T04:46:26.3250449Z NV_LIBNCCL_PACKAGE_NAME=libnccl2
2025-03-14T04:46:26.3250798Z CI=true
2025-03-14T04:46:26.3251077Z NV_LIBNCCL_PACKAGE_VERSION=2.21.5-1
2025-03-14T04:46:26.3251571Z GITHUB_REPOSITORY_OWNER=pytorch
2025-03-14T04:46:26.3251916Z JOB_ID=38755508671
2025-03-14T04:46:26.3252209Z INSTALLED_PROTOBUF=yes
2025-03-14T04:46:26.3252524Z GITHUB_HEAD_REF=
2025-03-14T04:46:26.3252796Z GITHUB_ACTION_REF=
2025-03-14T04:46:26.3253158Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2
2025-03-14T04:46:26.3253590Z TEST_SHOWLOCALS=False
2025-03-14T04:46:26.3253899Z GITHUB_WORKFLOW=pull
2025-03-14T04:46:26.3254219Z DEBIAN_FRONTEND=noninteractive
2025-03-14T04:46:26.3254970Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_afae4307-1b02-4888-936b-deab1dfa8330
2025-03-14T04:46:26.3255732Z NO_TD=False
2025-03-14T04:46:26.3256017Z SKIP_SCCACHE_INITIALIZATION=1
2025-03-14T04:46:26.3256357Z _=/usr/bin/env
2025-03-14T04:46:26.3256641Z + echo 'Testing pytorch'
2025-03-14T04:46:26.3256957Z Testing pytorch
2025-03-14T04:46:26.3257252Z + export LANG=C.UTF-8
2025-03-14T04:46:26.3257547Z + LANG=C.UTF-8
2025-03-14T04:46:26.3257821Z + PR_NUMBER=
2025-03-14T04:46:26.3258118Z + [[ pr_time_benchmarks == \d\e\f\a\u\l\t ]]
2025-03-14T04:46:26.3258552Z + [[ pr_time_benchmarks == \d\i\s\t\r\i\b\u\t\e\d ]]
2025-03-14T04:46:26.3258974Z + [[ pr_time_benchmarks == \s\l\o\w ]]
2025-03-14T04:46:26.3259467Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm75 == *slow-gradcheck* ]]
2025-03-14T04:46:26.3260028Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm75 == *cuda* ]]
2025-03-14T04:46:26.3260502Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda
2025-03-14T04:46:26.3260928Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda
2025-03-14T04:46:26.3261326Z + [[ pr_time_benchmarks == *crossref* ]]
2025-03-14T04:46:26.3261766Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm75 == *rocm* ]]
2025-03-14T04:46:26.3262260Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm75 == *xpu* ]]
2025-03-14T04:46:26.3262772Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm75 != *-bazel-* ]]
2025-03-14T04:46:26.3263235Z + pip_install --user ninja==1.10.2
2025-03-14T04:46:26.3263705Z + pip_install_pkg='python3 -m pip install --progress-bar off'
2025-03-14T04:46:26.3264288Z + python3 -m pip install --progress-bar off --user ninja==1.10.2
2025-03-14T04:46:26.8493089Z Collecting ninja==1.10.2
2025-03-14T04:46:26.8884130Z   Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB)
2025-03-14T04:46:26.9007564Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB)
2025-03-14T04:46:27.4801358Z Installing collected packages: ninja
2025-03-14T04:46:27.4876492Z   WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH.
2025-03-14T04:46:27.4877917Z   Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
2025-03-14T04:46:27.4941553Z Successfully installed ninja-1.10.2
2025-03-14T04:46:27.5835334Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2025-03-14T04:46:27.5837843Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2025-03-14T04:46:27.5839027Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm75 == *aarch64* ]]
2025-03-14T04:46:27.5839483Z + install_tlparse
2025-03-14T04:46:27.5839906Z + pip_install --user tlparse==0.3.30
2025-03-14T04:46:27.5840376Z + pip_install_pkg='python3 -m pip install --progress-bar off'
2025-03-14T04:46:27.5840954Z + python3 -m pip install --progress-bar off --user tlparse==0.3.30
2025-03-14T04:46:28.0426423Z Collecting tlparse==0.3.30
2025-03-14T04:46:28.0814106Z   Downloading tlparse-0.3.30-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.9 kB)
2025-03-14T04:46:28.0924822Z Downloading tlparse-0.3.30-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.3 MB)
2025-03-14T04:46:28.7039790Z Installing collected packages: tlparse
2025-03-14T04:46:28.7384489Z Successfully installed tlparse-0.3.30
2025-03-14T04:46:28.8370159Z ++ python -m site --user-base
2025-03-14T04:46:28.8547366Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2025-03-14T04:46:28.8548713Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm75 == *asan* ]]
2025-03-14T04:46:28.8550310Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm75 == *-debug* ]]
2025-03-14T04:46:28.8550851Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm75 != *-bazel-* ]]
2025-03-14T04:46:28.8551571Z + echo 'We are not in debug mode: linux-focal-cuda12.4-py3.10-gcc9-sm75. Expect the assertion to pass'
2025-03-14T04:46:28.8552444Z We are not in debug mode: linux-focal-cuda12.4-py3.10-gcc9-sm75. Expect the assertion to pass
2025-03-14T04:46:28.8554677Z + cd test
2025-03-14T04:46:28.8555128Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)'
2025-03-14T04:46:30.5016799Z + [[ pr_time_benchmarks == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]]
2025-03-14T04:46:30.5017398Z + [[ pr_time_benchmarks == \n\o\g\p\u\_\A\V\X\5\1\2 ]]
2025-03-14T04:46:30.5022851Z + DYNAMO_BENCHMARK_FLAGS=()
2025-03-14T04:46:30.5023296Z + [[ pr_time_benchmarks == *pr_time_benchmarks* ]]
2025-03-14T04:46:30.5023728Z + pr_time_benchmarks
2025-03-14T04:46:30.5024063Z + pip_install --user fbscribelogger
2025-03-14T04:46:30.5024548Z + pip_install_pkg='python3 -m pip install --progress-bar off'
2025-03-14T04:46:30.5025147Z + python3 -m pip install --progress-bar off --user fbscribelogger
2025-03-14T04:46:30.8891294Z Requirement already satisfied: fbscribelogger in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (0.1.7)
2025-03-14T04:46:30.8907987Z Requirement already satisfied: aiohttp<4.0.0,>=3.10.4 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from fbscribelogger) (3.11.13)
2025-03-14T04:46:30.8909300Z Requirement already satisfied: thriftpy2<0.6.0,>=0.5.2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from fbscribelogger) (0.5.2)
2025-03-14T04:46:30.8912190Z Requirement already satisfied: typing-extensions<5.0.0,>=4.12.2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from fbscribelogger) (4.12.2)
2025-03-14T04:46:30.8948415Z Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.10.4->fbscribelogger) (2.6.1)
2025-03-14T04:46:30.8949953Z Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.10.4->fbscribelogger) (1.3.2)
2025-03-14T04:46:30.8954803Z Requirement already satisfied: async-timeout<6.0,>=4.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.10.4->fbscribelogger) (5.0.1)
2025-03-14T04:46:30.8958893Z Requirement already satisfied: attrs>=17.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.10.4->fbscribelogger) (25.3.0)
2025-03-14T04:46:30.8962967Z Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.10.4->fbscribelogger) (1.5.0)
2025-03-14T04:46:30.8967209Z Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.10.4->fbscribelogger) (6.1.0)
2025-03-14T04:46:30.8971281Z Requirement already satisfied: propcache>=0.2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.10.4->fbscribelogger) (0.3.0)
2025-03-14T04:46:30.8975565Z Requirement already satisfied: yarl<2.0,>=1.17.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.10.4->fbscribelogger) (1.18.3)
2025-03-14T04:46:30.8994277Z Requirement already satisfied: Cython>=3.0.10 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from thriftpy2<0.6.0,>=0.5.2->fbscribelogger) (3.0.12)
2025-03-14T04:46:30.8996416Z Requirement already satisfied: ply<4.0,>=3.4 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from thriftpy2<0.6.0,>=0.5.2->fbscribelogger) (3.11)
2025-03-14T04:46:30.9001770Z Requirement already satisfied: six~=1.15 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from thriftpy2<0.6.0,>=0.5.2->fbscribelogger) (1.17.0)
2025-03-14T04:46:30.9191959Z Requirement already satisfied: idna>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from yarl<2.0,>=1.17.0->aiohttp<4.0.0,>=3.10.4->fbscribelogger) (3.10)
2025-03-14T04:46:31.5415788Z ++ pwd
2025-03-14T04:46:31.5416913Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports
2025-03-14T04:46:31.5418412Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports
2025-03-14T04:46:31.5438590Z ++ pwd
2025-03-14T04:46:31.5439950Z + PYTHONPATH=/var/lib/jenkins/workspace/benchmarks/dynamo/pr_time_benchmarks
2025-03-14T04:46:31.5441564Z + source benchmarks/dynamo/pr_time_benchmarks/benchmark_runner.sh /var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv benchmarks/dynamo/pr_time_benchmarks/benchmarks
2025-03-14T04:46:31.5442762Z ++ '[' 2 -eq 0 ']'
2025-03-14T04:46:31.5443070Z ++ '[' 2 -eq 1 ']'
2025-03-14T04:46:31.5443666Z ++ output_file=/var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv
2025-03-14T04:46:31.5444462Z ++ python_programs_dir=benchmarks/dynamo/pr_time_benchmarks/benchmarks
2025-03-14T04:46:31.5445000Z +++ date +%s
2025-03-14T04:46:31.5454413Z ++ start=1741927591
2025-03-14T04:46:31.5455655Z ++ for file in $python_programs_dir/*.py
2025-03-14T04:46:31.5456673Z ++ '[' benchmarks/dynamo/pr_time_benchmarks/benchmarks/add_loop.py '!=' benchmarks/dynamo/pr_time_benchmarks/benchmarks/benchmark_base.py ']'
2025-03-14T04:46:31.5458078Z ++ python benchmarks/dynamo/pr_time_benchmarks/benchmarks/add_loop.py /var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv
2025-03-14T04:46:34.0348042Z collecting compile time instruction count for add_loop_eager
2025-03-14T04:46:36.1217046Z compile time instruction count for iteration 0 is 2927276495
2025-03-14T04:46:36.9828567Z compile time instruction count for iteration 1 is 2816055819
2025-03-14T04:46:37.8369704Z compile time instruction count for iteration 2 is 2815319418
2025-03-14T04:46:38.6884782Z compile time instruction count for iteration 3 is 2812164289
2025-03-14T04:46:39.5431129Z compile time instruction count for iteration 4 is 2812459333
2025-03-14T04:46:39.5664213Z collecting compile time instruction count for add_loop_eager_dynamic
2025-03-14T04:46:41.1468237Z compile time instruction count for iteration 0 is 5830844869
2025-03-14T04:46:42.6680377Z compile time instruction count for iteration 1 is 5508400769
2025-03-14T04:46:44.1918795Z compile time instruction count for iteration 2 is 5503950221
2025-03-14T04:46:45.7113552Z compile time instruction count for iteration 3 is 5498325731
2025-03-14T04:46:47.2249003Z compile time instruction count for iteration 4 is 5497853707
2025-03-14T04:46:47.2251457Z collecting compile time instruction count for add_loop_inductor
2025-03-14T04:47:05.6455464Z compile time instruction count for iteration 0 is 55619400771
2025-03-14T04:47:14.4554095Z compile time instruction count for iteration 1 is 27634645419
2025-03-14T04:47:23.3166421Z compile time instruction count for iteration 2 is 27627472757
2025-03-14T04:47:32.2388632Z compile time instruction count for iteration 3 is 27643195064
2025-03-14T04:47:41.2062554Z compile time instruction count for iteration 4 is 27654656955
2025-03-14T04:47:41.2281514Z collecting compile time instruction count for add_loop_inductor_gpu
2025-03-14T04:48:08.1814210Z compile time instruction count for iteration 0 is 24061193973
2025-03-14T04:48:35.2880389Z compile time instruction count for iteration 1 is 24051429086
2025-03-14T04:49:02.5111205Z compile time instruction count for iteration 2 is 24059782487
2025-03-14T04:49:30.6252258Z compile time instruction count for iteration 3 is 24061823129
2025-03-14T04:49:58.2693666Z compile time instruction count for iteration 4 is 24066875214
2025-03-14T04:49:58.2695957Z collecting compile time instruction count for add_loop_inductor_dynamic_gpu
2025-03-14T04:50:30.0930670Z compile time instruction count for iteration 0 is 40773496100
2025-03-14T04:51:01.6063736Z compile time instruction count for iteration 1 is 40656163350
2025-03-14T04:51:33.3910404Z compile time instruction count for iteration 2 is 40637727267
2025-03-14T04:52:06.1379604Z compile time instruction count for iteration 3 is 40638079784
2025-03-14T04:52:38.9262012Z compile time instruction count for iteration 4 is 40618027081
2025-03-14T04:52:44.7036459Z ++ for file in $python_programs_dir/*.py
2025-03-14T04:52:44.7039229Z ++ '[' benchmarks/dynamo/pr_time_benchmarks/benchmarks/aotdispatcher.py '!=' benchmarks/dynamo/pr_time_benchmarks/benchmarks/benchmark_base.py ']'
2025-03-14T04:52:44.7040830Z ++ python benchmarks/dynamo/pr_time_benchmarks/benchmarks/aotdispatcher.py /var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv
2025-03-14T04:52:47.1678779Z collecting compile time instruction count for aotdispatcher_inference_nosubclass_cpu
2025-03-14T04:52:49.6758539Z compile time instruction count for iteration 0 is 6564818973
2025-03-14T04:52:50.3529708Z compile time instruction count for iteration 1 is 2012547536
2025-03-14T04:52:51.0534289Z compile time instruction count for iteration 2 is 1996715737
2025-03-14T04:52:51.7349400Z compile time instruction count for iteration 3 is 1990292470
2025-03-14T04:52:52.4164013Z compile time instruction count for iteration 4 is 1988800708
2025-03-14T04:52:52.4397623Z collecting compile time instruction count for aotdispatcher_training_nosubclass_cpu
2025-03-14T04:52:53.4861047Z compile time instruction count for iteration 0 is 3629640857
2025-03-14T04:52:54.5808290Z compile time instruction count for iteration 1 is 3582562531
2025-03-14T04:52:55.5927974Z compile time instruction count for iteration 2 is 3578556746
2025-03-14T04:52:56.6713509Z compile time instruction count for iteration 3 is 3576847867
2025-03-14T04:52:57.6780420Z compile time instruction count for iteration 4 is 3575437159
2025-03-14T04:52:57.6798682Z collecting compile time instruction count for aotdispatcher_inference_subclass_cpu
2025-03-14T04:52:59.2609468Z compile time instruction count for iteration 0 is 5743221603
2025-03-14T04:53:00.8040896Z compile time instruction count for iteration 1 is 5741123399
2025-03-14T04:53:02.3352620Z compile time instruction count for iteration 2 is 5740190413
2025-03-14T04:53:03.8733852Z compile time instruction count for iteration 3 is 5738299351
2025-03-14T04:53:05.4192359Z compile time instruction count for iteration 4 is 5738901783
2025-03-14T04:53:05.4214451Z collecting compile time instruction count for aotdispatcher_training_subclass_cpu
2025-03-14T04:53:07.8390289Z compile time instruction count for iteration 0 is 9770334675
2025-03-14T04:53:10.2706608Z compile time instruction count for iteration 1 is 9770561631
2025-03-14T04:53:12.7070089Z compile time instruction count for iteration 2 is 9770717397
2025-03-14T04:53:15.1420660Z compile time instruction count for iteration 3 is 9766345772
2025-03-14T04:53:17.5888891Z compile time instruction count for iteration 4 is 9766739369
2025-03-14T04:53:19.4251296Z ++ for file in $python_programs_dir/*.py
2025-03-14T04:53:19.4252422Z ++ '[' benchmarks/dynamo/pr_time_benchmarks/benchmarks/aotdispatcher_partitioner.py '!=' benchmarks/dynamo/pr_time_benchmarks/benchmarks/benchmark_base.py ']'
2025-03-14T04:53:19.4254073Z ++ python benchmarks/dynamo/pr_time_benchmarks/benchmarks/aotdispatcher_partitioner.py /var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv
2025-03-14T04:53:21.8925113Z collecting compile time instruction count for aotdispatcher_partitioner_cpu
2025-03-14T04:53:25.9183426Z compile time instruction count for iteration 0 is 12491359877
2025-03-14T04:53:28.0994756Z compile time instruction count for iteration 1 is 7841859925
2025-03-14T04:53:30.2597426Z compile time instruction count for iteration 2 is 7830452110
2025-03-14T04:53:32.4388161Z compile time instruction count for iteration 3 is 7822254432
2025-03-14T04:53:34.6242713Z compile time instruction count for iteration 4 is 7816699182
2025-03-14T04:53:36.5898166Z ++ for file in $python_programs_dir/*.py
2025-03-14T04:53:36.5899409Z ++ '[' benchmarks/dynamo/pr_time_benchmarks/benchmarks/aotdispatcher_partitioner2.py '!=' benchmarks/dynamo/pr_time_benchmarks/benchmarks/benchmark_base.py ']'
2025-03-14T04:53:36.5901074Z ++ python benchmarks/dynamo/pr_time_benchmarks/benchmarks/aotdispatcher_partitioner2.py /var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv
2025-03-14T04:53:39.0843009Z collecting compile time instruction count for aotdispatcher_partitioner_cpu2
2025-03-14T04:53:41.5375846Z compile time instruction count for iteration 0 is 6312099257
2025-03-14T04:53:42.1583424Z compile time instruction count for iteration 1 is 1729888517
2025-03-14T04:53:42.8088877Z compile time instruction count for iteration 2 is 1719312422
2025-03-14T04:53:43.4440018Z compile time instruction count for iteration 3 is 1713423073
2025-03-14T04:53:44.0795293Z compile time instruction count for iteration 4 is 1712693456
2025-03-14T04:53:46.7038423Z ++ for file in $python_programs_dir/*.py
2025-03-14T04:53:46.7039592Z ++ '[' benchmarks/dynamo/pr_time_benchmarks/benchmarks/basic_modules_benchmarks.py '!=' benchmarks/dynamo/pr_time_benchmarks/benchmarks/benchmark_base.py ']'
2025-03-14T04:53:46.7041173Z ++ python benchmarks/dynamo/pr_time_benchmarks/benchmarks/basic_modules_benchmarks.py /var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv
2025-03-14T04:53:49.1893562Z collecting compile time instruction count for basic_modules_ListOfLinears_eager
2025-03-14T04:53:50.6460387Z compile time instruction count for iteration 0 is 1110487260
2025-03-14T04:53:50.9700715Z compile time instruction count for iteration 1 is 968713028
2025-03-14T04:53:51.2807803Z compile time instruction count for iteration 2 is 963267625
2025-03-14T04:53:51.5907357Z compile time instruction count for iteration 3 is 958661311
2025-03-14T04:53:51.9389096Z compile time instruction count for iteration 4 is 954238007
2025-03-14T04:53:51.9632613Z collecting compile time instruction count for basic_modules_ListOfLinears_inductor
2025-03-14T04:54:07.6449366Z compile time instruction count for iteration 0 is 41600643381
2025-03-14T04:54:13.7931661Z compile time instruction count for iteration 1 is 17271627059
2025-03-14T04:54:19.9657100Z compile time instruction count for iteration 2 is 17235273867
2025-03-14T04:54:26.1692855Z compile time instruction count for iteration 3 is 17232564466
2025-03-14T04:54:32.4274707Z compile time instruction count for iteration 4 is 17228575585
2025-03-14T04:54:32.4364203Z collecting compile time instruction count for basic_modules_ListOfLinears_inductor_gpu
2025-03-14T04:54:35.2646341Z W0314 04:54:35.263000 4899 site-packages/torch/_inductor/utils.py:1268] [0/0] Not enough SMs to use max_autotune_gemm mode
2025-03-14T04:54:36.3265067Z compile time instruction count for iteration 0 is 9835381502
2025-03-14T04:54:40.1012379Z compile time instruction count for iteration 1 is 9673202988
2025-03-14T04:54:43.8810367Z compile time instruction count for iteration 2 is 9673559787
2025-03-14T04:54:47.6853051Z compile time instruction count for iteration 3 is 9673032249
2025-03-14T04:54:51.4995313Z compile time instruction count for iteration 4 is 9660950370
2025-03-14T04:54:51.5013712Z collecting compile time instruction count for basic_modules_ListOfLinears_inductor_gpu_force_shape_pad
2025-03-14T04:54:56.7192520Z compile time instruction count for iteration 0 is 15514007041
2025-03-14T04:55:01.8135862Z compile time instruction count for iteration 1 is 15482709082
2025-03-14T04:55:06.9578404Z compile time instruction count for iteration 2 is 15577880164
2025-03-14T04:55:12.1638370Z compile time instruction count for iteration 3 is 15675955574
2025-03-14T04:55:17.5663183Z compile time instruction count for iteration 4 is 15772351397
2025-03-14T04:55:21.3452624Z ++ for file in $python_programs_dir/*.py
2025-03-14T04:55:21.3455311Z ++ '[' benchmarks/dynamo/pr_time_benchmarks/benchmarks/benchmark_base.py '!=' benchmarks/dynamo/pr_time_benchmarks/benchmarks/benchmark_base.py ']'
2025-03-14T04:55:21.3456314Z ++ for file in $python_programs_dir/*.py
2025-03-14T04:55:21.3457224Z ++ '[' benchmarks/dynamo/pr_time_benchmarks/benchmarks/float_args.py '!=' benchmarks/dynamo/pr_time_benchmarks/benchmarks/benchmark_base.py ']'
2025-03-14T04:55:21.3458669Z ++ python benchmarks/dynamo/pr_time_benchmarks/benchmarks/float_args.py /var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv
2025-03-14T04:55:23.8137228Z collecting compile time instruction count for float_args
2025-03-14T04:55:38.1075737Z compile time instruction count for iteration 0 is 24137191202
2025-03-14T04:55:41.6837894Z compile time instruction count for iteration 1 is 414312014
2025-03-14T04:55:45.2592411Z compile time instruction count for iteration 2 is 413690992
2025-03-14T04:55:48.8413561Z compile time instruction count for iteration 3 is 413261836
2025-03-14T04:55:52.4208182Z compile time instruction count for iteration 4 is 412857147
2025-03-14T04:55:54.2249307Z ++ for file in $python_programs_dir/*.py
2025-03-14T04:55:54.2253059Z ++ '[' benchmarks/dynamo/pr_time_benchmarks/benchmarks/sum_floordiv.py '!=' benchmarks/dynamo/pr_time_benchmarks/benchmarks/benchmark_base.py ']'
2025-03-14T04:55:54.2254571Z ++ python benchmarks/dynamo/pr_time_benchmarks/benchmarks/sum_floordiv.py /var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv
2025-03-14T04:55:56.7040345Z collecting compile time instruction count for sum_floordiv_regression
2025-03-14T04:55:58.5480259Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/backends/mkldnn/__init__.py:78: UserWarning: TF32 acceleration on top of oneDNN is available for Intel GPUs. The current Torch version does not have Intel GPU Support. (Triggered internally at /var/lib/jenkins/workspace/aten/src/ATen/Context.cpp:148.)
2025-03-14T04:55:58.5482196Z   torch._C._set_onednn_allow_tf32(_allow_tf32)
2025-03-14T04:55:58.7228107Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/backends/mkldnn/__init__.py:78: UserWarning: TF32 acceleration on top of oneDNN is available for Intel GPUs. The current Torch version does not have Intel GPU Support. (Triggered internally at /var/lib/jenkins/workspace/aten/src/ATen/Context.cpp:148.)
2025-03-14T04:55:58.7229883Z   torch._C._set_onednn_allow_tf32(_allow_tf32)
2025-03-14T04:55:58.7699888Z compile time instruction count for iteration 0 is 1772581974
2025-03-14T04:55:59.3705302Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/backends/mkldnn/__init__.py:78: UserWarning: TF32 acceleration on top of oneDNN is available for Intel GPUs. The current Torch version does not have Intel GPU Support. (Triggered internally at /var/lib/jenkins/workspace/aten/src/ATen/Context.cpp:148.)
2025-03-14T04:55:59.3707069Z   torch._C._set_onednn_allow_tf32(_allow_tf32)
2025-03-14T04:55:59.5326014Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/backends/mkldnn/__init__.py:78: UserWarning: TF32 acceleration on top of oneDNN is available for Intel GPUs. The current Torch version does not have Intel GPU Support. (Triggered internally at /var/lib/jenkins/workspace/aten/src/ATen/Context.cpp:148.)
2025-03-14T04:55:59.5327774Z   torch._C._set_onednn_allow_tf32(_allow_tf32)
2025-03-14T04:55:59.5802581Z compile time instruction count for iteration 1 is 1042165539
2025-03-14T04:56:00.2220328Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/backends/mkldnn/__init__.py:78: UserWarning: TF32 acceleration on top of oneDNN is available for Intel GPUs. The current Torch version does not have Intel GPU Support. (Triggered internally at /var/lib/jenkins/workspace/aten/src/ATen/Context.cpp:148.)
2025-03-14T04:56:00.2222156Z   torch._C._set_onednn_allow_tf32(_allow_tf32)
2025-03-14T04:56:00.3824861Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/backends/mkldnn/__init__.py:78: UserWarning: TF32 acceleration on top of oneDNN is available for Intel GPUs. The current Torch version does not have Intel GPU Support. (Triggered internally at /var/lib/jenkins/workspace/aten/src/ATen/Context.cpp:148.)
2025-03-14T04:56:00.3827032Z   torch._C._set_onednn_allow_tf32(_allow_tf32)
2025-03-14T04:56:00.4278648Z compile time instruction count for iteration 2 is 1036536074
2025-03-14T04:56:01.0178882Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/backends/mkldnn/__init__.py:78: UserWarning: TF32 acceleration on top of oneDNN is available for Intel GPUs. The current Torch version does not have Intel GPU Support. (Triggered internally at /var/lib/jenkins/workspace/aten/src/ATen/Context.cpp:148.)
2025-03-14T04:56:01.0180661Z   torch._C._set_onednn_allow_tf32(_allow_tf32)
2025-03-14T04:56:01.1780947Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/backends/mkldnn/__init__.py:78: UserWarning: TF32 acceleration on top of oneDNN is available for Intel GPUs. The current Torch version does not have Intel GPU Support. (Triggered internally at /var/lib/jenkins/workspace/aten/src/ATen/Context.cpp:148.)
2025-03-14T04:56:01.1782721Z   torch._C._set_onednn_allow_tf32(_allow_tf32)
2025-03-14T04:56:01.2232924Z compile time instruction count for iteration 3 is 1031842593
2025-03-14T04:56:01.8590138Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/backends/mkldnn/__init__.py:78: UserWarning: TF32 acceleration on top of oneDNN is available for Intel GPUs. The current Torch version does not have Intel GPU Support. (Triggered internally at /var/lib/jenkins/workspace/aten/src/ATen/Context.cpp:148.)
2025-03-14T04:56:01.8591913Z   torch._C._set_onednn_allow_tf32(_allow_tf32)
2025-03-14T04:56:02.0214659Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/backends/mkldnn/__init__.py:78: UserWarning: TF32 acceleration on top of oneDNN is available for Intel GPUs. The current Torch version does not have Intel GPU Support. (Triggered internally at /var/lib/jenkins/workspace/aten/src/ATen/Context.cpp:148.)
2025-03-14T04:56:02.0216437Z   torch._C._set_onednn_allow_tf32(_allow_tf32)
2025-03-14T04:56:02.0668084Z compile time instruction count for iteration 4 is 1028278641
2025-03-14T04:56:03.0032725Z ++ for file in $python_programs_dir/*.py
2025-03-14T04:56:03.0034167Z ++ '[' benchmarks/dynamo/pr_time_benchmarks/benchmarks/symint_sum.py '!=' benchmarks/dynamo/pr_time_benchmarks/benchmarks/benchmark_base.py ']'
2025-03-14T04:56:03.0035720Z ++ python benchmarks/dynamo/pr_time_benchmarks/benchmarks/symint_sum.py /var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv
2025-03-14T04:56:05.4734602Z collecting compile time instruction count for symint_sum
2025-03-14T04:56:19.6133690Z compile time instruction count for iteration 0 is 33529604605
2025-03-14T04:56:20.6773287Z compile time instruction count for iteration 1 is 3079518032
2025-03-14T04:56:21.7349776Z compile time instruction count for iteration 2 is 3039785477
2025-03-14T04:56:22.8397121Z compile time instruction count for iteration 3 is 3033180464
2025-03-14T04:56:23.9011610Z compile time instruction count for iteration 4 is 3031502979
2025-03-14T04:56:23.9246959Z collecting compile time instruction count for symint_sum_loop
2025-03-14T04:56:25.7059158Z compile time instruction count for iteration 0 is 6685536569
2025-03-14T04:56:26.9461752Z compile time instruction count for iteration 1 is 3981634721
2025-03-14T04:56:28.1925091Z compile time instruction count for iteration 2 is 3981749202
2025-03-14T04:56:29.4378310Z compile time instruction count for iteration 3 is 3981426349
2025-03-14T04:56:30.6894450Z compile time instruction count for iteration 4 is 3981435498
2025-03-14T04:56:32.6962335Z ++ for file in $python_programs_dir/*.py
2025-03-14T04:56:32.6964879Z ++ '[' benchmarks/dynamo/pr_time_benchmarks/benchmarks/update_hint_benchmark.py '!=' benchmarks/dynamo/pr_time_benchmarks/benchmarks/benchmark_base.py ']'
2025-03-14T04:56:32.6966489Z ++ python benchmarks/dynamo/pr_time_benchmarks/benchmarks/update_hint_benchmark.py /var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv
2025-03-14T04:56:35.1579966Z collecting compile time instruction count for update_hint_regression
2025-03-14T04:56:40.2487808Z compile time instruction count for iteration 0 is 30543965937
2025-03-14T04:56:40.8369454Z compile time instruction count for iteration 1 is 1545603539
2025-03-14T04:56:41.4400885Z compile time instruction count for iteration 2 is 1535498459
2025-03-14T04:56:42.0291762Z compile time instruction count for iteration 3 is 1530663592
2025-03-14T04:56:42.6164742Z compile time instruction count for iteration 4 is 1528920882
2025-03-14T04:56:44.5103085Z +++ date +%s
2025-03-14T04:56:44.5114808Z ++ end=1741928204
2025-03-14T04:56:44.5115160Z ++ runtime=613
2025-03-14T04:56:44.5115538Z ++ echo 'total time to run benchmarks is:'
2025-03-14T04:56:44.5115989Z total time to run benchmarks is:
2025-03-14T04:56:44.5116357Z ++ echo 613
2025-03-14T04:56:44.5116641Z 613
2025-03-14T04:56:44.5117110Z ++ python benchmarks/dynamo/pr_time_benchmarks/log_benchmarking_time.py 613
2025-03-14T04:56:46.6259758Z + echo 'benchmark results on current PR: '
2025-03-14T04:56:46.6260353Z benchmark results on current PR: 
2025-03-14T04:56:46.6261059Z + cat /var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv
2025-03-14T04:56:46.6270732Z add_loop_eager,compile_time_instruction_count,2812164289
2025-03-14T04:56:46.6271098Z 
2025-03-14T04:56:46.6271345Z add_loop_eager_dynamic,compile_time_instruction_count,5497853707
2025-03-14T04:56:46.6271746Z 
2025-03-14T04:56:46.6271978Z add_loop_inductor,compile_time_instruction_count,27627472757
2025-03-14T04:56:46.6272353Z 
2025-03-14T04:56:46.6272963Z add_loop_inductor_gpu,compile_time_instruction_count,24051429086
2025-03-14T04:56:46.6273368Z 
2025-03-14T04:56:46.6273670Z add_loop_inductor_dynamic_gpu,compile_time_instruction_count,40618027081
2025-03-14T04:56:46.6274098Z 
2025-03-14T04:56:46.6274435Z aotdispatcher_inference_nosubclass_cpu,compile_time_instruction_count,1988800708
2025-03-14T04:56:46.6274924Z 
2025-03-14T04:56:46.6275259Z aotdispatcher_training_nosubclass_cpu,compile_time_instruction_count,3575437159
2025-03-14T04:56:46.6275760Z 
2025-03-14T04:56:46.6276469Z aotdispatcher_inference_subclass_cpu,compile_time_instruction_count,5738299351
2025-03-14T04:56:46.6276980Z 
2025-03-14T04:56:46.6277298Z aotdispatcher_training_subclass_cpu,compile_time_instruction_count,9766345772
2025-03-14T04:56:46.6277776Z 
2025-03-14T04:56:46.6278070Z aotdispatcher_partitioner_cpu,compile_time_instruction_count,7816699182
2025-03-14T04:56:46.6278519Z 
2025-03-14T04:56:46.6278815Z aotdispatcher_partitioner_cpu2,compile_time_instruction_count,1712693456
2025-03-14T04:56:46.6279275Z 
2025-03-14T04:56:46.6279576Z basic_modules_ListOfLinears_eager,compile_time_instruction_count,954238007
2025-03-14T04:56:46.6280034Z 
2025-03-14T04:56:46.6280353Z basic_modules_ListOfLinears_inductor,compile_time_instruction_count,17228575585
2025-03-14T04:56:46.6280837Z 
2025-03-14T04:56:46.6281170Z basic_modules_ListOfLinears_inductor_gpu,compile_time_instruction_count,9660950370
2025-03-14T04:56:46.6281681Z 
2025-03-14T04:56:46.6282092Z basic_modules_ListOfLinears_inductor_gpu_force_shape_pad,compile_time_instruction_count,15482709082
2025-03-14T04:56:46.6282658Z 
2025-03-14T04:56:46.6282848Z float_args,compile_time_instruction_count,412857147
2025-03-14T04:56:46.6283195Z 
2025-03-14T04:56:46.6283454Z sum_floordiv_regression,compile_time_instruction_count,1028278641
2025-03-14T04:56:46.6283870Z 
2025-03-14T04:56:46.6284067Z symint_sum,compile_time_instruction_count,3031502979
2025-03-14T04:56:46.6284411Z 
2025-03-14T04:56:46.6284631Z symint_sum_loop,compile_time_instruction_count,3981426349
2025-03-14T04:56:46.6285103Z 
2025-03-14T04:56:46.6285355Z update_hint_regression,compile_time_instruction_count,1528920882
2025-03-14T04:56:46.6285753Z 
2025-03-14T04:56:46.6285868Z ++ pwd
2025-03-14T04:56:46.6286353Z + PYTHONPATH=/var/lib/jenkins/workspace/benchmarks/dynamo/pr_time_benchmarks
2025-03-14T04:56:46.6288165Z + python benchmarks/dynamo/pr_time_benchmarks/check_results.py benchmarks/dynamo/pr_time_benchmarks/expected_results.csv /var/lib/jenkins/workspace/test/test-reports/pr_time_benchmarks_results.csv /var/lib/jenkins/workspace/test/test-reports/new_expected_results.csv
2025-03-14T04:56:48.2481179Z PASS: benchmark ('add_loop_eager', 'compile_time_instruction_count') pass, actual result 2812164289 +0.22% is within expected 2806000000 ±1.50%
2025-03-14T04:56:48.2482033Z 
2025-03-14T04:56:48.2685122Z PASS: benchmark ('add_loop_eager_dynamic', 'compile_time_instruction_count') pass, actual result 5497853707 +0.69% is within expected 5460000000 ±2.50%
2025-03-14T04:56:48.2685908Z 
2025-03-14T04:56:48.2686677Z PASS: benchmark ('add_loop_inductor', 'compile_time_instruction_count') pass, actual result 27627472757 +0.39% is within expected 27520000000 ±1.50%
2025-03-14T04:56:48.2687398Z 
2025-03-14T04:56:48.2688220Z PASS: benchmark ('add_loop_inductor_dynamic_gpu', 'compile_time_instruction_count') pass, actual result 40618027081 +0.51% is within expected 40410000000 ±2.50%
2025-03-14T04:56:48.2689029Z 
2025-03-14T04:56:48.2689877Z PASS: benchmark ('add_loop_inductor_gpu', 'compile_time_instruction_count') pass, actual result 24051429086 +0.34% is within expected 23970000000 ±1.50%
2025-03-14T04:56:48.2690627Z 
2025-03-14T04:56:48.2691467Z PASS: benchmark ('basic_modules_ListOfLinears_eager', 'compile_time_instruction_count') pass, actual result 954238007 +0.05% is within expected 953800000 ±1.50%
2025-03-14T04:56:48.2692276Z 
2025-03-14T04:56:48.2693124Z PASS: benchmark ('basic_modules_ListOfLinears_inductor', 'compile_time_instruction_count') pass, actual result 17228575585 +0.93% is within expected 17070000000 ±1.50%
2025-03-14T04:56:48.2693963Z 
2025-03-14T04:56:48.2694927Z PASS: benchmark ('basic_modules_ListOfLinears_inductor_gpu_force_shape_pad', 'compile_time_instruction_count') pass, actual result 15482709082 +1.06% is within expected 15320000000 ±1.50%
2025-03-14T04:56:48.2695852Z 
2025-03-14T04:56:48.2696742Z PASS: benchmark ('basic_modules_ListOfLinears_inductor_gpu', 'compile_time_instruction_count') pass, actual result 9660950370 -0.55% is within expected 9714000000 ±20.00%
2025-03-14T04:56:48.2697606Z 
2025-03-14T04:56:48.2698642Z PASS: benchmark ('update_hint_regression', 'compile_time_instruction_count') pass, actual result 1528920882 +0.39% is within expected 1523000000 ±2.00%
2025-03-14T04:56:48.2699424Z 
2025-03-14T04:56:48.2700193Z PASS: benchmark ('sum_floordiv_regression', 'compile_time_instruction_count') pass, actual result 1028278641 +0.22% is within expected 1026000000 ±1.50%
2025-03-14T04:56:48.2700959Z 
2025-03-14T04:56:48.2701643Z PASS: benchmark ('symint_sum', 'compile_time_instruction_count') pass, actual result 3031502979 +0.61% is within expected 3013000000 ±1.50%
2025-03-14T04:56:48.2702339Z 
2025-03-14T04:56:48.2703218Z PASS: benchmark ('aotdispatcher_inference_nosubclass_cpu', 'compile_time_instruction_count') pass, actual result 1988800708 +1.26% is within expected 1964000000 ±1.50%
2025-03-14T04:56:48.2704046Z 
2025-03-14T04:56:48.2704907Z PASS: benchmark ('aotdispatcher_inference_subclass_cpu', 'compile_time_instruction_count') pass, actual result 5738299351 +1.17% is within expected 5672000000 ±1.50%
2025-03-14T04:56:48.2705741Z 
2025-03-14T04:56:48.2706573Z PASS: benchmark ('aotdispatcher_partitioner_cpu', 'compile_time_instruction_count') pass, actual result 7816699182 +0.83% is within expected 7752000000 ±1.50%
2025-03-14T04:56:48.2707358Z 
2025-03-14T04:56:48.2708231Z PASS: benchmark ('aotdispatcher_training_nosubclass_cpu', 'compile_time_instruction_count') pass, actual result 3575437159 +1.09% is within expected 3537000000 ±1.50%
2025-03-14T04:56:48.2709045Z 
2025-03-14T04:56:48.2709898Z PASS: benchmark ('aotdispatcher_training_subclass_cpu', 'compile_time_instruction_count') pass, actual result 9766345772 +1.08% is within expected 9662000000 ±1.50%
2025-03-14T04:56:48.2710706Z 
2025-03-14T04:56:48.2711378Z MISSING REGRESSION TEST: benchmark ('aotdispatcher_partitioner_cpu2', 'compile_time_instruction_count') does not have a regression test enabled for it.
2025-03-14T04:56:48.2712287Z 
2025-03-14T04:56:48.2712848Z MISSING REGRESSION TEST: benchmark ('float_args', 'compile_time_instruction_count') does not have a regression test enabled for it.
2025-03-14T04:56:48.2713552Z 
2025-03-14T04:56:48.2714110Z MISSING REGRESSION TEST: benchmark ('symint_sum_loop', 'compile_time_instruction_count') does not have a regression test enabled for it.
2025-03-14T04:56:48.2714830Z 
2025-03-14T04:56:48.2715010Z ================================================================================
2025-03-14T04:56:48.2715474Z ================================================================================
2025-03-14T04:56:48.2715934Z ================================================================================
2025-03-14T04:56:48.2716442Z To update expected results, run the following command:
2025-03-14T04:56:48.2716793Z 
2025-03-14T04:56:48.2717074Z cat > benchmarks/dynamo/pr_time_benchmarks/expected_results.csv << EOF
2025-03-14T04:56:48.2717732Z add_loop_eager,compile_time_instruction_count,2806000000,0.015
2025-03-14T04:56:48.2718121Z 
2025-03-14T04:56:48.2718126Z 
2025-03-14T04:56:48.2718130Z 
2025-03-14T04:56:48.2718400Z add_loop_eager_dynamic,compile_time_instruction_count,5460000000,0.025
2025-03-14T04:56:48.2718816Z 
2025-03-14T04:56:48.2718820Z 
2025-03-14T04:56:48.2718825Z 
2025-03-14T04:56:48.2719078Z add_loop_inductor,compile_time_instruction_count,27520000000,0.015
2025-03-14T04:56:48.2719481Z 
2025-03-14T04:56:48.2719484Z 
2025-03-14T04:56:48.2719488Z 
2025-03-14T04:56:48.2719810Z add_loop_inductor_dynamic_gpu,compile_time_instruction_count,40410000000,0.025
2025-03-14T04:56:48.2720240Z 
2025-03-14T04:56:48.2720244Z 
2025-03-14T04:56:48.2720248Z 
2025-03-14T04:56:48.2720541Z add_loop_inductor_gpu,compile_time_instruction_count,23970000000,0.015
2025-03-14T04:56:48.2720932Z 
2025-03-14T04:56:48.2720936Z 
2025-03-14T04:56:48.2720940Z 
2025-03-14T04:56:48.2721281Z basic_modules_ListOfLinears_eager,compile_time_instruction_count,953800000,0.015
2025-03-14T04:56:48.2721734Z 
2025-03-14T04:56:48.2721742Z 
2025-03-14T04:56:48.2721746Z 
2025-03-14T04:56:48.2722118Z basic_modules_ListOfLinears_inductor,compile_time_instruction_count,17220000000,0.015
2025-03-14T04:56:48.2722598Z 
2025-03-14T04:56:48.2722659Z 
2025-03-14T04:56:48.2722664Z 
2025-03-14T04:56:48.2723126Z basic_modules_ListOfLinears_inductor_gpu_force_shape_pad,compile_time_instruction_count,15480000000,0.015
2025-03-14T04:56:48.2723696Z 
2025-03-14T04:56:48.2723700Z 
2025-03-14T04:56:48.2723729Z 
2025-03-14T04:56:48.2724072Z basic_modules_ListOfLinears_inductor_gpu,compile_time_instruction_count,9714000000,0.2
2025-03-14T04:56:48.2724580Z 
2025-03-14T04:56:48.2724585Z 
2025-03-14T04:56:48.2724589Z 
2025-03-14T04:56:48.2724858Z update_hint_regression,compile_time_instruction_count,1523000000,0.02
2025-03-14T04:56:48.2725284Z 
2025-03-14T04:56:48.2725288Z 
2025-03-14T04:56:48.2725292Z 
2025-03-14T04:56:48.2725571Z sum_floordiv_regression,compile_time_instruction_count,1026000000,0.015
2025-03-14T04:56:48.2726007Z 
2025-03-14T04:56:48.2726016Z 
2025-03-14T04:56:48.2726020Z 
2025-03-14T04:56:48.2726236Z symint_sum,compile_time_instruction_count,3031000000,0.015
2025-03-14T04:56:48.2726597Z 
2025-03-14T04:56:48.2726601Z 
2025-03-14T04:56:48.2726609Z 
2025-03-14T04:56:48.2726958Z aotdispatcher_inference_nosubclass_cpu,compile_time_instruction_count,1988000000,0.015
2025-03-14T04:56:48.2727468Z 
2025-03-14T04:56:48.2727472Z 
2025-03-14T04:56:48.2727476Z 
2025-03-14T04:56:48.2727813Z aotdispatcher_inference_subclass_cpu,compile_time_instruction_count,5738000000,0.015
2025-03-14T04:56:48.2728311Z 
2025-03-14T04:56:48.2728316Z 
2025-03-14T04:56:48.2728319Z 
2025-03-14T04:56:48.2728650Z aotdispatcher_partitioner_cpu,compile_time_instruction_count,7816000000,0.015
2025-03-14T04:56:48.2729095Z 
2025-03-14T04:56:48.2729099Z 
2025-03-14T04:56:48.2729103Z 
2025-03-14T04:56:48.2729474Z aotdispatcher_training_nosubclass_cpu,compile_time_instruction_count,3575000000,0.015
2025-03-14T04:56:48.2729957Z 
2025-03-14T04:56:48.2729961Z 
2025-03-14T04:56:48.2730023Z 
2025-03-14T04:56:48.2730383Z aotdispatcher_training_subclass_cpu,compile_time_instruction_count,9766000000,0.015
2025-03-14T04:56:48.2730986Z EOF
2025-03-14T04:56:48.2731125Z 
2025-03-14T04:56:48.2731304Z ================================================================================
2025-03-14T04:56:48.2731759Z ================================================================================
2025-03-14T04:56:48.2732212Z ================================================================================
2025-03-14T04:56:48.2732651Z All benchmarks passed
2025-03-14T04:56:48.5907021Z + exit 0
2025-03-14T04:56:48.5907448Z + cleanup_workspace
2025-03-14T04:56:48.5908155Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.'
2025-03-14T04:56:48.5909143Z sudo may print the following warning message that can be ignored. The chown command will still run.
2025-03-14T04:56:48.5909935Z + echo '    sudo: setrlimit(RLIMIT_STACK): Operation not permitted'
2025-03-14T04:56:48.5910576Z     sudo: setrlimit(RLIMIT_STACK): Operation not permitted
2025-03-14T04:56:48.5911256Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42'
2025-03-14T04:56:48.5912037Z For more details refer to https://github.com/sudo-project/sudo/issues/42
2025-03-14T04:56:48.5912657Z + sudo chown -R 1000 /var/lib/jenkins/workspace
2025-03-14T04:56:49.0882652Z ##[group]Run pytorch/test-infra/.github/actions/upload-benchmark-results@main
2025-03-14T04:56:49.0883232Z with:
2025-03-14T04:56:49.0883569Z   benchmark-results-dir: test/test-reports
2025-03-14T04:56:49.0883976Z   dry-run: false
2025-03-14T04:56:49.0884293Z   schema-version: v3
2025-03-14T04:56:49.0884838Z   github-token: ***
2025-03-14T04:56:49.0885146Z env:
2025-03-14T04:56:49.0885440Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:49.0885836Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:49.0886491Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:49.0887085Z ##[endgroup]
2025-03-14T04:56:49.1021591Z ##[group]Run set -eux
2025-03-14T04:56:49.1021911Z set -eux
2025-03-14T04:56:49.1022234Z python3 -mpip install boto3==1.35.33
2025-03-14T04:56:49.1031000Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:56:49.1031435Z env:
2025-03-14T04:56:49.1031753Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:49.1032150Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:49.1032797Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:49.1033377Z ##[endgroup]
2025-03-14T04:56:49.1061868Z + python3 -mpip install boto3==1.35.33
2025-03-14T04:56:49.3396631Z Defaulting to user installation because normal site-packages is not writeable
2025-03-14T04:56:50.3519676Z Collecting boto3==1.35.33
2025-03-14T04:56:50.3539779Z   Using cached boto3-1.35.33-py3-none-any.whl (139 kB)
2025-03-14T04:56:50.3601804Z Requirement already satisfied: botocore<1.36.0,>=1.35.33 in /home/ec2-user/.local/lib/python3.9/site-packages (from boto3==1.35.33) (1.35.99)
2025-03-14T04:56:50.3605203Z Requirement already satisfied: s3transfer<0.11.0,>=0.10.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from boto3==1.35.33) (0.10.4)
2025-03-14T04:56:50.3609697Z Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /usr/lib/python3.9/site-packages (from boto3==1.35.33) (0.10.0)
2025-03-14T04:56:50.3670839Z Requirement already satisfied: urllib3<1.27,>=1.25.4 in /usr/lib/python3.9/site-packages (from botocore<1.36.0,>=1.35.33->boto3==1.35.33) (1.25.10)
2025-03-14T04:56:50.3675194Z Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /home/ec2-user/.local/lib/python3.9/site-packages (from botocore<1.36.0,>=1.35.33->boto3==1.35.33) (2.8.2)
2025-03-14T04:56:50.3716152Z Requirement already satisfied: six>=1.5 in /usr/lib/python3.9/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.36.0,>=1.35.33->boto3==1.35.33) (1.15.0)
2025-03-14T04:56:50.5290875Z Installing collected packages: boto3
2025-03-14T04:56:50.5291532Z   Attempting uninstall: boto3
2025-03-14T04:56:50.5291935Z     Found existing installation: boto3 1.35.42
2025-03-14T04:56:50.5381663Z     Uninstalling boto3-1.35.42:
2025-03-14T04:56:50.5392000Z       Successfully uninstalled boto3-1.35.42
2025-03-14T04:56:50.5961359Z Successfully installed boto3-1.35.33
2025-03-14T04:56:50.6585499Z ##[group]Run set -eux
2025-03-14T04:56:50.6585819Z set -eux
2025-03-14T04:56:50.6586078Z 
2025-03-14T04:56:50.6586351Z if [[ -z "${GITHUB_TOKEN}" ]]; then
2025-03-14T04:56:50.6586756Z   echo "Missing github-token input"
2025-03-14T04:56:50.6587119Z   exit 1
2025-03-14T04:56:50.6587379Z fi
2025-03-14T04:56:50.6594381Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:56:50.6594851Z env:
2025-03-14T04:56:50.6595133Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:50.6595541Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:50.6596240Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:50.6597144Z   GITHUB_TOKEN: ***
2025-03-14T04:56:50.6597437Z ##[endgroup]
2025-03-14T04:56:50.6622683Z + [[ -z *** ]]
2025-03-14T04:56:50.6686560Z ##[group]Run pytorch/test-infra/.github/actions/get-workflow-job-id@main
2025-03-14T04:56:50.6687242Z with:
2025-03-14T04:56:50.6687678Z   github-token: ***
2025-03-14T04:56:50.6687982Z env:
2025-03-14T04:56:50.6688270Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:50.6688682Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:50.6689329Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:50.6689966Z ##[endgroup]
2025-03-14T04:56:50.6717600Z ##[group]Run set -eux
2025-03-14T04:56:50.6717933Z set -eux
2025-03-14T04:56:50.6718225Z 
2025-03-14T04:56:50.6718796Z python3 "${GITHUB_ACTION_PATH}/../../scripts/get_workflow_job_id.py" "${GITHUB_RUN_ID}" "${RUNNER_NAME}"
2025-03-14T04:56:50.6724213Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:56:50.6724675Z env:
2025-03-14T04:56:50.6724964Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:50.6725368Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:50.6726029Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:50.6726873Z   GITHUB_TOKEN: ***
2025-03-14T04:56:50.6727161Z ##[endgroup]
2025-03-14T04:56:50.6750178Z + python3 /home/ec2-user/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/get-workflow-job-id/../../scripts/get_workflow_job_id.py 13849515290 i-0d8b0d116e8f11da9
2025-03-14T04:56:54.4794311Z setting job-id=38755508671
2025-03-14T04:56:54.4795246Z setting job-name=cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu)
2025-03-14T04:56:54.4914494Z ##[group]Run set -eux
2025-03-14T04:56:54.4914909Z set -eux
2025-03-14T04:56:54.4915223Z 
2025-03-14T04:56:54.4915707Z python3 "${GITHUB_ACTION_PATH}/../../scripts/benchmarks/gather_metadata.py" \
2025-03-14T04:56:54.4916339Z   --schema-version "${SCHEMA_VERSION}" \
2025-03-14T04:56:54.4916759Z   --repo "${REPO}" \
2025-03-14T04:56:54.4917138Z   --head-branch "${HEAD_BRANCH}" \
2025-03-14T04:56:54.4917584Z   --head-sha "${HEAD_SHA}" \
2025-03-14T04:56:54.4918004Z   --workflow-id "${WORKFLOW_RUN_ID}" \
2025-03-14T04:56:54.4918439Z   --run-attempt "${RUN_ATTEMPT}" \
2025-03-14T04:56:54.4918849Z   --job-id "${JOB_ID}" \
2025-03-14T04:56:54.4919228Z   --job-name "${JOB_NAME}"
2025-03-14T04:56:54.4924203Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:56:54.4924666Z env:
2025-03-14T04:56:54.4924952Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:54.4925370Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:54.4926021Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:54.4927654Z   SCHEMA_VERSION: v3
2025-03-14T04:56:54.4927956Z   REPO: pytorch/pytorch
2025-03-14T04:56:54.4928305Z   HEAD_BRANCH: refs/heads/main
2025-03-14T04:56:54.4928703Z   HEAD_SHA: aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:56:54.4929154Z   WORKFLOW_RUN_ID: 13849515290
2025-03-14T04:56:54.4929506Z   RUN_ATTEMPT: 1
2025-03-14T04:56:54.4929818Z   JOB_ID: 38755508671
2025-03-14T04:56:54.4930384Z   JOB_NAME: cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu)
2025-03-14T04:56:54.4930994Z ##[endgroup]
2025-03-14T04:56:54.4958264Z + python3 /home/ec2-user/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/upload-benchmark-results/../../scripts/benchmarks/gather_metadata.py --schema-version v3 --repo pytorch/pytorch --head-branch refs/heads/main --head-sha aed0b7a742a2d7b7901790622829cbd2135049a4 --workflow-id 13849515290 --run-attempt 1 --job-id 38755508671 --job-name 'cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu)'
2025-03-14T04:56:54.5297074Z ##[group]Run set -eux
2025-03-14T04:56:54.5297425Z set -eux
2025-03-14T04:56:54.5297737Z 
2025-03-14T04:56:54.5298062Z # TODO (huydhn): Implement this part
2025-03-14T04:56:54.5298677Z echo "runners=[]" >> "${GITHUB_OUTPUT}"
2025-03-14T04:56:54.5303210Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:56:54.5303683Z env:
2025-03-14T04:56:54.5303981Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:54.5304392Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:54.5305049Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:54.5305638Z ##[endgroup]
2025-03-14T04:56:54.5329094Z + echo 'runners=[]'
2025-03-14T04:56:54.5361668Z ##[group]Run set -eux
2025-03-14T04:56:54.5361994Z set -eux
2025-03-14T04:56:54.5362271Z 
2025-03-14T04:56:54.5362609Z # TODO (huydhn): Implement this part
2025-03-14T04:56:54.5363072Z echo "dependencies={}" >> "${GITHUB_OUTPUT}"
2025-03-14T04:56:54.5368336Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:56:54.5368761Z env:
2025-03-14T04:56:54.5369022Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:54.5369431Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:54.5370051Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:54.5370611Z ##[endgroup]
2025-03-14T04:56:54.5396057Z + echo 'dependencies={}'
2025-03-14T04:56:54.5435187Z ##[group]Run set -eux
2025-03-14T04:56:54.5435546Z set -eux
2025-03-14T04:56:54.5435853Z 
2025-03-14T04:56:54.5436203Z if [[ ! -d "${BENCHMARK_RESULTS_DIR}" ]]; then
2025-03-14T04:56:54.5436740Z   echo "${BENCHMARK_RESULTS_DIR} does not exist, skipping"
2025-03-14T04:56:54.5437324Z   # We don't want the job to fail if the directory doesn't exist
2025-03-14T04:56:54.5437816Z   exit 0
2025-03-14T04:56:54.5438109Z fi
2025-03-14T04:56:54.5438393Z 
2025-03-14T04:56:54.5438703Z if [[ "${DRY_RUN}" == "true" ]]; then
2025-03-14T04:56:54.5439264Z   python3 "${GITHUB_ACTION_PATH}/../../scripts/upload_benchmark_results.py" \
2025-03-14T04:56:54.5439933Z     --benchmark-results-dir "${BENCHMARK_RESULTS_DIR}" \
2025-03-14T04:56:54.5440442Z     --metadata "${BENCHMARK_METADATA}" \
2025-03-14T04:56:54.5440884Z     --runners "${RUNNER_INFO}" \
2025-03-14T04:56:54.5441318Z     --dependencies "${DEPENDENCIES}" \
2025-03-14T04:56:54.5441738Z     --dry-run
2025-03-14T04:56:54.5442065Z else
2025-03-14T04:56:54.5442528Z   python3 "${GITHUB_ACTION_PATH}/../../scripts/upload_benchmark_results.py" \
2025-03-14T04:56:54.5443175Z     --benchmark-results-dir "${BENCHMARK_RESULTS_DIR}" \
2025-03-14T04:56:54.5443820Z     --metadata "${BENCHMARK_METADATA}" \
2025-03-14T04:56:54.5444230Z     --runners "${RUNNER_INFO}" \
2025-03-14T04:56:54.5444661Z     --dependencies "${DEPENDENCIES}"
2025-03-14T04:56:54.5445063Z fi
2025-03-14T04:56:54.5449820Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:56:54.5450271Z env:
2025-03-14T04:56:54.5450562Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:54.5450983Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:54.5451640Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:54.5452265Z   BENCHMARK_RESULTS_DIR: test/test-reports
2025-03-14T04:56:54.5452663Z   DRY_RUN: false
2025-03-14T04:56:54.5454142Z   BENCHMARK_METADATA: {"timestamp": 1741928214, "schema_version": "v3", "name": "cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu)", "repo": "pytorch/pytorch", "head_branch": "refs/heads/main", "head_sha": "aed0b7a742a2d7b7901790622829cbd2135049a4", "workflow_id": 13849515290, "run_attempt": 1, "job_id": 38755508671}
2025-03-14T04:56:54.5455707Z   RUNNER_INFO: []
2025-03-14T04:56:54.5456017Z   DEPENDENCIES: {}
2025-03-14T04:56:54.5456319Z ##[endgroup]
2025-03-14T04:56:54.5478526Z + [[ ! -d test/test-reports ]]
2025-03-14T04:56:54.5479529Z + [[ false == \t\r\u\e ]]
2025-03-14T04:56:54.5482289Z + python3 /home/ec2-user/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/upload-benchmark-results/../../scripts/upload_benchmark_results.py --benchmark-results-dir test/test-reports --metadata '{"timestamp": 1741928214, "schema_version": "v3", "name": "cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu)", "repo": "pytorch/pytorch", "head_branch": "refs/heads/main", "head_sha": "aed0b7a742a2d7b7901790622829cbd2135049a4", "workflow_id": 13849515290, "run_attempt": 1, "job_id": 38755508671}' --runners '[]' --dependencies '{}'
2025-03-14T04:56:54.6843165Z INFO:root:Upload test/test-reports/add_loop_eager.json to s3://ossci-benchmarks/v3/pytorch/pytorch/13849515290/38755508671/add_loop_eager.json
2025-03-14T04:56:54.7079012Z INFO:botocore.credentials:Found credentials from IAM Role: gh-ci-github-action-runners-runner-role
2025-03-14T04:56:54.8574644Z INFO:root:Upload test/test-reports/add_loop_eager_dynamic.json to s3://ossci-benchmarks/v3/pytorch/pytorch/13849515290/38755508671/add_loop_eager_dynamic.json
2025-03-14T04:56:54.9713032Z INFO:root:Upload test/test-reports/add_loop_inductor.json to s3://ossci-benchmarks/v3/pytorch/pytorch/13849515290/38755508671/add_loop_inductor.json
2025-03-14T04:56:55.0522940Z INFO:root:Upload test/test-reports/add_loop_inductor_gpu.json to s3://ossci-benchmarks/v3/pytorch/pytorch/13849515290/38755508671/add_loop_inductor_gpu.json
2025-03-14T04:56:55.1566103Z INFO:root:Upload test/test-reports/add_loop_inductor_dynamic_gpu.json to s3://ossci-benchmarks/v3/pytorch/pytorch/13849515290/38755508671/add_loop_inductor_dynamic_gpu.json
2025-03-14T04:56:55.2322150Z INFO:root:Upload test/test-reports/aotdispatcher_inference_nosubclass_cpu.json to s3://ossci-benchmarks/v3/pytorch/pytorch/13849515290/38755508671/aotdispatcher_inference_nosubclass_cpu.json
2025-03-14T04:56:55.3112996Z INFO:root:Upload test/test-reports/aotdispatcher_training_nosubclass_cpu.json to s3://ossci-benchmarks/v3/pytorch/pytorch/13849515290/38755508671/aotdispatcher_training_nosubclass_cpu.json
2025-03-14T04:56:55.4163541Z INFO:root:Upload test/test-reports/aotdispatcher_inference_subclass_cpu.json to s3://ossci-benchmarks/v3/pytorch/pytorch/13849515290/38755508671/aotdispatcher_inference_subclass_cpu.json
2025-03-14T04:56:55.4912383Z INFO:root:Upload test/test-reports/aotdispatcher_training_subclass_cpu.json to s3://ossci-benchmarks/v3/pytorch/pytorch/13849515290/38755508671/aotdispatcher_training_subclass_cpu.json
2025-03-14T04:56:55.5868534Z INFO:root:Upload test/test-reports/aotdispatcher_partitioner_cpu.json to s3://ossci-benchmarks/v3/pytorch/pytorch/13849515290/38755508671/aotdispatcher_partitioner_cpu.json
2025-03-14T04:56:55.6633755Z INFO:root:Upload test/test-reports/aotdispatcher_partitioner_cpu2.json to s3://ossci-benchmarks/v3/pytorch/pytorch/13849515290/38755508671/aotdispatcher_partitioner_cpu2.json
2025-03-14T04:56:55.7128273Z INFO:root:Upload test/test-reports/basic_modules_ListOfLinears_eager.json to s3://ossci-benchmarks/v3/pytorch/pytorch/13849515290/38755508671/basic_modules_ListOfLinears_eager.json
2025-03-14T04:56:55.7984115Z INFO:root:Upload test/test-reports/basic_modules_ListOfLinears_inductor.json to s3://ossci-benchmarks/v3/pytorch/pytorch/13849515290/38755508671/basic_modules_ListOfLinears_inductor.json
2025-03-14T04:56:55.8662536Z INFO:root:Upload test/test-reports/basic_modules_ListOfLinears_inductor_gpu.json to s3://ossci-benchmarks/v3/pytorch/pytorch/13849515290/38755508671/basic_modules_ListOfLinears_inductor_gpu.json
2025-03-14T04:56:55.9516412Z INFO:root:Upload test/test-reports/basic_modules_ListOfLinears_inductor_gpu_force_shape_pad.json to s3://ossci-benchmarks/v3/pytorch/pytorch/13849515290/38755508671/basic_modules_ListOfLinears_inductor_gpu_force_shape_pad.json
2025-03-14T04:56:56.0296524Z INFO:root:Upload test/test-reports/float_args.json to s3://ossci-benchmarks/v3/pytorch/pytorch/13849515290/38755508671/float_args.json
2025-03-14T04:56:56.1000081Z INFO:root:Upload test/test-reports/sum_floordiv_regression.json to s3://ossci-benchmarks/v3/pytorch/pytorch/13849515290/38755508671/sum_floordiv_regression.json
2025-03-14T04:56:56.1841458Z INFO:root:Upload test/test-reports/symint_sum.json to s3://ossci-benchmarks/v3/pytorch/pytorch/13849515290/38755508671/symint_sum.json
2025-03-14T04:56:56.2685023Z INFO:root:Upload test/test-reports/symint_sum_loop.json to s3://ossci-benchmarks/v3/pytorch/pytorch/13849515290/38755508671/symint_sum_loop.json
2025-03-14T04:56:56.3445724Z INFO:root:Upload test/test-reports/update_hint_regression.json to s3://ossci-benchmarks/v3/pytorch/pytorch/13849515290/38755508671/update_hint_regression.json
2025-03-14T04:56:56.4988474Z ##[group]Run cat test/**/*_toprint.log || true
2025-03-14T04:56:56.4988898Z cat test/**/*_toprint.log || true
2025-03-14T04:56:56.4994753Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:56:56.4995246Z env:
2025-03-14T04:56:56.4995537Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:56.4995928Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:56.4996615Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:56.4997236Z ##[endgroup]
2025-03-14T04:56:56.5067124Z cat: 'test/**/*_toprint.log': No such file or directory
2025-03-14T04:56:56.5125184Z ##[group]Run kill "$MONITOR_SCRIPT_PID"
2025-03-14T04:56:56.5125619Z kill "$MONITOR_SCRIPT_PID"
2025-03-14T04:56:56.5130378Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:56:56.5130804Z env:
2025-03-14T04:56:56.5131057Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:56.5131428Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:56.5132057Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:56.5132627Z   MONITOR_SCRIPT_PID: 85965
2025-03-14T04:56:56.5132918Z ##[endgroup]
2025-03-14T04:56:56.5293378Z Prepare all required actions
2025-03-14T04:56:56.5293923Z Getting action download info
2025-03-14T04:56:56.6429676Z Download action repository 'actions/upload-artifact@v4' (SHA:4cec3d8aa04e39d1a68397de0c4cd6fb9dce8ec1)
2025-03-14T04:56:57.1037017Z ##[group]Run ./.github/actions/upload-test-artifacts
2025-03-14T04:56:57.1037482Z with:
2025-03-14T04:56:57.1037983Z   file-suffix: test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_38755508671
2025-03-14T04:56:57.1038617Z   s3-bucket: gha-artifacts
2025-03-14T04:56:57.1038958Z env:
2025-03-14T04:56:57.1039237Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:57.1039644Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:57.1040322Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:57.1041074Z ##[endgroup]
2025-03-14T04:56:57.1073892Z ##[group]Run # Remove any previous test jsons if they exist
2025-03-14T04:56:57.1074544Z # Remove any previous test jsons if they exist
2025-03-14T04:56:57.1075029Z rm -f test-jsons-*.zip
2025-03-14T04:56:57.1075563Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test/test-reports -i '*.json'
2025-03-14T04:56:57.1081101Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:56:57.1081592Z env:
2025-03-14T04:56:57.1081899Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:57.1082330Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:57.1083030Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:57.1083866Z   FILE_SUFFIX: test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_38755508671
2025-03-14T04:56:57.1084460Z ##[endgroup]
2025-03-14T04:56:57.1128599Z   adding: test/test-reports/add_loop_eager.json (deflated 40%)
2025-03-14T04:56:57.1129391Z   adding: test/test-reports/add_loop_eager_dynamic.json (deflated 41%)
2025-03-14T04:56:57.1130068Z   adding: test/test-reports/add_loop_inductor.json (deflated 40%)
2025-03-14T04:56:57.1130736Z   adding: test/test-reports/add_loop_inductor_gpu.json (deflated 40%)
2025-03-14T04:56:57.1131436Z   adding: test/test-reports/add_loop_inductor_dynamic_gpu.json (deflated 41%)
2025-03-14T04:56:57.1132241Z   adding: test/test-reports/aotdispatcher_inference_nosubclass_cpu.json (deflated 43%)
2025-03-14T04:56:57.1133101Z   adding: test/test-reports/aotdispatcher_training_nosubclass_cpu.json (deflated 42%)
2025-03-14T04:56:57.1133940Z   adding: test/test-reports/aotdispatcher_inference_subclass_cpu.json (deflated 43%)
2025-03-14T04:56:57.1134779Z   adding: test/test-reports/aotdispatcher_training_subclass_cpu.json (deflated 42%)
2025-03-14T04:56:57.1135579Z   adding: test/test-reports/aotdispatcher_partitioner_cpu.json (deflated 46%)
2025-03-14T04:56:57.1136402Z   adding: test/test-reports/aotdispatcher_partitioner_cpu2.json (deflated 42%)
2025-03-14T04:56:57.1137227Z   adding: test/test-reports/basic_modules_ListOfLinears_eager.json (deflated 39%)
2025-03-14T04:56:57.1138047Z   adding: test/test-reports/basic_modules_ListOfLinears_inductor.json (deflated 40%)
2025-03-14T04:56:57.1138885Z   adding: test/test-reports/basic_modules_ListOfLinears_inductor_gpu.json (deflated 39%)
2025-03-14T04:56:57.1139821Z   adding: test/test-reports/basic_modules_ListOfLinears_inductor_gpu_force_shape_pad.json (deflated 39%)
2025-03-14T04:56:57.1140622Z   adding: test/test-reports/float_args.json (deflated 41%)
2025-03-14T04:56:57.1141247Z   adding: test/test-reports/sum_floordiv_regression.json (deflated 39%)
2025-03-14T04:56:57.1141865Z   adding: test/test-reports/symint_sum.json (deflated 34%)
2025-03-14T04:56:57.1142472Z   adding: test/test-reports/symint_sum_loop.json (deflated 34%)
2025-03-14T04:56:57.1143124Z   adding: test/test-reports/update_hint_regression.json (deflated 40%)
2025-03-14T04:56:57.1169676Z ##[group]Run # Remove any previous test reports if they exist
2025-03-14T04:56:57.1170278Z # Remove any previous test reports if they exist
2025-03-14T04:56:57.1170914Z rm -f test-reports-*.zip
2025-03-14T04:56:57.1171524Z zip -r "test-reports-${FILE_SUFFIX}.zip" test/test-reports -i '*.xml' -i '*.csv'
2025-03-14T04:56:57.1176775Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:56:57.1177258Z env:
2025-03-14T04:56:57.1177533Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:57.1177966Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:57.1178656Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:57.1179486Z   FILE_SUFFIX: test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_38755508671
2025-03-14T04:56:57.1180080Z ##[endgroup]
2025-03-14T04:56:57.1218320Z   adding: test/test-reports/pr_time_benchmarks_results.csv (deflated 73%)
2025-03-14T04:56:57.1219298Z   adding: test/test-reports/new_expected_results.csv (deflated 78%)
2025-03-14T04:56:57.1248232Z ##[group]Run # Remove any previous usage logs if they exist
2025-03-14T04:56:57.1248830Z # Remove any previous usage logs if they exist
2025-03-14T04:56:57.1249307Z rm -f logs-*.zip
2025-03-14T04:56:57.1249891Z # this workflow is also run in bazel build test, but we dont generate usage reports for it
2025-03-14T04:56:57.1250565Z # so check to see if the file exists first
2025-03-14T04:56:57.1251130Z if [ -f 'usage_log.txt' ]; then
2025-03-14T04:56:57.1251591Z     zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt'
2025-03-14T04:56:57.1252026Z fi
2025-03-14T04:56:57.1252482Z if find "test/test-reports" -name "*.log" 2>/dev/null | grep -q .; then
2025-03-14T04:56:57.1279162Z     zip -r "logs-${FILE_SUFFIX}.zip" test/test-reports -i '*.log'
2025-03-14T04:56:57.1279670Z fi
2025-03-14T04:56:57.1284642Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:56:57.1285114Z env:
2025-03-14T04:56:57.1285368Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:57.1285776Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:57.1286471Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:57.1287268Z   FILE_SUFFIX: test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_38755508671
2025-03-14T04:56:57.1287835Z ##[endgroup]
2025-03-14T04:56:57.1338560Z   adding: usage_log.txt (deflated 98%)
2025-03-14T04:56:57.1388522Z ##[group]Run # Remove any previous debugging artifacts if they exist
2025-03-14T04:56:57.1389153Z # Remove any previous debugging artifacts if they exist
2025-03-14T04:56:57.1389632Z rm -f debug-*.zip
2025-03-14T04:56:57.1390048Z if [ -d 'test/debug' ]; then
2025-03-14T04:56:57.1390666Z   zip -r "debug-${FILE_SUFFIX}.zip" test/debug
2025-03-14T04:56:57.1391098Z fi
2025-03-14T04:56:57.1395975Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:56:57.1396443Z env:
2025-03-14T04:56:57.1396704Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:57.1397134Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:57.1397818Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:57.1398659Z   FILE_SUFFIX: test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_38755508671
2025-03-14T04:56:57.1399254Z ##[endgroup]
2025-03-14T04:56:57.1482006Z ##[group]Run seemethere/upload-artifact-s3@v5
2025-03-14T04:56:57.1482500Z with:
2025-03-14T04:56:57.1482783Z   s3-bucket: gha-artifacts
2025-03-14T04:56:57.1483155Z   s3-prefix: pytorch/pytorch/13849515290/1/artifact

2025-03-14T04:56:57.1483551Z   retention-days: 14
2025-03-14T04:56:57.1483836Z   if-no-files-found: warn
2025-03-14T04:56:57.1484147Z   path: test-jsons-*.zip
2025-03-14T04:56:57.1484448Z   name: artifact
2025-03-14T04:56:57.1484724Z   region: us-east-1
2025-03-14T04:56:57.1484987Z env:
2025-03-14T04:56:57.1485248Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:57.1485622Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:57.1486385Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:57.1486931Z ##[endgroup]
2025-03-14T04:56:57.5053810Z NOTE: s3-prefix specified, ignoring name parameter
2025-03-14T04:56:57.5055196Z With the provided path, there will be 1 file uploaded
2025-03-14T04:56:57.5056585Z Uploading to s3 prefix: pytorch/pytorch/13849515290/1/artifact
2025-03-14T04:56:57.5087458Z Starting upload of test-jsons-test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_38755508671.zip
2025-03-14T04:56:57.6611788Z Finished upload of test-jsons-test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_38755508671.zip
2025-03-14T04:56:57.6771032Z ##[group]Run seemethere/upload-artifact-s3@v5
2025-03-14T04:56:57.6771444Z with:
2025-03-14T04:56:57.6771730Z   s3-bucket: gha-artifacts
2025-03-14T04:56:57.6772292Z   s3-prefix: pytorch/pytorch/13849515290/1/artifact

2025-03-14T04:56:57.6772712Z   retention-days: 14
2025-03-14T04:56:57.6773039Z   if-no-files-found: error
2025-03-14T04:56:57.6773397Z   path: test-reports-*.zip
2025-03-14T04:56:57.6773732Z   name: artifact
2025-03-14T04:56:57.6774003Z   region: us-east-1
2025-03-14T04:56:57.6774291Z env:
2025-03-14T04:56:57.6774563Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:57.6774965Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:57.6775610Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:57.6776192Z ##[endgroup]
2025-03-14T04:56:57.9914567Z NOTE: s3-prefix specified, ignoring name parameter
2025-03-14T04:56:57.9915949Z With the provided path, there will be 1 file uploaded
2025-03-14T04:56:57.9917340Z Uploading to s3 prefix: pytorch/pytorch/13849515290/1/artifact
2025-03-14T04:56:57.9945999Z Starting upload of test-reports-test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_38755508671.zip
2025-03-14T04:56:58.1842105Z Finished upload of test-reports-test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_38755508671.zip
2025-03-14T04:56:58.2008717Z ##[group]Run seemethere/upload-artifact-s3@v5
2025-03-14T04:56:58.2009138Z with:
2025-03-14T04:56:58.2009434Z   s3-bucket: gha-artifacts
2025-03-14T04:56:58.2009842Z   s3-prefix: pytorch/pytorch/13849515290/1/artifact

2025-03-14T04:56:58.2010276Z   retention-days: 14
2025-03-14T04:56:58.2010608Z   if-no-files-found: ignore
2025-03-14T04:56:58.2010954Z   path: logs-*.zip
2025-03-14T04:56:58.2011267Z   name: artifact
2025-03-14T04:56:58.2011543Z   region: us-east-1
2025-03-14T04:56:58.2011838Z env:
2025-03-14T04:56:58.2012119Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:58.2012529Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:58.2013184Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:58.2013764Z ##[endgroup]
2025-03-14T04:56:58.5143612Z NOTE: s3-prefix specified, ignoring name parameter
2025-03-14T04:56:58.5144998Z With the provided path, there will be 1 file uploaded
2025-03-14T04:56:58.5146441Z Uploading to s3 prefix: pytorch/pytorch/13849515290/1/artifact
2025-03-14T04:56:58.5173921Z Starting upload of logs-test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_38755508671.zip
2025-03-14T04:56:58.6670043Z Finished upload of logs-test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_38755508671.zip
2025-03-14T04:56:58.6827522Z ##[group]Run seemethere/upload-artifact-s3@v5
2025-03-14T04:56:58.6827940Z with:
2025-03-14T04:56:58.6828237Z   s3-bucket: gha-artifacts
2025-03-14T04:56:58.6828638Z   s3-prefix: pytorch/pytorch/13849515290/1/artifact

2025-03-14T04:56:58.6829073Z   retention-days: 14
2025-03-14T04:56:58.6829402Z   if-no-files-found: ignore
2025-03-14T04:56:58.6829748Z   path: debug-*.zip
2025-03-14T04:56:58.6830046Z   name: artifact
2025-03-14T04:56:58.6830346Z   region: us-east-1
2025-03-14T04:56:58.6830640Z env:
2025-03-14T04:56:58.6830933Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:58.6831334Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:58.6832137Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:58.6832719Z ##[endgroup]
2025-03-14T04:56:58.9908701Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded.
2025-03-14T04:56:59.0080351Z ##[group]Run # shellcheck disable=SC2156
2025-03-14T04:56:59.0080818Z # shellcheck disable=SC2156
2025-03-14T04:56:59.0081489Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \;
2025-03-14T04:56:59.0087131Z shell: /usr/bin/bash -e {0}
2025-03-14T04:56:59.0087478Z env:
2025-03-14T04:56:59.0087732Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:59.0088132Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:59.0088774Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:59.0089512Z ##[endgroup]
2025-03-14T04:56:59.1856331Z Prepare all required actions
2025-03-14T04:56:59.1856820Z Getting action download info
2025-03-14T04:56:59.3035363Z ##[group]Run ./.github/actions/upload-utilization-stats
2025-03-14T04:56:59.3035802Z with:
2025-03-14T04:56:59.3036077Z   job_id: 38755508671
2025-03-14T04:56:59.3036601Z   job_name: cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu)
2025-03-14T04:56:59.3037210Z   workflow_name: pull
2025-03-14T04:56:59.3037540Z   workflow_run_id: 13849515290
2025-03-14T04:56:59.3037885Z   workflow_attempt: 1
2025-03-14T04:56:59.3038185Z env:
2025-03-14T04:56:59.3038458Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:59.3038854Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:59.3039500Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:59.3040082Z ##[endgroup]
2025-03-14T04:56:59.3066230Z ##[group]Run echo "workflow_id: 13849515290"
2025-03-14T04:56:59.3066761Z echo "workflow_id: 13849515290"
2025-03-14T04:56:59.3067158Z echo "workflow_attempt: 1"
2025-03-14T04:56:59.3067540Z echo "workflow_Name: pull"
2025-03-14T04:56:59.3067910Z echo "job_id: 38755508671"
2025-03-14T04:56:59.3068563Z echo "job_name:  cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu)"
2025-03-14T04:56:59.3073739Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:56:59.3074196Z env:
2025-03-14T04:56:59.3074471Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:59.3074868Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:59.3075505Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:59.3076084Z ##[endgroup]
2025-03-14T04:56:59.3097011Z workflow_id: 13849515290
2025-03-14T04:56:59.3097412Z workflow_attempt: 1
2025-03-14T04:56:59.3097744Z workflow_Name: pull
2025-03-14T04:56:59.3098139Z job_id: 38755508671
2025-03-14T04:56:59.3098743Z job_name:  cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu)
2025-03-14T04:56:59.3141299Z ##[group]Run nick-fields/retry@v3.0.0
2025-03-14T04:56:59.3141675Z with:
2025-03-14T04:56:59.3141950Z   shell: bash
2025-03-14T04:56:59.3142219Z   timeout_minutes: 5
2025-03-14T04:56:59.3142502Z   max_attempts: 5
2025-03-14T04:56:59.3142781Z   retry_wait_seconds: 30
2025-03-14T04:56:59.3143306Z   command: set -eu
python3 -m pip install python-dateutil==2.8.2 boto3==1.35.42 pandas==2.1.3

2025-03-14T04:56:59.3143881Z   polling_interval_seconds: 1
2025-03-14T04:56:59.3144210Z   warning_on_retry: true
2025-03-14T04:56:59.3144531Z   continue_on_error: false
2025-03-14T04:56:59.3144835Z env:
2025-03-14T04:56:59.3145087Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:56:59.3145460Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:56:59.3146082Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:56:59.3146671Z ##[endgroup]
2025-03-14T04:56:59.6360184Z Defaulting to user installation because normal site-packages is not writeable
2025-03-14T04:56:59.6511705Z Requirement already satisfied: python-dateutil==2.8.2 in /home/ec2-user/.local/lib/python3.9/site-packages (2.8.2)
2025-03-14T04:57:00.6555997Z Collecting boto3==1.35.42
2025-03-14T04:57:00.6579332Z   Using cached boto3-1.35.42-py3-none-any.whl (139 kB)
2025-03-14T04:57:00.6604822Z Requirement already satisfied: pandas==2.1.3 in /home/ec2-user/.local/lib/python3.9/site-packages (2.1.3)
2025-03-14T04:57:00.6619541Z Requirement already satisfied: six>=1.5 in /usr/lib/python3.9/site-packages (from python-dateutil==2.8.2) (1.15.0)
2025-03-14T04:57:00.6666386Z Requirement already satisfied: botocore<1.36.0,>=1.35.42 in /home/ec2-user/.local/lib/python3.9/site-packages (from boto3==1.35.42) (1.35.99)
2025-03-14T04:57:00.6668855Z Requirement already satisfied: s3transfer<0.11.0,>=0.10.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from boto3==1.35.42) (0.10.4)
2025-03-14T04:57:00.6673244Z Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /usr/lib/python3.9/site-packages (from boto3==1.35.42) (0.10.0)
2025-03-14T04:57:00.7500723Z Requirement already satisfied: tzdata>=2022.1 in /home/ec2-user/.local/lib/python3.9/site-packages (from pandas==2.1.3) (2025.1)
2025-03-14T04:57:00.7507595Z Requirement already satisfied: numpy<2,>=1.22.4 in /home/ec2-user/.local/lib/python3.9/site-packages (from pandas==2.1.3) (1.26.4)
2025-03-14T04:57:00.7513492Z Requirement already satisfied: pytz>=2020.1 in /usr/lib/python3.9/site-packages (from pandas==2.1.3) (2022.7.1)
2025-03-14T04:57:00.7574235Z Requirement already satisfied: urllib3<1.27,>=1.25.4 in /usr/lib/python3.9/site-packages (from botocore<1.36.0,>=1.35.42->boto3==1.35.42) (1.25.10)
2025-03-14T04:57:00.8409708Z Installing collected packages: boto3
2025-03-14T04:57:00.8410172Z   Attempting uninstall: boto3
2025-03-14T04:57:00.8410629Z     Found existing installation: boto3 1.35.33
2025-03-14T04:57:00.8502699Z     Uninstalling boto3-1.35.33:
2025-03-14T04:57:00.8516526Z       Successfully uninstalled boto3-1.35.33
2025-03-14T04:57:00.9055225Z Successfully installed boto3-1.35.42
2025-03-14T04:57:01.3913385Z Command completed after 1 attempt(s).
2025-03-14T04:57:01.3976458Z ##[group]Run python3 -m tools.stats.upload_utilization_stats.upload_utilization_stats \
2025-03-14T04:57:01.3977289Z python3 -m tools.stats.upload_utilization_stats.upload_utilization_stats \
2025-03-14T04:57:01.3977861Z   --workflow-run-id "13849515290" \
2025-03-14T04:57:01.3978271Z   --workflow-name "pull" \
2025-03-14T04:57:01.3978661Z   --workflow-run-attempt "1" \
2025-03-14T04:57:01.3979047Z   --job-id "38755508671" \
2025-03-14T04:57:01.3979666Z   --job-name "cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu)"
2025-03-14T04:57:01.3987150Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:57:01.3987609Z env:
2025-03-14T04:57:01.3987919Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:57:01.3988323Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:57:01.3988963Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:57:01.3989538Z ##[endgroup]
2025-03-14T04:57:02.6441430Z repo: pytorch/pytorch
2025-03-14T04:57:02.6443070Z Downloading logs-test-pr_time_benchmarks-1-1-linux.g4dn.metal.nvidia.gpu_38755508671.zip
2025-03-14T04:57:02.6444826Z Converted Log Model: UtilizationMetadata:
2025-03-14T04:57:02.6449232Z  UtilizationMetadata(level='metadata', workflow_id='13849515290', job_id='38755508671', workflow_name='pull', job_name='cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu)', usage_collect_interval=1.0, data_model_version=1.0, start_at=1741927524, gpu_count=8, cpu_count=96, gpu_type='pynvml', error=None)
2025-03-14T04:57:02.6451230Z [Db Segments] detected pytest cmd: 15, generated segments: 15
2025-03-14T04:57:02.6451760Z [db model] Peek db timeseries 
2025-03-14T04:57:02.6452097Z :{
2025-03-14T04:57:02.6452366Z     "created_at": 1741928222,
2025-03-14T04:57:02.6452706Z     "type": "utilization",
2025-03-14T04:57:02.6453018Z     "tags": [
2025-03-14T04:57:02.6453294Z         "record"
2025-03-14T04:57:02.6453568Z     ],
2025-03-14T04:57:02.6453838Z     "time_stamp": 1741927524,
2025-03-14T04:57:02.6454162Z     "repo": "pytorch/pytorch",
2025-03-14T04:57:02.6454504Z     "workflow_id": 13849515290,
2025-03-14T04:57:02.6454839Z     "run_attempt": 1,
2025-03-14T04:57:02.6455141Z     "job_id": 38755508671,
2025-03-14T04:57:02.6455470Z     "workflow_name": "pull",
2025-03-14T04:57:02.6456045Z     "job_name": "cuda12.4-py3.10-gcc9-sm75 / test (pr_time_benchmarks, 1, 1, linux.g4dn.metal.nvidia.gpu)",
2025-03-14T04:57:02.6456647Z     "json_data": "{}"
2025-03-14T04:57:02.6456935Z }
2025-03-14T04:57:02.6457505Z Writing 1 documents to S3 ossci-utilization/util_metadata/v_1.0/pytorch/pytorch/13849515290/1/38755508671/metadata
2025-03-14T04:57:02.6458932Z Done! Finish writing document to S3 ossci-utilization/util_metadata/v_1.0/pytorch/pytorch/13849515290/1/38755508671/metadata 
2025-03-14T04:57:02.6459997Z Writing 136 documents to S3 ossci-utilization/util_timeseries/v_1.0/pytorch/pytorch/13849515290/1/38755508671/time_series
2025-03-14T04:57:02.6461085Z Done! Finish writing document to S3 ossci-utilization/util_timeseries/v_1.0/pytorch/pytorch/13849515290/1/38755508671/time_series 
2025-03-14T04:57:02.7336537Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main
2025-03-14T04:57:02.7337065Z with:
2025-03-14T04:57:02.7337344Z env:
2025-03-14T04:57:02.7337627Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:57:02.7338043Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:57:02.7338718Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:57:02.7339326Z ##[endgroup]
2025-03-14T04:57:02.7364929Z ##[group]Run set -eou pipefail
2025-03-14T04:57:02.7365473Z set -eou pipefail
2025-03-14T04:57:02.7365810Z 
2025-03-14T04:57:02.7366254Z echo "Holding runner for 2 hours until all ssh sessions have logged out"
2025-03-14T04:57:02.7366800Z for _ in $(seq 1440); do
2025-03-14T04:57:02.7367199Z     # Break if no ssh session exists anymore
2025-03-14T04:57:02.7367627Z     if [ "$(who)" = "" ]; then
2025-03-14T04:57:02.7367984Z       break
2025-03-14T04:57:02.7368331Z     fi
2025-03-14T04:57:02.7368599Z     echo "."
2025-03-14T04:57:02.7368895Z     sleep 5
2025-03-14T04:57:02.7369184Z done
2025-03-14T04:57:02.7374526Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:57:02.7374950Z env:
2025-03-14T04:57:02.7375213Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:57:02.7375585Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:57:02.7376201Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:57:02.7376775Z ##[endgroup]
2025-03-14T04:57:02.7399349Z Holding runner for 2 hours until all ssh sessions have logged out
2025-03-14T04:57:02.7458030Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty
2025-03-14T04:57:02.7458764Z # ignore expansion of "docker ps -q" since it could be empty
2025-03-14T04:57:02.7459276Z # shellcheck disable=SC2046
2025-03-14T04:57:02.7459688Z docker stop $(docker ps -q) || true
2025-03-14T04:57:02.7460087Z # Prune all of the docker images
2025-03-14T04:57:02.7460460Z docker system prune -af
2025-03-14T04:57:02.7465661Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:57:02.7466085Z env:
2025-03-14T04:57:02.7466335Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:57:02.7466724Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2025-03-14T04:57:02.7467346Z   DOCKER_CONTAINER_ID: 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:57:02.7467919Z ##[endgroup]
2025-03-14T04:57:03.3942316Z 6761c124b60f
2025-03-14T04:57:03.8254949Z Deleted Containers:
2025-03-14T04:57:03.8255724Z 6761c124b60f67b92b745aaef0b15ae6f03b49db2ceb690f2a90388fde20da73
2025-03-14T04:57:03.8256196Z 
2025-03-14T04:57:09.3713213Z Deleted Images:
2025-03-14T04:57:09.3714431Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:57:09.3716384Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks@sha256:679ab340c4f18c247440dc36ea7b4cea489f03bffd640ad0dbf06f3e027a1eaf
2025-03-14T04:57:09.3717853Z deleted: sha256:67c80a026a8320d7a154e47f931cc5dfab55468390062655a56216de80309c6a
2025-03-14T04:57:09.3718744Z deleted: sha256:40625ca1a2b44a9d41fbf46180bff383e3d14e979376c10f030694e9c187e1db
2025-03-14T04:57:09.3719553Z deleted: sha256:c12d931a2d6b397904ca4b0cb162b287efad4b3ac4afeff6e19f26985a858efc
2025-03-14T04:57:09.3720709Z deleted: sha256:d665950d366220cd3bd4f34023b007484cb4b11a3bea4327fd7d94aaebf56885
2025-03-14T04:57:09.3721507Z deleted: sha256:1be7efb86bb15f1d6d7bc71aeed5d5e031134c9fc7307d80f420486e6dce2936
2025-03-14T04:57:09.3722308Z deleted: sha256:9bdcb736e64671151598b2e6874ef71312ef03cec911e4d38a5f99f272a56210
2025-03-14T04:57:09.3723103Z deleted: sha256:acf30f2c7e802c41d1541987e21bbbcc96517089fceffcc000dfc914af4d1899
2025-03-14T04:57:09.3724075Z deleted: sha256:488c190054f2bcf83fbd1b0cb58c4f8b4ddd2b68a31caa95fe55045ac11e0e40
2025-03-14T04:57:09.3724903Z deleted: sha256:bf36f2eaa9ad07ef179df2884252f67cf47b224451e7afbc4d111db31f0db1a4
2025-03-14T04:57:09.3725675Z deleted: sha256:7104bd58e85ebc19703ad7d5c9f0792a41b80a35034cd8754588a29d2b0f1dad
2025-03-14T04:57:09.3726460Z deleted: sha256:98ac7500e216a52ab6627ac6bb56990b4e306338e7b899c94b697e60bc14bed7
2025-03-14T04:57:09.3727243Z deleted: sha256:bfde259e178b914612a38b278521fee4447ae5a041144cf279caa1868faf9a70
2025-03-14T04:57:09.3728052Z deleted: sha256:b3cca56f6b705a19ef11fa8d31a87cda67efe0d7682b79ccfc5b7f3941f166ff
2025-03-14T04:57:09.3728857Z deleted: sha256:19a7d1be6b78f162ab9f8d98d168d928ba9eba67a5caf574b4f3f4ff8e84ac16
2025-03-14T04:57:09.3729646Z deleted: sha256:6c3c60101e9433d7538d5f38f1ba08fdacf21b399787877a40ff0a76d9fd3a1b
2025-03-14T04:57:09.3730422Z deleted: sha256:42d7e305829b0b65392b5262e577b2cc07ea6d6d417668c0ec3ad43d8beeced7
2025-03-14T04:57:09.3731202Z deleted: sha256:657b97ce986866f52699ff93293f026f18ac6bdb14847b8f2456c18b28228c8a
2025-03-14T04:57:09.3731989Z deleted: sha256:d77c3c6d67183eb3818da5dd8c18be5365e39fcf4fab6fc6993aa06eccc85469
2025-03-14T04:57:09.3732778Z deleted: sha256:35b21180f98b76bd56412e5b29c00471dfd50989f108b96266a870669ef63ca7
2025-03-14T04:57:09.3733563Z deleted: sha256:b840fdafae7230be08f4ac7f11e3b3ceb508922623d17f08bea1b2e9d7b66399
2025-03-14T04:57:09.3734357Z deleted: sha256:f5786198f86bae5f8cc23759fb32e9e04dd4611ec61d420eb0c936c489df945a
2025-03-14T04:57:09.3735156Z deleted: sha256:08ffcf381aef0d3d0f2e7abddff3b12ff3165650f90927952a4c373024dafec5
2025-03-14T04:57:09.3735945Z deleted: sha256:a88ddd350809d04e885a7f25234421f4f849447347a4ab2b4eba46700aab9911
2025-03-14T04:57:09.3736723Z deleted: sha256:fbc013b190410d07be3a3af3749bc56e54536437c4318ae88acdcc40ad1c2e46
2025-03-14T04:57:09.3737510Z deleted: sha256:fca8a3e62cae8759cf51d56428276cefa80fa163bf33391727bbcc5783957b1f
2025-03-14T04:57:09.3738301Z deleted: sha256:1d23342d3b7ae5a9d49c51bb207777d326fc9671fbb15933daa107b2c3f0b015
2025-03-14T04:57:09.3739189Z deleted: sha256:0e0396b6936a6d6ca27106c65a670dbf59bb9c470c8f2019db4b80ee5156d0ab
2025-03-14T04:57:09.3739969Z deleted: sha256:62a1fa797919eb56207c3d0e78cdb2a392b1e96f1038da33268364f162943668
2025-03-14T04:57:09.3740751Z deleted: sha256:97082647f4a7fd45e5ec099920d85c53d503e8ffbee7308e677e8e3c39e6e954
2025-03-14T04:57:09.3741536Z deleted: sha256:f3a002fd4e939240dad2573639d6056beae8dbc013d7aba2ec9b36d28e261adc
2025-03-14T04:57:09.3742337Z deleted: sha256:9a5e5bebc3f658e514163eab5efb2597200de2313095655e52b4429faa4c044a
2025-03-14T04:57:09.3743132Z deleted: sha256:58b77e89e7ba88ddab08aed9c35cbaf2f0632c394eac74b0964750c9932fb320
2025-03-14T04:57:09.3743947Z deleted: sha256:9a6af9f0ccbe13ee9d4847a4fa6beed566f1ab25f7f3c4129d3be1eb6304ffcc
2025-03-14T04:57:09.3744752Z deleted: sha256:c0c5540f012da3fb7de8dc6e46e6c44ba343b978e5f833ed341329f54d0832ee
2025-03-14T04:57:09.3745548Z deleted: sha256:344150374c889ebe1cd261b8c397a4fa403de19b476216b12efc668ef437a3df
2025-03-14T04:57:09.3746333Z deleted: sha256:a642b956b1ed4d9958e16766ee5e4521981cd2d6e77faa11d6e936b4301ddb7c
2025-03-14T04:57:09.3747119Z deleted: sha256:59bd7728f83e385becd5fbce3fc217a333491b2fa3c1166b83a5377499b4eb61
2025-03-14T04:57:09.3747904Z deleted: sha256:4855a2611a95bfdbc97eed1d957f76987171395ad56eeb6863855c0af33f8b00
2025-03-14T04:57:09.3748692Z deleted: sha256:03eaea063af89cce9146e3530c57d61651d90b23fdd9f7a707a3c91df5bf0fb9
2025-03-14T04:57:09.3749488Z deleted: sha256:4b561a0a79b66e47df272ba55b0531ca2b3e3c7c50caabc03ac4994e0a830a27
2025-03-14T04:57:09.3750362Z deleted: sha256:10853587ad7261f3dcf57b470a70c43129dc826d3dc25090559a7ac22bcfd0db
2025-03-14T04:57:09.3751141Z deleted: sha256:761406860364f6cdbb44690d8df43348066b8abd4e5f3880919b8f6cd5e5f945
2025-03-14T04:57:09.3751922Z deleted: sha256:a0759d8692c15c9b4116f47901797782ca6b2cbcb5dd9af599358240cd56e543
2025-03-14T04:57:09.3752717Z deleted: sha256:b59dcda6e10737d6eb188890273e6174e2b1bfe59f7611965b31630353627a9f
2025-03-14T04:57:09.3754546Z deleted: sha256:4e7af660d883f2d8aeb960e327b37031467e26ce5409354d87ccf50dc3e67bdf
2025-03-14T04:57:09.3755356Z deleted: sha256:d656a236e7891db67aa6ff07c8e313a398452ed0886a25ff2f9f47c54a45623d
2025-03-14T04:57:09.3756162Z deleted: sha256:00c7bb60f0918db8deb3e0e7145eabe39f0fe86a3ce58b31502331f5a6fa3b5f
2025-03-14T04:57:09.3756963Z deleted: sha256:045f73f8a482b68584c74d83fd76979d2a5fc143ab27370598d3237cc68063e3
2025-03-14T04:57:09.3757814Z deleted: sha256:6ebf9bd537c40c30663e40e73b98b23cb24ab102b6eef14d3a5e8d3ae5e88d55
2025-03-14T04:57:09.3758698Z deleted: sha256:f760d3392cbd23e26d73f7d7b9525d051ef26ae2e9dfcc9767a336fbcb4fbe08
2025-03-14T04:57:09.3759491Z deleted: sha256:2a996c4bc78020a4310a7cb8a84233a41c402dd601b650e1153b79715f672c2b
2025-03-14T04:57:09.3760279Z deleted: sha256:c1272ab132d026e38f3e2cffb45b1961d5fac5435edb360a4592154b77470687
2025-03-14T04:57:09.3761086Z deleted: sha256:77479a031fe3baf6c6bdf2900bcb9071f343429a0f9acf4f7e643c4f4a34f654
2025-03-14T04:57:09.3761892Z deleted: sha256:86bc58379547f5f34fa8bf1933bef29fed6008f1a5cd65ad8814f20266c50e19
2025-03-14T04:57:09.3762679Z deleted: sha256:706400f4ce95379ffa4446b2cae323625af3813c59e33422a4443c3c2b52de81
2025-03-14T04:57:09.3763459Z deleted: sha256:c04697c3225def0526b7c04d6ad2187924ec250a205773de146c9649af09f9d9
2025-03-14T04:57:09.3764243Z deleted: sha256:808fb4279092c552aadedc4df525762b07894d64bd44c4cb9ff3fc15ed7f54b0
2025-03-14T04:57:09.3765039Z deleted: sha256:34c4c12b7441407f0e8cfea25737a885afe97185c88fa19228411d81a778d7ad
2025-03-14T04:57:09.3765831Z deleted: sha256:2b091c3e12a745a8e9e8205d5d68f467d0f6f37a11f5b26f78e13b57e91243b6
2025-03-14T04:57:09.3766630Z deleted: sha256:2d9868b1e9c9b4eaa0cd0ae2e5203ad7e80723770de2bd17140b7cadbaac0959
2025-03-14T04:57:09.3767428Z deleted: sha256:d788ef62f670c9d1443bbdfeb71f64254f260c2fa61d43742ee97c50373b10f7
2025-03-14T04:57:09.3768239Z deleted: sha256:aa1b164cd38fcd42160b7acdc18f531e92c9047aada91f79a6c3f6de130cf893
2025-03-14T04:57:09.3769054Z deleted: sha256:bf0182b1b8e542b153cfe5eca8aab6a927b4d07c3d5e6aea98e39d750938b2ff
2025-03-14T04:57:09.3769840Z deleted: sha256:857d97870c85cd9af7b83dd04450c5d4bf8aee2e5caf17e6f93e229c9de247f7
2025-03-14T04:57:09.3770631Z deleted: sha256:3f265b751cf81b52963fdb896139978759e48c35f17f34d9d9bd5fadd69909a7
2025-03-14T04:57:09.3771415Z deleted: sha256:b473ef368d4cd4ba8760a917f28ab30ec48e0063fca3f689a9bfc7ab3b0ceeb4
2025-03-14T04:57:09.3772312Z deleted: sha256:5dc5e1c2c677be2e4ab1edc3031a8fd50233ac09d7cd11c4f7fead8f0de63dab
2025-03-14T04:57:09.3773083Z deleted: sha256:8dde1ca9e7b7359d881f1d12c03e170dd91184924e14388768aafc2c141d43a7
2025-03-14T04:57:09.3773849Z deleted: sha256:89d8c3f0d0649a7293e50c7bd007e97600e51f8cd2fac38af3f6ea59d85c3987
2025-03-14T04:57:09.3774620Z deleted: sha256:bd8ad07b09ab00f36c0fd5ecb3db3906aa0262f5a01ab1db560c1804885e8b79
2025-03-14T04:57:09.3775393Z deleted: sha256:959299be5fe77ef16ef79f1bf605e7886b8b39fc935f5ae0097dc1eedd1ba3f4
2025-03-14T04:57:09.3776161Z deleted: sha256:3887d34f9c36dbe600fc7c62b7dfb8c7882e0abdd4d5058b9543c61f8e6c048c
2025-03-14T04:57:09.3776930Z deleted: sha256:3386e25983944ce3b2ebdde10d840832d581a7dfafd1df66568822a18979e268
2025-03-14T04:57:09.3777689Z deleted: sha256:d3a036e6bd36f014b3a1540d5b848655ecdf72a20dc81c0da5f80430700cfa9a
2025-03-14T04:57:09.3778454Z deleted: sha256:96d0556708ff4ec0d5d727d5947d6cb29a9d6c16e1a1ee76a92dc85aa3ea8f25
2025-03-14T04:57:09.3779226Z deleted: sha256:f3c2fb3abccb3a33dbf204d3f0d8857537348e6c1d3cc87a431afe866435d41b
2025-03-14T04:57:09.3779980Z deleted: sha256:45cd6c9a41f12991ee60a89e775abb604b0892ef3936526c18184bbbf4de57da
2025-03-14T04:57:09.3780805Z deleted: sha256:b3cacd0508bbfd2fd668f30c539033e7c6b606bf669258c8e8a3cbf80bf050c6
2025-03-14T04:57:09.3781577Z deleted: sha256:de72ce930361a96c151a8a4725f8360f33df6c812b6ed6d3dd5b6e6710a5d2c0
2025-03-14T04:57:09.3782328Z deleted: sha256:2e5d223d2bfa0f47e628b631a540d20c284dabb380ad9ee4caec65bf8d0ecde2
2025-03-14T04:57:09.3783080Z deleted: sha256:14a85e604a0401093323d356b3bb7f75a8473abd712b4bf1a2ef01fa432b344c
2025-03-14T04:57:09.3783890Z deleted: sha256:15a95a1e8f47d7c18dcbf9a6e2fc530cddcb161c0093c663181f3e73dc9f599e
2025-03-14T04:57:09.3784642Z deleted: sha256:106e8431b412f51ccd75ea46a2d5cb4343b23273cbcf50188377cb93aa9a6d82
2025-03-14T04:57:09.3785073Z 
2025-03-14T04:57:09.3785230Z Total reclaimed space: 50.35GB
2025-03-14T04:57:09.3897310Z Post job cleanup.
2025-03-14T04:57:09.3955412Z Post job cleanup.
2025-03-14T04:57:09.4910706Z [command]/usr/bin/git version
2025-03-14T04:57:09.4944506Z git version 2.47.1
2025-03-14T04:57:09.4980034Z Copying '/home/ec2-user/.gitconfig' to '/home/ec2-user/actions-runner/_work/_temp/65b60a5d-1e1e-465b-acaa-cdeba0fff406/.gitconfig'
2025-03-14T04:57:09.4991932Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/65b60a5d-1e1e-465b-acaa-cdeba0fff406' before making global git config changes
2025-03-14T04:57:09.4992998Z Adding repository directory to the temporary git global config as a safe directory
2025-03-14T04:57:09.4996079Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch
2025-03-14T04:57:09.5044211Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2025-03-14T04:57:09.5072038Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
2025-03-14T04:57:09.5352434Z Entering 'android/libs/fbjni'
2025-03-14T04:57:09.5403691Z Entering 'third_party/FP16'
2025-03-14T04:57:09.5461674Z Entering 'third_party/FXdiv'
2025-03-14T04:57:09.5514137Z Entering 'third_party/NNPACK'
2025-03-14T04:57:09.5569702Z Entering 'third_party/NVTX'
2025-03-14T04:57:09.5625344Z Entering 'third_party/VulkanMemoryAllocator'
2025-03-14T04:57:09.5681367Z Entering 'third_party/XNNPACK'
2025-03-14T04:57:09.5751015Z Entering 'third_party/benchmark'
2025-03-14T04:57:09.5805902Z Entering 'third_party/composable_kernel'
2025-03-14T04:57:09.5859989Z Entering 'third_party/cpp-httplib'
2025-03-14T04:57:09.5914261Z Entering 'third_party/cpuinfo'
2025-03-14T04:57:09.5972859Z Entering 'third_party/cudnn_frontend'
2025-03-14T04:57:09.6031308Z Entering 'third_party/cutlass'
2025-03-14T04:57:09.6099618Z Entering 'third_party/eigen'
2025-03-14T04:57:09.6156149Z Entering 'third_party/fbgemm'
2025-03-14T04:57:09.6211455Z Entering 'third_party/fbgemm/third_party/asmjit'
2025-03-14T04:57:09.6257876Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2025-03-14T04:57:09.6304995Z Entering 'third_party/fbgemm/third_party/cutlass'
2025-03-14T04:57:09.6357422Z Entering 'third_party/fbgemm/third_party/googletest'
2025-03-14T04:57:09.6412820Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2025-03-14T04:57:09.6467888Z Entering 'third_party/flash-attention'
2025-03-14T04:57:09.6524132Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-03-14T04:57:09.6579065Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-03-14T04:57:09.6643025Z Entering 'third_party/flatbuffers'
2025-03-14T04:57:09.6698321Z Entering 'third_party/fmt'
2025-03-14T04:57:09.6753586Z Entering 'third_party/gemmlowp/gemmlowp'
2025-03-14T04:57:09.6807235Z Entering 'third_party/gloo'
2025-03-14T04:57:09.6854894Z Entering 'third_party/googletest'
2025-03-14T04:57:09.6910018Z Entering 'third_party/ideep'
2025-03-14T04:57:09.6966712Z Entering 'third_party/ideep/mkl-dnn'
2025-03-14T04:57:09.7031759Z Entering 'third_party/ittapi'
2025-03-14T04:57:09.7084648Z Entering 'third_party/kineto'
2025-03-14T04:57:09.7134301Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-03-14T04:57:09.7188521Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-03-14T04:57:09.7245509Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-03-14T04:57:09.7298053Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-03-14T04:57:09.7345547Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-03-14T04:57:09.7401817Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-03-14T04:57:09.7457656Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-03-14T04:57:09.7504502Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-03-14T04:57:09.7554981Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-03-14T04:57:09.7615219Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-03-14T04:57:09.7676136Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-03-14T04:57:09.7728232Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-03-14T04:57:09.7779936Z Entering 'third_party/kleidiai'
2025-03-14T04:57:09.7828112Z Entering 'third_party/mimalloc'
2025-03-14T04:57:09.7882461Z Entering 'third_party/nlohmann'
2025-03-14T04:57:09.7935172Z Entering 'third_party/onnx'
2025-03-14T04:57:09.8004757Z Entering 'third_party/onnx/third_party/pybind11'
2025-03-14T04:57:09.8060822Z Entering 'third_party/opentelemetry-cpp'
2025-03-14T04:57:09.8110181Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-03-14T04:57:09.8168530Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-03-14T04:57:09.8217242Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-03-14T04:57:09.8268308Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-03-14T04:57:09.8319562Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-03-14T04:57:09.8368524Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-03-14T04:57:09.8425050Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-03-14T04:57:09.8476485Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-03-14T04:57:09.8526167Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-03-14T04:57:09.8579443Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-03-14T04:57:09.8644924Z Entering 'third_party/pocketfft'
2025-03-14T04:57:09.8693906Z Entering 'third_party/protobuf'
2025-03-14T04:57:09.8748723Z Entering 'third_party/protobuf/third_party/benchmark'
2025-03-14T04:57:09.8804616Z Entering 'third_party/protobuf/third_party/googletest'
2025-03-14T04:57:09.8853492Z Entering 'third_party/psimd'
2025-03-14T04:57:09.8899614Z Entering 'third_party/pthreadpool'
2025-03-14T04:57:09.8957497Z Entering 'third_party/pybind11'
2025-03-14T04:57:09.9005752Z Entering 'third_party/python-peachpy'
2025-03-14T04:57:09.9058969Z Entering 'third_party/sleef'
2025-03-14T04:57:09.9108369Z Entering 'third_party/tensorpipe'
2025-03-14T04:57:09.9169883Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-03-14T04:57:09.9226100Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-03-14T04:57:09.9277725Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-03-14T04:57:09.9327395Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-03-14T04:57:09.9376036Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-03-14T04:57:09.9451778Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2025-03-14T04:57:09.9473447Z http.https://github.com/.extraheader
2025-03-14T04:57:09.9483756Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader
2025-03-14T04:57:09.9510801Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
2025-03-14T04:57:09.9776781Z Entering 'android/libs/fbjni'
2025-03-14T04:57:09.9812285Z http.https://github.com/.extraheader
2025-03-14T04:57:09.9854085Z Entering 'third_party/FP16'
2025-03-14T04:57:09.9890943Z http.https://github.com/.extraheader
2025-03-14T04:57:09.9926398Z Entering 'third_party/FXdiv'
2025-03-14T04:57:09.9960571Z http.https://github.com/.extraheader
2025-03-14T04:57:10.0003851Z Entering 'third_party/NNPACK'
2025-03-14T04:57:10.0040551Z http.https://github.com/.extraheader
2025-03-14T04:57:10.0071919Z Entering 'third_party/NVTX'
2025-03-14T04:57:10.0105676Z http.https://github.com/.extraheader
2025-03-14T04:57:10.0141266Z Entering 'third_party/VulkanMemoryAllocator'
2025-03-14T04:57:10.0179778Z http.https://github.com/.extraheader
2025-03-14T04:57:10.0215771Z Entering 'third_party/XNNPACK'
2025-03-14T04:57:10.0249977Z http.https://github.com/.extraheader
2025-03-14T04:57:10.0299035Z Entering 'third_party/benchmark'
2025-03-14T04:57:10.0330834Z http.https://github.com/.extraheader
2025-03-14T04:57:10.0366345Z Entering 'third_party/composable_kernel'
2025-03-14T04:57:10.0402122Z http.https://github.com/.extraheader
2025-03-14T04:57:10.0443427Z Entering 'third_party/cpp-httplib'
2025-03-14T04:57:10.0481993Z http.https://github.com/.extraheader
2025-03-14T04:57:10.0513967Z Entering 'third_party/cpuinfo'
2025-03-14T04:57:10.0547790Z http.https://github.com/.extraheader
2025-03-14T04:57:10.0584456Z Entering 'third_party/cudnn_frontend'
2025-03-14T04:57:10.0619038Z http.https://github.com/.extraheader
2025-03-14T04:57:10.0655342Z Entering 'third_party/cutlass'
2025-03-14T04:57:10.0686842Z http.https://github.com/.extraheader
2025-03-14T04:57:10.0728892Z Entering 'third_party/eigen'
2025-03-14T04:57:10.0760776Z http.https://github.com/.extraheader
2025-03-14T04:57:10.0800031Z Entering 'third_party/fbgemm'
2025-03-14T04:57:10.0840061Z http.https://github.com/.extraheader
2025-03-14T04:57:10.0874245Z Entering 'third_party/fbgemm/third_party/asmjit'
2025-03-14T04:57:10.0904925Z http.https://github.com/.extraheader
2025-03-14T04:57:10.0939659Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2025-03-14T04:57:10.0970267Z http.https://github.com/.extraheader
2025-03-14T04:57:10.1008781Z Entering 'third_party/fbgemm/third_party/cutlass'
2025-03-14T04:57:10.1039467Z http.https://github.com/.extraheader
2025-03-14T04:57:10.1086329Z Entering 'third_party/fbgemm/third_party/googletest'
2025-03-14T04:57:10.1120920Z http.https://github.com/.extraheader
2025-03-14T04:57:10.1155922Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2025-03-14T04:57:10.1190678Z http.https://github.com/.extraheader
2025-03-14T04:57:10.1225790Z Entering 'third_party/flash-attention'
2025-03-14T04:57:10.1259175Z http.https://github.com/.extraheader
2025-03-14T04:57:10.1297771Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-03-14T04:57:10.1327582Z http.https://github.com/.extraheader
2025-03-14T04:57:10.1373079Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-03-14T04:57:10.1411504Z http.https://github.com/.extraheader
2025-03-14T04:57:10.1455667Z Entering 'third_party/flatbuffers'
2025-03-14T04:57:10.1489132Z http.https://github.com/.extraheader
2025-03-14T04:57:10.1526802Z Entering 'third_party/fmt'
2025-03-14T04:57:10.1562354Z http.https://github.com/.extraheader
2025-03-14T04:57:10.1595466Z Entering 'third_party/gemmlowp/gemmlowp'
2025-03-14T04:57:10.1628348Z http.https://github.com/.extraheader
2025-03-14T04:57:10.1669449Z Entering 'third_party/gloo'
2025-03-14T04:57:10.1700930Z http.https://github.com/.extraheader
2025-03-14T04:57:10.1734158Z Entering 'third_party/googletest'
2025-03-14T04:57:10.1765923Z http.https://github.com/.extraheader
2025-03-14T04:57:10.1807206Z Entering 'third_party/ideep'
2025-03-14T04:57:10.1841213Z http.https://github.com/.extraheader
2025-03-14T04:57:10.1873224Z Entering 'third_party/ideep/mkl-dnn'
2025-03-14T04:57:10.1904495Z http.https://github.com/.extraheader
2025-03-14T04:57:10.1951257Z Entering 'third_party/ittapi'
2025-03-14T04:57:10.1988491Z http.https://github.com/.extraheader
2025-03-14T04:57:10.2022834Z Entering 'third_party/kineto'
2025-03-14T04:57:10.2057475Z http.https://github.com/.extraheader
2025-03-14T04:57:10.2089909Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-03-14T04:57:10.2121031Z http.https://github.com/.extraheader
2025-03-14T04:57:10.2155609Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-03-14T04:57:10.2198941Z http.https://github.com/.extraheader
2025-03-14T04:57:10.2231600Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-03-14T04:57:10.2262567Z http.https://github.com/.extraheader
2025-03-14T04:57:10.2299776Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-03-14T04:57:10.2333766Z http.https://github.com/.extraheader
2025-03-14T04:57:10.2367033Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-03-14T04:57:10.2399636Z http.https://github.com/.extraheader
2025-03-14T04:57:10.2438116Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-03-14T04:57:10.2470701Z http.https://github.com/.extraheader
2025-03-14T04:57:10.2510022Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-03-14T04:57:10.2540372Z http.https://github.com/.extraheader
2025-03-14T04:57:10.2583183Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-03-14T04:57:10.2616180Z http.https://github.com/.extraheader
2025-03-14T04:57:10.2654972Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-03-14T04:57:10.2690614Z http.https://github.com/.extraheader
2025-03-14T04:57:10.2728437Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-03-14T04:57:10.2761756Z http.https://github.com/.extraheader
2025-03-14T04:57:10.2799790Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-03-14T04:57:10.2839078Z http.https://github.com/.extraheader
2025-03-14T04:57:10.2875081Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-03-14T04:57:10.2912118Z http.https://github.com/.extraheader
2025-03-14T04:57:10.2948752Z Entering 'third_party/kleidiai'
2025-03-14T04:57:10.2982255Z http.https://github.com/.extraheader
2025-03-14T04:57:10.3016521Z Entering 'third_party/mimalloc'
2025-03-14T04:57:10.3049808Z http.https://github.com/.extraheader
2025-03-14T04:57:10.3084587Z Entering 'third_party/nlohmann'
2025-03-14T04:57:10.3121510Z http.https://github.com/.extraheader
2025-03-14T04:57:10.3157490Z Entering 'third_party/onnx'
2025-03-14T04:57:10.3196791Z http.https://github.com/.extraheader
2025-03-14T04:57:10.3243279Z Entering 'third_party/onnx/third_party/pybind11'
2025-03-14T04:57:10.3279306Z http.https://github.com/.extraheader
2025-03-14T04:57:10.3311896Z Entering 'third_party/opentelemetry-cpp'
2025-03-14T04:57:10.3345628Z http.https://github.com/.extraheader
2025-03-14T04:57:10.3378774Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-03-14T04:57:10.3409865Z http.https://github.com/.extraheader
2025-03-14T04:57:10.3442160Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-03-14T04:57:10.3477396Z http.https://github.com/.extraheader
2025-03-14T04:57:10.3514310Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-03-14T04:57:10.3545923Z http.https://github.com/.extraheader
2025-03-14T04:57:10.3582819Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-03-14T04:57:10.3619852Z http.https://github.com/.extraheader
2025-03-14T04:57:10.3654428Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-03-14T04:57:10.3689924Z http.https://github.com/.extraheader
2025-03-14T04:57:10.3725414Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-03-14T04:57:10.3756456Z http.https://github.com/.extraheader
2025-03-14T04:57:10.3792623Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-03-14T04:57:10.3825353Z http.https://github.com/.extraheader
2025-03-14T04:57:10.3856162Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-03-14T04:57:10.3886628Z http.https://github.com/.extraheader
2025-03-14T04:57:10.3922604Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-03-14T04:57:10.3953305Z http.https://github.com/.extraheader
2025-03-14T04:57:10.3994361Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-03-14T04:57:10.4027756Z http.https://github.com/.extraheader
2025-03-14T04:57:10.4083573Z Entering 'third_party/pocketfft'
2025-03-14T04:57:10.4120153Z http.https://github.com/.extraheader
2025-03-14T04:57:10.4154171Z Entering 'third_party/protobuf'
2025-03-14T04:57:10.4189715Z http.https://github.com/.extraheader
2025-03-14T04:57:10.4224236Z Entering 'third_party/protobuf/third_party/benchmark'
2025-03-14T04:57:10.4258814Z http.https://github.com/.extraheader
2025-03-14T04:57:10.4294950Z Entering 'third_party/protobuf/third_party/googletest'
2025-03-14T04:57:10.4327512Z http.https://github.com/.extraheader
2025-03-14T04:57:10.4369630Z Entering 'third_party/psimd'
2025-03-14T04:57:10.4409275Z http.https://github.com/.extraheader
2025-03-14T04:57:10.4444299Z Entering 'third_party/pthreadpool'
2025-03-14T04:57:10.4481436Z http.https://github.com/.extraheader
2025-03-14T04:57:10.4512585Z Entering 'third_party/pybind11'
2025-03-14T04:57:10.4547752Z http.https://github.com/.extraheader
2025-03-14T04:57:10.4585046Z Entering 'third_party/python-peachpy'
2025-03-14T04:57:10.4620951Z http.https://github.com/.extraheader
2025-03-14T04:57:10.4656587Z Entering 'third_party/sleef'
2025-03-14T04:57:10.4690936Z http.https://github.com/.extraheader
2025-03-14T04:57:10.4726836Z Entering 'third_party/tensorpipe'
2025-03-14T04:57:10.4762145Z http.https://github.com/.extraheader
2025-03-14T04:57:10.4795298Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-03-14T04:57:10.4827110Z http.https://github.com/.extraheader
2025-03-14T04:57:10.4868322Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-03-14T04:57:10.4901076Z http.https://github.com/.extraheader
2025-03-14T04:57:10.4934344Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-03-14T04:57:10.4970856Z http.https://github.com/.extraheader
2025-03-14T04:57:10.5006169Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-03-14T04:57:10.5039675Z http.https://github.com/.extraheader
2025-03-14T04:57:10.5072727Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-03-14T04:57:10.5104611Z http.https://github.com/.extraheader
2025-03-14T04:57:10.5248446Z A job completed hook has been configured by the self-hosted runner administrator
2025-03-14T04:57:10.5275642Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh'
2025-03-14T04:57:10.5279984Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:57:10.5280443Z ##[endgroup]
2025-03-14T04:57:10.5358531Z [!ALERT!] Swap in detected! [!ALERT!]
2025-03-14T04:57:20.6387163Z [!ALERT!] Swap out detected [!ALERT!]
2025-03-14T04:57:36.4900795Z Cleaning up orphan processes