2025-04-21T21:42:05.6111042Z Current runner version: '2.323.0' 2025-04-21T21:42:05.6116857Z Runner name: 'i-09a65d907c0d198ba' 2025-04-21T21:42:05.6117816Z Machine name: 'ip-10-0-6-215' 2025-04-21T21:42:05.6122228Z ##[group]GITHUB_TOKEN Permissions 2025-04-21T21:42:05.6124615Z Actions: read 2025-04-21T21:42:05.6125157Z Attestations: read 2025-04-21T21:42:05.6125680Z Checks: read 2025-04-21T21:42:05.6126165Z Contents: read 2025-04-21T21:42:05.6126656Z Deployments: read 2025-04-21T21:42:05.6127162Z Discussions: read 2025-04-21T21:42:05.6127662Z Issues: read 2025-04-21T21:42:05.6128132Z Metadata: read 2025-04-21T21:42:05.6128620Z Models: read 2025-04-21T21:42:05.6129100Z Packages: read 2025-04-21T21:42:05.6129579Z Pages: read 2025-04-21T21:42:05.6130060Z PullRequests: read 2025-04-21T21:42:05.6130592Z RepositoryProjects: read 2025-04-21T21:42:05.6131160Z SecurityEvents: read 2025-04-21T21:42:05.6131744Z Statuses: read 2025-04-21T21:42:05.6132241Z ##[endgroup] 2025-04-21T21:42:05.6134128Z Secret source: Actions 2025-04-21T21:42:05.6134879Z Prepare workflow directory 2025-04-21T21:42:06.0937469Z Prepare all required actions 2025-04-21T21:42:06.0979732Z Getting action download info 2025-04-21T21:42:06.3272401Z Download action repository 'pytorch/test-infra@main' (SHA:d0e960f2e16f063377744672abd0cabc2c9a0167) 2025-04-21T21:42:07.8701338Z Download action repository 'pytorch/pytorch@main' (SHA:c312d8c5018f6c742fa9df7223fee74c71c4a68b) 2025-04-21T21:42:22.3708076Z Download action repository 'aws-actions/configure-aws-credentials@v4' (SHA:e3dd6a429d7300a6a4c196c26e071d42e0343502) 2025-04-21T21:42:22.5911627Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2025-04-21T21:42:22.8855891Z Getting action download info 2025-04-21T21:42:23.0038099Z Download action repository 'actions/checkout@v4' (SHA:11bd71901bbe5b1630ceea73d27597364c9af683) 2025-04-21T21:42:23.2738773Z Getting action download info 2025-04-21T21:42:23.3980629Z Download action repository 'nick-fields/retry@v3.0.0' (SHA:7152eba30c6575329ac0576536151aca5a72780e) 2025-04-21T21:42:23.5719926Z Getting action download info 2025-04-21T21:42:23.6609711Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2025-04-21T21:42:23.8387895Z Getting action download info 2025-04-21T21:42:23.9718123Z Complete job name: cuda12.6-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2025-04-21T21:42:24.0249640Z A job started hook has been configured by the self-hosted runner administrator 2025-04-21T21:42:24.0346707Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2025-04-21T21:42:24.0357613Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:42:24.0358194Z ##[endgroup] 2025-04-21T21:42:25.8570129Z Runner Type: linux.g5.4xlarge.nvidia.gpu 2025-04-21T21:42:25.8570621Z Instance Type: g5.4xlarge 2025-04-21T21:42:25.8570875Z AMI Name: unknown 2025-04-21T21:42:25.8607677Z AMI ID: ami-071226ecf16aa7d96 2025-04-21T21:42:31.2518982Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2025-04-21T21:42:31.2519375Z with: 2025-04-21T21:42:31.2519890Z github-secret: *** 2025-04-21T21:42:31.2520541Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2025-04-21T21:42:31.2521224Z activate-with-label: false 2025-04-21T21:42:31.2521483Z label: with-ssh 2025-04-21T21:42:31.2521717Z remove-existing-keys: true 2025-04-21T21:42:31.2521982Z fail-silently: true 2025-04-21T21:42:31.2522235Z env: 2025-04-21T21:42:31.2522442Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:42:31.2522682Z ##[endgroup] 2025-04-21T21:42:31.3724955Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2025-04-21T21:42:31.3726400Z ciflow reference detected, attempting to extract PR number 2025-04-21T21:42:31.7588649Z Grabbing public ssh keys from https://github.com/pytorch-bot[bot].keys 2025-04-21T21:42:31.8175243Z No SSH keys found for user pytorch-bot[bot] 2025-04-21T21:42:31.8175739Z Grabbing public ssh keys from https://github.com/laithsakka.keys 2025-04-21T21:42:31.8768978Z ~/.ssh/authorized_keys file found on node, removing ~/.ssh and starting fresh 2025-04-21T21:42:31.8784266Z Public keys pulled and installed to /home/ec2-user/.ssh/authorized_keys 2025-04-21T21:42:31.8818730Z Login using: ssh ec2-user@ec2-34-204-78-42.compute-1.amazonaws.com 2025-04-21T21:42:31.8819270Z All testing is done inside the container, to start an interactive session run: 2025-04-21T21:42:31.8819777Z docker exec -it $(docker container ps --format '{{.ID}}') bash 2025-04-21T21:42:31.8988601Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2025-04-21T21:42:31.8989014Z with: 2025-04-21T21:42:31.8989227Z no-sudo: true 2025-04-21T21:42:31.8989454Z submodules: recursive 2025-04-21T21:42:31.8989716Z fetch-depth: 0 2025-04-21T21:42:31.8989944Z env: 2025-04-21T21:42:31.8990150Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:42:31.8990416Z ##[endgroup] 2025-04-21T21:42:31.9072044Z ##[group]Run echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-04-21T21:42:31.9073048Z echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-04-21T21:42:31.9084512Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:42:31.9084870Z env: 2025-04-21T21:42:31.9085086Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:42:31.9085341Z ##[endgroup] 2025-04-21T21:42:31.9187200Z ##[group]Run # Use all available CPUs for fetching 2025-04-21T21:42:31.9187622Z # Use all available CPUs for fetching 2025-04-21T21:42:31.9187950Z cd "${GITHUB_WORKSPACE}" 2025-04-21T21:42:31.9188277Z git config --global fetch.parallel 0 2025-04-21T21:42:31.9188650Z git config --global submodule.fetchJobs 0 2025-04-21T21:42:31.9188973Z  2025-04-21T21:42:31.9189319Z # Clean workspace. The default checkout action should also do this, but 2025-04-21T21:42:31.9189781Z # do it here as well just in case 2025-04-21T21:42:31.9190127Z if [[ -d .git ]]; then 2025-04-21T21:42:31.9190407Z  if [ -z "${NO_SUDO}" ]; then 2025-04-21T21:42:31.9190703Z  sudo git clean -ffdx 2025-04-21T21:42:31.9190978Z  else 2025-04-21T21:42:31.9191286Z  git clean -ffdx 2025-04-21T21:42:31.9191548Z  fi 2025-04-21T21:42:31.9191755Z fi 2025-04-21T21:42:31.9200653Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:42:31.9201002Z env: 2025-04-21T21:42:31.9201216Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:42:31.9201470Z NO_SUDO: true 2025-04-21T21:42:31.9201685Z ##[endgroup] 2025-04-21T21:42:32.2141820Z Removing .additional_ci_files/ 2025-04-21T21:42:32.2142476Z Removing .pytest_cache/ 2025-04-21T21:42:32.2142965Z Removing build/ 2025-04-21T21:42:32.2143408Z Removing dist/ 2025-04-21T21:42:32.2144165Z Removing logs-test-inductor-2-2-linux.g5.4xlarge.nvidia.gpu_40895106338.zip 2025-04-21T21:42:32.2145694Z Removing test-jsons-test-inductor-2-2-linux.g5.4xlarge.nvidia.gpu_40895106338.zip 2025-04-21T21:42:32.2147019Z Removing test-reports-test-inductor-2-2-linux.g5.4xlarge.nvidia.gpu_40895106338.zip 2025-04-21T21:42:32.2147596Z Removing test/.pytorch-disabled-tests.json 2025-04-21T21:42:32.2147919Z Removing test/__pycache__/ 2025-04-21T21:42:32.2148192Z Removing test/inductor/__pycache__/ 2025-04-21T21:42:32.2148479Z Removing test/test-reports/ 2025-04-21T21:42:32.2148763Z Removing test/torch_compile_debug/ 2025-04-21T21:42:32.2149050Z Removing tools/__pycache__/ 2025-04-21T21:42:32.2149330Z Removing tools/stats/__pycache__/ 2025-04-21T21:42:32.2149675Z Removing tools/stats/upload_utilization_stats/__pycache__/ 2025-04-21T21:42:32.2150053Z Removing tools/testing/__pycache__/ 2025-04-21T21:42:32.2150406Z Removing tools/testing/target_determination/__pycache__/ 2025-04-21T21:42:32.2151440Z Removing tools/testing/target_determination/heuristics/__pycache__/ 2025-04-21T21:42:32.2151834Z Removing usage_log.txt 2025-04-21T21:42:32.2220598Z ##[group]Run actions/checkout@v4 2025-04-21T21:42:32.2220870Z with: 2025-04-21T21:42:32.2221101Z ref: b68c0effe97351b0c24b2f7aef671559be6aa977 2025-04-21T21:42:32.2221416Z fetch-depth: 0 2025-04-21T21:42:32.2221647Z submodules: recursive 2025-04-21T21:42:32.2221885Z show-progress: false 2025-04-21T21:42:32.2222141Z repository: pytorch/pytorch 2025-04-21T21:42:32.2222536Z token: *** 2025-04-21T21:42:32.2222752Z ssh-strict: true 2025-04-21T21:42:32.2223191Z ssh-user: git 2025-04-21T21:42:32.2223433Z persist-credentials: true 2025-04-21T21:42:32.2223689Z clean: true 2025-04-21T21:42:32.2223930Z sparse-checkout-cone-mode: true 2025-04-21T21:42:32.2224213Z fetch-tags: false 2025-04-21T21:42:32.2224442Z lfs: false 2025-04-21T21:42:32.2224657Z set-safe-directory: true 2025-04-21T21:42:32.2224910Z env: 2025-04-21T21:42:32.2225128Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:42:32.2225510Z ##[endgroup] 2025-04-21T21:42:32.3359750Z Syncing repository: pytorch/pytorch 2025-04-21T21:42:32.3360932Z ##[group]Getting Git version info 2025-04-21T21:42:32.3361370Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2025-04-21T21:42:32.3361970Z [command]/usr/bin/git version 2025-04-21T21:42:32.3362236Z git version 2.47.1 2025-04-21T21:42:32.3379277Z ##[endgroup] 2025-04-21T21:42:32.3389844Z Copying '/home/ec2-user/.gitconfig' to '/home/ec2-user/actions-runner/_work/_temp/a82249ce-b6ba-4dca-a571-918af3664e86/.gitconfig' 2025-04-21T21:42:32.3411641Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/a82249ce-b6ba-4dca-a571-918af3664e86' before making global git config changes 2025-04-21T21:42:32.3412682Z Adding repository directory to the temporary git global config as a safe directory 2025-04-21T21:42:32.3416501Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2025-04-21T21:42:32.3464360Z [command]/usr/bin/git config --local --get remote.origin.url 2025-04-21T21:42:32.3497227Z https://github.com/pytorch/pytorch 2025-04-21T21:42:32.3515962Z ##[group]Removing previously created refs, to avoid conflicts 2025-04-21T21:42:32.3519714Z [command]/usr/bin/git rev-parse --symbolic-full-name --verify --quiet HEAD 2025-04-21T21:42:32.3545953Z HEAD 2025-04-21T21:42:32.3589175Z ##[endgroup] 2025-04-21T21:42:32.3592297Z [command]/usr/bin/git submodule status 2025-04-21T21:42:32.3997346Z 7e1e1fe3858c63c251c637ae41a20de425dde96f android/libs/fbjni (v0.1.0-12-g7e1e1fe) 2025-04-21T21:42:32.4095071Z 4dfe081cf6bcd15db339cf2680b9281b8451eeb3 third_party/FP16 (4dfe081) 2025-04-21T21:42:32.4193835Z b408327ac2a15ec3e43352421954f5b1967701d1 third_party/FXdiv (b408327) 2025-04-21T21:42:32.4305012Z c07e3a0400713d546e0dea2d5466dd22ea389c73 third_party/NNPACK (c07e3a0) 2025-04-21T21:42:32.4333621Z e170594ac7cf1dac584da473d4ca9301087090c1 third_party/NVTX (v3.1.0) 2025-04-21T21:42:32.4424774Z a6bfc237255a6bac1513f7c1ebde6d8aed6b5191 third_party/VulkanMemoryAllocator (v2.1.0-705-ga6bfc23) 2025-04-21T21:42:32.5027650Z 51a0103656eff6fc9bfd39a4597923c4b542c883 third_party/XNNPACK (remotes/origin/ds/ndk-1243-g51a010365) 2025-04-21T21:42:32.5058321Z 0d98dba29d66e93259db7daa53a9327df767a415 third_party/benchmark (v1.6.1) 2025-04-21T21:42:32.5094169Z 8086bbe3a78d931eb96fe12fdc014082e18d18d3 third_party/composable_kernel (rocm-6.4.0-6-g8086bbe3a) 2025-04-21T21:42:32.5245319Z 3b6597bba913d51161383657829b7e644e59c006 third_party/cpp-httplib (v0.15.3-20-g3b6597b) 2025-04-21T21:42:32.5388101Z 5e3d2445e6a84d9599bee2bf78edbb4d80865e1d third_party/cpuinfo (5e3d244) 2025-04-21T21:42:32.5433821Z 8801fd7b31c2f798732ed1e1dd4711a9cde4217d third_party/cudnn_frontend (v0.5-46-g8801fd7) 2025-04-21T21:42:32.5538373Z afa1772203677c5118fcd82537a9c8fefbcc7008 third_party/cutlass (v3.8.0) 2025-04-21T21:42:32.6315924Z 3147391d946bb4b6c68edd901f2add6ac1f31f8c third_party/eigen (3.4.0) 2025-04-21T21:42:32.6702691Z dbc3157bf256f1339b3fa1fef2be89ac4078be0e third_party/fbgemm (v0.4.1-446-gdbc3157b) 2025-04-21T21:42:32.6803734Z 979702c87a8713a8e0a5e9fee122b90d2ef13be5 third_party/flash-attention (v2.7.4) 2025-04-21T21:42:32.6838448Z 01834de25e4bf3975a9a00e816292b1ad0fe184b third_party/flatbuffers (v23.3.3) 2025-04-21T21:42:32.7316356Z 123913715afeb8a437e6388b4473fcc4753e1c9a third_party/fmt (11.1.4) 2025-04-21T21:42:32.7456621Z 3fb5c176c17c765a3492cd2f0321b0dab712f350 third_party/gemmlowp/gemmlowp (remotes/origin/revert-87-master-135-g3fb5c17) 2025-04-21T21:42:32.7555374Z c61070427610ccd923efe3e7f8b3eca12bbcc31a third_party/gloo (heads/main) 2025-04-21T21:42:32.7849099Z b514bdc898e2951020cbdca1304b75f5950d1f59 third_party/googletest (release-1.8.0-3484-gb514bdc8) 2025-04-21T21:42:32.7951847Z 719d8e6cd7f7a0e01b155657526d693acf97c2b3 third_party/ideep (pytorch-rls-v3.7.1) 2025-04-21T21:42:32.8026810Z 5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42 third_party/ittapi (v3.23.0-14-g5b8a7d7) 2025-04-21T21:42:32.8322065Z 5fa4bd8c8fb581a621f72957d0cb11431cc4714f third_party/kineto (remotes/origin/sraikund/test-64-g5fa4bd8) 2025-04-21T21:42:32.8353729Z ef685a13cfbe8d418aa2ed34350e21e4938358b6 third_party/kleidiai (v1.3.0) 2025-04-21T21:42:32.8385315Z b66e3214d8a104669c2ec05ae91ebc26a8f5ab78 third_party/mimalloc (v1.8.2) 2025-04-21T21:42:32.8921247Z 87cda1d6646592ac5866dc703c8e1839046a6806 third_party/nlohmann (v3.10.1-113-g87cda1d6) 2025-04-21T21:42:32.9260309Z b8baa8446686496da4cc8fda09f2b6fe65c2a02c third_party/onnx (v1.17.0) 2025-04-21T21:42:32.9290447Z a799f4aed9c94b765dcdaabaeab7d5e7e2310878 third_party/opentelemetry-cpp (v1.14.2) 2025-04-21T21:42:32.9320258Z 9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe third_party/pocketfft (release_for_eigen-11-g9d3ab05) 2025-04-21T21:42:32.9771480Z d1eca4e4b421cd2997495c4b4e65cea6be4e9b8a third_party/protobuf (v3.7.0-rc.2-1279-gd1eca4e4b) 2025-04-21T21:42:32.9867472Z 072586a71b55b7f8c584153d223e95687148a900 third_party/psimd (heads/master) 2025-04-21T21:42:32.9943711Z 4fe0e1e183925bf8cfa6aae24237e724a96479b8 third_party/pthreadpool (0.1-144-g4fe0e1e) 2025-04-21T21:42:33.0003310Z a2e59f0e7065404b44dfe92a28aca47ba1378dc4 third_party/pybind11 (v2.11.0-182-ga2e59f0e) 2025-04-21T21:42:33.0093619Z f45429b087dd7d5bc78bb40dc7cf06425c252d67 third_party/python-peachpy (remotes/origin/pre-generated) 2025-04-21T21:42:33.0223332Z 56e1f79cb140fb9326d612d0be06b5250565cade third_party/sleef (3.7-33-g56e1f79) 2025-04-21T21:42:33.0316328Z 52791a2fd214b2a9dc5759d36725909c1daa7f2e third_party/tensorpipe (remotes/origin/master) 2025-04-21T21:42:33.0332604Z ##[group]Cleaning the repository 2025-04-21T21:42:33.0336689Z [command]/usr/bin/git clean -ffdx 2025-04-21T21:42:33.0570774Z [command]/usr/bin/git reset --hard HEAD 2025-04-21T21:42:34.1160674Z Updating files: 60% (10031/16510) 2025-04-21T21:42:34.1248574Z Updating files: 61% (10072/16510) 2025-04-21T21:42:34.1336127Z Updating files: 62% (10237/16510) 2025-04-21T21:42:34.1420939Z Updating files: 63% (10402/16510) 2025-04-21T21:42:34.1507268Z Updating files: 64% (10567/16510) 2025-04-21T21:42:34.1604346Z Updating files: 65% (10732/16510) 2025-04-21T21:42:34.1897844Z Updating files: 66% (10897/16510) 2025-04-21T21:42:34.2169991Z Updating files: 67% (11062/16510) 2025-04-21T21:42:34.2255520Z Updating files: 68% (11227/16510) 2025-04-21T21:42:34.2343786Z Updating files: 69% (11392/16510) 2025-04-21T21:42:34.2544799Z Updating files: 70% (11557/16510) 2025-04-21T21:42:34.2840769Z Updating files: 71% (11723/16510) 2025-04-21T21:42:34.3131118Z Updating files: 72% (11888/16510) 2025-04-21T21:42:34.3540398Z Updating files: 73% (12053/16510) 2025-04-21T21:42:34.3734648Z Updating files: 74% (12218/16510) 2025-04-21T21:42:34.3885325Z Updating files: 75% (12383/16510) 2025-04-21T21:42:34.4172540Z Updating files: 76% (12548/16510) 2025-04-21T21:42:34.4374234Z Updating files: 77% (12713/16510) 2025-04-21T21:42:34.4704005Z Updating files: 78% (12878/16510) 2025-04-21T21:42:34.4991478Z Updating files: 79% (13043/16510) 2025-04-21T21:42:34.5240967Z Updating files: 80% (13208/16510) 2025-04-21T21:42:34.5426945Z Updating files: 81% (13374/16510) 2025-04-21T21:42:34.5591924Z Updating files: 82% (13539/16510) 2025-04-21T21:42:34.5781498Z Updating files: 83% (13704/16510) 2025-04-21T21:42:34.5979170Z Updating files: 84% (13869/16510) 2025-04-21T21:42:34.6163619Z Updating files: 85% (14034/16510) 2025-04-21T21:42:34.6345012Z Updating files: 86% (14199/16510) 2025-04-21T21:42:34.6507811Z Updating files: 87% (14364/16510) 2025-04-21T21:42:34.6689660Z Updating files: 88% (14529/16510) 2025-04-21T21:42:34.6909705Z Updating files: 89% (14694/16510) 2025-04-21T21:42:34.7101634Z Updating files: 90% (14859/16510) 2025-04-21T21:42:34.7265641Z Updating files: 91% (15025/16510) 2025-04-21T21:42:34.7449277Z Updating files: 92% (15190/16510) 2025-04-21T21:42:34.7696260Z Updating files: 93% (15355/16510) 2025-04-21T21:42:34.7945689Z Updating files: 94% (15520/16510) 2025-04-21T21:42:34.8163192Z Updating files: 95% (15685/16510) 2025-04-21T21:42:34.8370582Z Updating files: 96% (15850/16510) 2025-04-21T21:42:34.8711605Z Updating files: 97% (16015/16510) 2025-04-21T21:42:34.8914033Z Updating files: 98% (16180/16510) 2025-04-21T21:42:34.9116137Z Updating files: 99% (16345/16510) 2025-04-21T21:42:34.9116675Z Updating files: 100% (16510/16510) 2025-04-21T21:42:34.9117227Z Updating files: 100% (16510/16510), done. 2025-04-21T21:42:34.9300870Z HEAD is now at f37e138bc4f [MPS] Enable log1p and sigmoid for int64 (#151791) 2025-04-21T21:42:34.9462860Z ##[endgroup] 2025-04-21T21:42:34.9464846Z ##[group]Disabling automatic garbage collection 2025-04-21T21:42:34.9470564Z [command]/usr/bin/git config --local gc.auto 0 2025-04-21T21:42:34.9506441Z ##[endgroup] 2025-04-21T21:42:34.9506992Z ##[group]Setting up auth 2025-04-21T21:42:34.9512142Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2025-04-21T21:42:34.9546197Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2025-04-21T21:42:34.9909475Z Entering 'android/libs/fbjni' 2025-04-21T21:42:34.9980607Z Entering 'third_party/FP16' 2025-04-21T21:42:35.0050220Z Entering 'third_party/FXdiv' 2025-04-21T21:42:35.0121323Z Entering 'third_party/NNPACK' 2025-04-21T21:42:35.0190554Z Entering 'third_party/NVTX' 2025-04-21T21:42:35.0260803Z Entering 'third_party/VulkanMemoryAllocator' 2025-04-21T21:42:35.0329308Z Entering 'third_party/XNNPACK' 2025-04-21T21:42:35.0417383Z Entering 'third_party/benchmark' 2025-04-21T21:42:35.0485181Z Entering 'third_party/composable_kernel' 2025-04-21T21:42:35.0560549Z Entering 'third_party/cpp-httplib' 2025-04-21T21:42:35.0630318Z Entering 'third_party/cpuinfo' 2025-04-21T21:42:35.0696586Z Entering 'third_party/cudnn_frontend' 2025-04-21T21:42:35.0765484Z Entering 'third_party/cutlass' 2025-04-21T21:42:35.0845193Z Entering 'third_party/eigen' 2025-04-21T21:42:35.0918434Z Entering 'third_party/fbgemm' 2025-04-21T21:42:35.0989435Z Entering 'third_party/fbgemm/third_party/asmjit' 2025-04-21T21:42:35.1060311Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2025-04-21T21:42:35.1127783Z Entering 'third_party/fbgemm/third_party/cutlass' 2025-04-21T21:42:35.1201901Z Entering 'third_party/fbgemm/third_party/googletest' 2025-04-21T21:42:35.1268924Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2025-04-21T21:42:35.1339246Z Entering 'third_party/flash-attention' 2025-04-21T21:42:35.1414811Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-04-21T21:42:35.1488486Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-04-21T21:42:35.1566102Z Entering 'third_party/flatbuffers' 2025-04-21T21:42:35.1637629Z Entering 'third_party/fmt' 2025-04-21T21:42:35.1707494Z Entering 'third_party/gemmlowp/gemmlowp' 2025-04-21T21:42:35.1781180Z Entering 'third_party/gloo' 2025-04-21T21:42:35.1852366Z Entering 'third_party/googletest' 2025-04-21T21:42:35.1921664Z Entering 'third_party/ideep' 2025-04-21T21:42:35.1989736Z Entering 'third_party/ideep/mkl-dnn' 2025-04-21T21:42:35.2070385Z Entering 'third_party/ittapi' 2025-04-21T21:42:35.2140229Z Entering 'third_party/kineto' 2025-04-21T21:42:35.2213239Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-04-21T21:42:35.2279342Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-04-21T21:42:35.2350043Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-04-21T21:42:35.2418683Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-04-21T21:42:35.2487447Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-04-21T21:42:35.2553872Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-04-21T21:42:35.2625727Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-04-21T21:42:35.2693975Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-04-21T21:42:35.2761968Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-04-21T21:42:35.2831641Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-04-21T21:42:35.2903542Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-04-21T21:42:35.2970544Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-04-21T21:42:35.3041899Z Entering 'third_party/kleidiai' 2025-04-21T21:42:35.3114164Z Entering 'third_party/mimalloc' 2025-04-21T21:42:35.3184305Z Entering 'third_party/nlohmann' 2025-04-21T21:42:35.3253534Z Entering 'third_party/onnx' 2025-04-21T21:42:35.3338755Z Entering 'third_party/onnx/third_party/pybind11' 2025-04-21T21:42:35.3412504Z Entering 'third_party/opentelemetry-cpp' 2025-04-21T21:42:35.3481650Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-04-21T21:42:35.3549408Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-04-21T21:42:35.3618274Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-04-21T21:42:35.3685125Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-04-21T21:42:35.3753340Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-04-21T21:42:35.3821459Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-04-21T21:42:35.3888309Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-04-21T21:42:35.3953047Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-04-21T21:42:35.4022094Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-04-21T21:42:35.4092580Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-04-21T21:42:35.4181833Z Entering 'third_party/pocketfft' 2025-04-21T21:42:35.4249278Z Entering 'third_party/protobuf' 2025-04-21T21:42:35.4324979Z Entering 'third_party/protobuf/third_party/benchmark' 2025-04-21T21:42:35.4393863Z Entering 'third_party/protobuf/third_party/googletest' 2025-04-21T21:42:35.4465440Z Entering 'third_party/psimd' 2025-04-21T21:42:35.4533738Z Entering 'third_party/pthreadpool' 2025-04-21T21:42:35.4603228Z Entering 'third_party/pybind11' 2025-04-21T21:42:35.4670597Z Entering 'third_party/python-peachpy' 2025-04-21T21:42:35.4738278Z Entering 'third_party/sleef' 2025-04-21T21:42:35.4811185Z Entering 'third_party/tensorpipe' 2025-04-21T21:42:35.4877699Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-04-21T21:42:35.4944997Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-04-21T21:42:35.5010950Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-04-21T21:42:35.5077546Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-04-21T21:42:35.5142155Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-04-21T21:42:35.5237472Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2025-04-21T21:42:35.5270291Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2025-04-21T21:42:35.5628457Z Entering 'android/libs/fbjni' 2025-04-21T21:42:35.5698717Z Entering 'third_party/FP16' 2025-04-21T21:42:35.5768198Z Entering 'third_party/FXdiv' 2025-04-21T21:42:35.5838059Z Entering 'third_party/NNPACK' 2025-04-21T21:42:35.5909060Z Entering 'third_party/NVTX' 2025-04-21T21:42:35.5979429Z Entering 'third_party/VulkanMemoryAllocator' 2025-04-21T21:42:35.6048750Z Entering 'third_party/XNNPACK' 2025-04-21T21:42:35.6134849Z Entering 'third_party/benchmark' 2025-04-21T21:42:35.6204229Z Entering 'third_party/composable_kernel' 2025-04-21T21:42:35.6278666Z Entering 'third_party/cpp-httplib' 2025-04-21T21:42:35.6347136Z Entering 'third_party/cpuinfo' 2025-04-21T21:42:35.6417372Z Entering 'third_party/cudnn_frontend' 2025-04-21T21:42:35.6487459Z Entering 'third_party/cutlass' 2025-04-21T21:42:35.6564667Z Entering 'third_party/eigen' 2025-04-21T21:42:35.6638035Z Entering 'third_party/fbgemm' 2025-04-21T21:42:35.6706972Z Entering 'third_party/fbgemm/third_party/asmjit' 2025-04-21T21:42:35.6776367Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2025-04-21T21:42:35.6845022Z Entering 'third_party/fbgemm/third_party/cutlass' 2025-04-21T21:42:35.6920339Z Entering 'third_party/fbgemm/third_party/googletest' 2025-04-21T21:42:35.6988191Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2025-04-21T21:42:35.7058151Z Entering 'third_party/flash-attention' 2025-04-21T21:42:35.7127923Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-04-21T21:42:35.7203043Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-04-21T21:42:35.7280481Z Entering 'third_party/flatbuffers' 2025-04-21T21:42:35.7353282Z Entering 'third_party/fmt' 2025-04-21T21:42:35.7423157Z Entering 'third_party/gemmlowp/gemmlowp' 2025-04-21T21:42:35.7494761Z Entering 'third_party/gloo' 2025-04-21T21:42:35.7564499Z Entering 'third_party/googletest' 2025-04-21T21:42:35.7634881Z Entering 'third_party/ideep' 2025-04-21T21:42:35.7702976Z Entering 'third_party/ideep/mkl-dnn' 2025-04-21T21:42:35.7779020Z Entering 'third_party/ittapi' 2025-04-21T21:42:35.7849068Z Entering 'third_party/kineto' 2025-04-21T21:42:35.7918233Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-04-21T21:42:35.7985706Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-04-21T21:42:35.8054594Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-04-21T21:42:35.8122102Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-04-21T21:42:35.8189200Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-04-21T21:42:35.8253891Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-04-21T21:42:35.8325873Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-04-21T21:42:35.8395511Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-04-21T21:42:35.8463601Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-04-21T21:42:35.8533841Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-04-21T21:42:35.8605682Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-04-21T21:42:35.8672115Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-04-21T21:42:35.8742856Z Entering 'third_party/kleidiai' 2025-04-21T21:42:35.8814163Z Entering 'third_party/mimalloc' 2025-04-21T21:42:35.8883551Z Entering 'third_party/nlohmann' 2025-04-21T21:42:35.8952831Z Entering 'third_party/onnx' 2025-04-21T21:42:35.9036309Z Entering 'third_party/onnx/third_party/pybind11' 2025-04-21T21:42:35.9110602Z Entering 'third_party/opentelemetry-cpp' 2025-04-21T21:42:35.9179395Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-04-21T21:42:35.9244632Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-04-21T21:42:35.9310818Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-04-21T21:42:35.9376839Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-04-21T21:42:35.9443922Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-04-21T21:42:35.9510009Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-04-21T21:42:35.9575923Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-04-21T21:42:35.9639912Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-04-21T21:42:35.9710262Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-04-21T21:42:35.9785978Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-04-21T21:42:35.9873676Z Entering 'third_party/pocketfft' 2025-04-21T21:42:35.9941887Z Entering 'third_party/protobuf' 2025-04-21T21:42:36.0012836Z Entering 'third_party/protobuf/third_party/benchmark' 2025-04-21T21:42:36.0080758Z Entering 'third_party/protobuf/third_party/googletest' 2025-04-21T21:42:36.0156490Z Entering 'third_party/psimd' 2025-04-21T21:42:36.0227172Z Entering 'third_party/pthreadpool' 2025-04-21T21:42:36.0295362Z Entering 'third_party/pybind11' 2025-04-21T21:42:36.0363406Z Entering 'third_party/python-peachpy' 2025-04-21T21:42:36.0434798Z Entering 'third_party/sleef' 2025-04-21T21:42:36.0505157Z Entering 'third_party/tensorpipe' 2025-04-21T21:42:36.0575246Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-04-21T21:42:36.0642050Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-04-21T21:42:36.0708349Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-04-21T21:42:36.0774445Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-04-21T21:42:36.0840099Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-04-21T21:42:36.0937923Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2025-04-21T21:42:36.0992664Z ##[endgroup] 2025-04-21T21:42:36.0993314Z ##[group]Fetching the repository 2025-04-21T21:42:36.1000702Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --no-recurse-submodules origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2025-04-21T21:42:36.4382308Z From https://github.com/pytorch/pytorch 2025-04-21T21:42:36.4382936Z - [deleted] (none) -> ciflow/inductor/148622 2025-04-21T21:42:36.5095081Z - [deleted] (none) -> ciflow/inductor/149690 2025-04-21T21:42:36.5095853Z - [deleted] (none) -> ciflow/inductor/149711 2025-04-21T21:42:36.5096582Z - [deleted] (none) -> ciflow/inductor/151506 2025-04-21T21:42:36.5097750Z - [deleted] (none) -> ciflow/trunk/148622 2025-04-21T21:42:36.5099880Z - [deleted] (none) -> ciflow/trunk/149711 2025-04-21T21:42:36.5101334Z - [deleted] (none) -> ciflow/trunk/151046 2025-04-21T21:42:36.5103209Z - [deleted] (none) -> ciflow/trunk/151672 2025-04-21T21:42:36.5104969Z - [deleted] (none) -> ciflow/trunk/151704 2025-04-21T21:42:36.5124605Z - [deleted] (none) -> ciflow/trunk/151716 2025-04-21T21:42:36.5125362Z - [deleted] (none) -> ciflow/trunk/151794 2025-04-21T21:42:37.6320970Z ed1e3930b3f..a840b9b1ba8 bf/fa-small-block-mask -> origin/bf/fa-small-block-mask 2025-04-21T21:42:37.6325913Z * [new branch] eqy-patch-6 -> origin/eqy-patch-6 2025-04-21T21:42:37.6359325Z b95d854985a..3e5aca84441 gh/laithsakka/119/base -> origin/gh/laithsakka/119/base 2025-04-21T21:42:37.6366804Z 2e26d078915..b68c0effe97 gh/laithsakka/119/head -> origin/gh/laithsakka/119/head 2025-04-21T21:42:37.6368182Z + 19dc41198c9...55d0c158068 gh/laithsakka/119/orig -> origin/gh/laithsakka/119/orig (forced update) 2025-04-21T21:42:37.6377139Z eaba5732110..a9a371e2cbb gh/laithsakka/143/base -> origin/gh/laithsakka/143/base 2025-04-21T21:42:37.6383985Z 12685772308..e408d0201d5 gh/laithsakka/143/head -> origin/gh/laithsakka/143/head 2025-04-21T21:42:37.6391547Z + 5a4cbcd8a74...e3fa1be9109 gh/laithsakka/143/orig -> origin/gh/laithsakka/143/orig (forced update) 2025-04-21T21:42:37.6416266Z bbd593720e2..0de0a192d66 gh/laithsakka/144/base -> origin/gh/laithsakka/144/base 2025-04-21T21:42:37.6423383Z 8b8f4016ba3..90c985a64c3 gh/laithsakka/144/head -> origin/gh/laithsakka/144/head 2025-04-21T21:42:37.6431289Z + 5cae6c1dc5e...c76ceea1d31 gh/laithsakka/144/orig -> origin/gh/laithsakka/144/orig (forced update) 2025-04-21T21:42:37.6438257Z 8b8f4016ba3..91110def275 gh/laithsakka/145/base -> origin/gh/laithsakka/145/base 2025-04-21T21:42:37.6445871Z 61c653a8a41..79bd94ff19f gh/laithsakka/145/head -> origin/gh/laithsakka/145/head 2025-04-21T21:42:37.6453278Z + 1c79f4f257d...cf312ce1ac8 gh/laithsakka/145/orig -> origin/gh/laithsakka/145/orig (forced update) 2025-04-21T21:42:37.6466801Z 5c603ac1ba8..f71a3d90cd1 gh/mlazos/29/base -> origin/gh/mlazos/29/base 2025-04-21T21:42:37.6474060Z d5699eb0ca4..ba361774d76 gh/mlazos/29/head -> origin/gh/mlazos/29/head 2025-04-21T21:42:37.6481442Z + ceacf9edd65...7463b297ab7 gh/mlazos/29/orig -> origin/gh/mlazos/29/orig (forced update) 2025-04-21T21:42:37.6488554Z fea3e205db8..f0051bf4cf3 gh/mlazos/30/base -> origin/gh/mlazos/30/base 2025-04-21T21:42:37.6495849Z b367c377e05..36d13fd7746 gh/mlazos/30/head -> origin/gh/mlazos/30/head 2025-04-21T21:42:37.6502608Z + 16f0eb41fb0...386d8af2dab gh/mlazos/30/orig -> origin/gh/mlazos/30/orig (forced update) 2025-04-21T21:42:37.6509958Z 4a9a383ded5..7423b341c9f gh/mlazos/31/base -> origin/gh/mlazos/31/base 2025-04-21T21:42:37.6516708Z 9dd543831cf..c2cf8791922 gh/mlazos/31/head -> origin/gh/mlazos/31/head 2025-04-21T21:42:37.6523515Z + 0ca0b7d756f...71abc0a02df gh/mlazos/31/orig -> origin/gh/mlazos/31/orig (forced update) 2025-04-21T21:42:37.6530709Z edd39a0d4ee..4447800f753 gh/syed-ahmed/1/head -> origin/gh/syed-ahmed/1/head 2025-04-21T21:42:37.6532046Z + 469a4dc35e1...0f317493c86 gh/syed-ahmed/1/orig -> origin/gh/syed-ahmed/1/orig (forced update) 2025-04-21T21:42:37.6544045Z 4c18280dfe5..0bb3c87007f gh/xmfan/214/base -> origin/gh/xmfan/214/base 2025-04-21T21:42:37.6552036Z f07fd36146a..ad7c2ba8b2d gh/xmfan/214/head -> origin/gh/xmfan/214/head 2025-04-21T21:42:37.6559222Z + 257a8caa97f...f7dd7d4da17 gh/xmfan/214/orig -> origin/gh/xmfan/214/orig (forced update) 2025-04-21T21:42:37.6566979Z 10eb069c930..1aeb85157e3 gh/xmfan/224/base -> origin/gh/xmfan/224/base 2025-04-21T21:42:37.6574517Z c5cad517a32..0c5e455e3eb gh/xmfan/224/head -> origin/gh/xmfan/224/head 2025-04-21T21:42:37.6582111Z + 9cf7d1444e7...29777b78279 gh/xmfan/224/orig -> origin/gh/xmfan/224/orig (forced update) 2025-04-21T21:42:37.6588619Z b7c70007280..c312d8c5018 main -> origin/main 2025-04-21T21:42:37.6597279Z + 0ca0b7d756f...71abc0a02df mlazos/hc-fixes4 -> origin/mlazos/hc-fixes4 (forced update) 2025-04-21T21:42:37.6600674Z * [new branch] pianpwk/01_auto_warning -> origin/pianpwk/01_auto_warning 2025-04-21T21:42:37.6602441Z * [new branch] pianpwk/dynamo_export_ctx -> origin/pianpwk/dynamo_export_ctx 2025-04-21T21:42:37.6604112Z 30cb4fc17d5..cceac144146 pianpwk/oblivious_expand -> origin/pianpwk/oblivious_expand 2025-04-21T21:42:37.6618948Z 10001864160..9d80fe4b89f pianpwk/skt_user_code -> origin/pianpwk/skt_user_code 2025-04-21T21:42:37.6622758Z 2eacdb91c3d..33808f0ebdc viable/strict -> origin/viable/strict 2025-04-21T21:42:37.6629445Z t [tag update] ciflow/inductor/148893 -> ciflow/inductor/148893 2025-04-21T21:42:37.6631194Z t [tag update] ciflow/inductor/149707 -> ciflow/inductor/149707 2025-04-21T21:42:37.6632875Z t [tag update] ciflow/inductor/150236 -> ciflow/inductor/150236 2025-04-21T21:42:37.6635147Z t [tag update] ciflow/inductor/151023 -> ciflow/inductor/151023 2025-04-21T21:42:37.6636256Z t [tag update] ciflow/inductor/151038 -> ciflow/inductor/151038 2025-04-21T21:42:37.6637856Z t [tag update] ciflow/inductor/151041 -> ciflow/inductor/151041 2025-04-21T21:42:37.6639576Z t [tag update] ciflow/inductor/151293 -> ciflow/inductor/151293 2025-04-21T21:42:37.6640639Z t [tag update] ciflow/inductor/151294 -> ciflow/inductor/151294 2025-04-21T21:42:37.6642446Z t [tag update] ciflow/inductor/151295 -> ciflow/inductor/151295 2025-04-21T21:42:37.6643995Z t [tag update] ciflow/inductor/151601 -> ciflow/inductor/151601 2025-04-21T21:42:37.6645183Z t [tag update] ciflow/inductor/151634 -> ciflow/inductor/151634 2025-04-21T21:42:37.6646994Z t [tag update] ciflow/inductor/151731 -> ciflow/inductor/151731 2025-04-21T21:42:37.6648711Z * [new tag] ciflow/inductor/151824 -> ciflow/inductor/151824 2025-04-21T21:42:37.6649871Z * [new tag] ciflow/inductor/151825 -> ciflow/inductor/151825 2025-04-21T21:42:37.6651555Z * [new tag] ciflow/inductor/151828 -> ciflow/inductor/151828 2025-04-21T21:42:37.6652538Z * [new tag] ciflow/inductor/151833 -> ciflow/inductor/151833 2025-04-21T21:42:37.6655156Z t [tag update] ciflow/mps/147644 -> ciflow/mps/147644 2025-04-21T21:42:37.6659153Z * [new tag] ciflow/trunk/147476 -> ciflow/trunk/147476 2025-04-21T21:42:37.6661507Z * [new tag] ciflow/trunk/150897 -> ciflow/trunk/150897 2025-04-21T21:42:37.6662948Z t [tag update] ciflow/trunk/151023 -> ciflow/trunk/151023 2025-04-21T21:42:37.6664555Z t [tag update] ciflow/trunk/151253 -> ciflow/trunk/151253 2025-04-21T21:42:37.6665737Z t [tag update] ciflow/trunk/151295 -> ciflow/trunk/151295 2025-04-21T21:42:37.6667708Z t [tag update] ciflow/trunk/151634 -> ciflow/trunk/151634 2025-04-21T21:42:37.6668756Z * [new tag] ciflow/trunk/151701 -> ciflow/trunk/151701 2025-04-21T21:42:37.6670400Z * [new tag] ciflow/trunk/151795 -> ciflow/trunk/151795 2025-04-21T21:42:37.6671454Z * [new tag] ciflow/trunk/151797 -> ciflow/trunk/151797 2025-04-21T21:42:37.6673289Z t [tag update] ciflow/trunk/151798 -> ciflow/trunk/151798 2025-04-21T21:42:37.6674367Z * [new tag] ciflow/trunk/151816 -> ciflow/trunk/151816 2025-04-21T21:42:37.7500105Z [command]/usr/bin/git rev-parse --verify --quiet b68c0effe97351b0c24b2f7aef671559be6aa977^{object} 2025-04-21T21:42:37.7529822Z b68c0effe97351b0c24b2f7aef671559be6aa977 2025-04-21T21:42:37.7535021Z ##[endgroup] 2025-04-21T21:42:37.7535436Z ##[group]Determining the checkout info 2025-04-21T21:42:37.7536301Z ##[endgroup] 2025-04-21T21:42:37.7541217Z [command]/usr/bin/git sparse-checkout disable 2025-04-21T21:42:38.0964334Z [command]/usr/bin/git config --local --unset-all extensions.worktreeConfig 2025-04-21T21:42:38.0998207Z ##[group]Checking out the ref 2025-04-21T21:42:38.1001557Z [command]/usr/bin/git checkout --progress --force b68c0effe97351b0c24b2f7aef671559be6aa977 2025-04-21T21:42:38.2605044Z Previous HEAD position was f37e138bc4f [MPS] Enable log1p and sigmoid for int64 (#151791) 2025-04-21T21:42:38.2620242Z HEAD is now at b68c0effe97 Update on "use statically known true instead of guard size oblivious in bmm and mm decompositions . " 2025-04-21T21:42:38.2673561Z ##[endgroup] 2025-04-21T21:42:38.2673984Z ##[group]Setting up auth for fetching submodules 2025-04-21T21:42:38.2681129Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2025-04-21T21:42:38.2732750Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2025-04-21T21:42:38.2765901Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2025-04-21T21:42:38.2798052Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2025-04-21T21:42:38.2827364Z ##[endgroup] 2025-04-21T21:42:38.2828004Z ##[group]Fetching submodules 2025-04-21T21:42:38.2830958Z [command]/usr/bin/git submodule sync --recursive 2025-04-21T21:42:38.3207739Z Synchronizing submodule url for 'android/libs/fbjni' 2025-04-21T21:42:38.3234947Z Synchronizing submodule url for 'third_party/FP16' 2025-04-21T21:42:38.3261304Z Synchronizing submodule url for 'third_party/FXdiv' 2025-04-21T21:42:38.3288047Z Synchronizing submodule url for 'third_party/NNPACK' 2025-04-21T21:42:38.3314417Z Synchronizing submodule url for 'third_party/NVTX' 2025-04-21T21:42:38.3340515Z Synchronizing submodule url for 'third_party/VulkanMemoryAllocator' 2025-04-21T21:42:38.3366874Z Synchronizing submodule url for 'third_party/XNNPACK' 2025-04-21T21:42:38.3409344Z Synchronizing submodule url for 'third_party/benchmark' 2025-04-21T21:42:38.3435897Z Synchronizing submodule url for 'third_party/composable_kernel' 2025-04-21T21:42:38.3468826Z Synchronizing submodule url for 'third_party/cpp-httplib' 2025-04-21T21:42:38.3495253Z Synchronizing submodule url for 'third_party/cpuinfo' 2025-04-21T21:42:38.3522575Z Synchronizing submodule url for 'third_party/cudnn_frontend' 2025-04-21T21:42:38.3548752Z Synchronizing submodule url for 'third_party/cutlass' 2025-04-21T21:42:38.3582941Z Synchronizing submodule url for 'third_party/eigen' 2025-04-21T21:42:38.3612362Z Synchronizing submodule url for 'third_party/fbgemm' 2025-04-21T21:42:38.3637642Z Synchronizing submodule url for 'third_party/fbgemm/third_party/asmjit' 2025-04-21T21:42:38.3665839Z Synchronizing submodule url for 'third_party/fbgemm/third_party/cpuinfo' 2025-04-21T21:42:38.3692861Z Synchronizing submodule url for 'third_party/fbgemm/third_party/cutlass' 2025-04-21T21:42:38.3725871Z Synchronizing submodule url for 'third_party/fbgemm/third_party/googletest' 2025-04-21T21:42:38.3751996Z Synchronizing submodule url for 'third_party/fbgemm/third_party/hipify_torch' 2025-04-21T21:42:38.3782516Z Synchronizing submodule url for 'third_party/flash-attention' 2025-04-21T21:42:38.3807749Z Synchronizing submodule url for 'third_party/flash-attention/csrc/composable_kernel' 2025-04-21T21:42:38.3840423Z Synchronizing submodule url for 'third_party/flash-attention/csrc/cutlass' 2025-04-21T21:42:38.3884414Z Synchronizing submodule url for 'third_party/flatbuffers' 2025-04-21T21:42:38.3912866Z Synchronizing submodule url for 'third_party/fmt' 2025-04-21T21:42:38.3939115Z Synchronizing submodule url for 'third_party/gemmlowp/gemmlowp' 2025-04-21T21:42:38.3964938Z Synchronizing submodule url for 'third_party/gloo' 2025-04-21T21:42:38.3991884Z Synchronizing submodule url for 'third_party/googletest' 2025-04-21T21:42:38.4018525Z Synchronizing submodule url for 'third_party/ideep' 2025-04-21T21:42:38.4041203Z Synchronizing submodule url for 'third_party/ideep/mkl-dnn' 2025-04-21T21:42:38.4077337Z Synchronizing submodule url for 'third_party/ittapi' 2025-04-21T21:42:38.4103670Z Synchronizing submodule url for 'third_party/kineto' 2025-04-21T21:42:38.4129317Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog' 2025-04-21T21:42:38.4153762Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-04-21T21:42:38.4182754Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-04-21T21:42:38.4210541Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-04-21T21:42:38.4238561Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-04-21T21:42:38.4263576Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-04-21T21:42:38.4294903Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-04-21T21:42:38.4323269Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-04-21T21:42:38.4350302Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-04-21T21:42:38.4380084Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-04-21T21:42:38.4409820Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/fmt' 2025-04-21T21:42:38.4436061Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/googletest' 2025-04-21T21:42:38.4464902Z Synchronizing submodule url for 'third_party/kleidiai' 2025-04-21T21:42:38.4492445Z Synchronizing submodule url for 'third_party/mimalloc' 2025-04-21T21:42:38.4518409Z Synchronizing submodule url for 'third_party/nlohmann' 2025-04-21T21:42:38.4546087Z Synchronizing submodule url for 'third_party/onnx' 2025-04-21T21:42:38.4586220Z Synchronizing submodule url for 'third_party/onnx/third_party/pybind11' 2025-04-21T21:42:38.4617819Z Synchronizing submodule url for 'third_party/opentelemetry-cpp' 2025-04-21T21:42:38.4643502Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-04-21T21:42:38.4669898Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/googletest' 2025-04-21T21:42:38.4697170Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-04-21T21:42:38.4723210Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-04-21T21:42:38.4750549Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-04-21T21:42:38.4776211Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-04-21T21:42:38.4804649Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-04-21T21:42:38.4828805Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-04-21T21:42:38.4856881Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-04-21T21:42:38.4886074Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-04-21T21:42:38.4934007Z Synchronizing submodule url for 'third_party/pocketfft' 2025-04-21T21:42:38.4961142Z Synchronizing submodule url for 'third_party/protobuf' 2025-04-21T21:42:38.4989776Z Synchronizing submodule url for 'third_party/protobuf/third_party/benchmark' 2025-04-21T21:42:38.5015543Z Synchronizing submodule url for 'third_party/protobuf/third_party/googletest' 2025-04-21T21:42:38.5045598Z Synchronizing submodule url for 'third_party/psimd' 2025-04-21T21:42:38.5072166Z Synchronizing submodule url for 'third_party/pthreadpool' 2025-04-21T21:42:38.5098794Z Synchronizing submodule url for 'third_party/pybind11' 2025-04-21T21:42:38.5125314Z Synchronizing submodule url for 'third_party/python-peachpy' 2025-04-21T21:42:38.5151574Z Synchronizing submodule url for 'third_party/sleef' 2025-04-21T21:42:38.5179005Z Synchronizing submodule url for 'third_party/tensorpipe' 2025-04-21T21:42:38.5203245Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/googletest' 2025-04-21T21:42:38.5229053Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/libnop' 2025-04-21T21:42:38.5254716Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/libuv' 2025-04-21T21:42:38.5282305Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/pybind11' 2025-04-21T21:42:38.5305575Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-04-21T21:42:38.5350181Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2025-04-21T21:42:38.5938204Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2025-04-21T21:42:38.6103495Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2025-04-21T21:42:38.6229067Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2025-04-21T21:42:38.6620491Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2025-04-21T21:42:38.7219254Z Submodule path 'third_party/NVTX': checked out 'e170594ac7cf1dac584da473d4ca9301087090c1' 2025-04-21T21:42:38.7762500Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2025-04-21T21:42:40.0589483Z Submodule path 'third_party/XNNPACK': checked out '51a0103656eff6fc9bfd39a4597923c4b542c883' 2025-04-21T21:42:40.0924430Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2025-04-21T21:42:40.5224474Z Submodule path 'third_party/composable_kernel': checked out '8086bbe3a78d931eb96fe12fdc014082e18d18d3' 2025-04-21T21:42:40.5801950Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2025-04-21T21:42:40.7148281Z Submodule path 'third_party/cpuinfo': checked out '5e3d2445e6a84d9599bee2bf78edbb4d80865e1d' 2025-04-21T21:42:40.7638690Z Submodule path 'third_party/cudnn_frontend': checked out '8801fd7b31c2f798732ed1e1dd4711a9cde4217d' 2025-04-21T21:42:41.7014970Z Submodule path 'third_party/cutlass': checked out 'afa1772203677c5118fcd82537a9c8fefbcc7008' 2025-04-21T21:42:42.0514921Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2025-04-21T21:42:42.1469447Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2025-04-21T21:42:42.2101676Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2025-04-21T21:42:42.3467091Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2025-04-21T21:42:43.0474997Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2025-04-21T21:42:43.1098521Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2025-04-21T21:42:43.1241089Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2025-04-21T21:42:43.2417281Z Submodule path 'third_party/flash-attention': checked out '979702c87a8713a8e0a5e9fee122b90d2ef13be5' 2025-04-21T21:42:43.6621536Z Submodule path 'third_party/flash-attention/csrc/composable_kernel': checked out '888317e698e9803c62bd38568abc9e05d7709f33' 2025-04-21T21:42:44.5920286Z Submodule path 'third_party/flash-attention/csrc/cutlass': checked out 'c506e16788cb08416a4a57e11a9067beeee29420' 2025-04-21T21:42:44.8119567Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2025-04-21T21:42:44.8566884Z Submodule path 'third_party/fmt': checked out '123913715afeb8a437e6388b4473fcc4753e1c9a' 2025-04-21T21:42:44.9075545Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2025-04-21T21:42:44.9474618Z Submodule path 'third_party/gloo': checked out 'c61070427610ccd923efe3e7f8b3eca12bbcc31a' 2025-04-21T21:42:45.0067405Z Submodule path 'third_party/googletest': checked out 'b514bdc898e2951020cbdca1304b75f5950d1f59' 2025-04-21T21:42:45.0249207Z Submodule path 'third_party/ideep': checked out '719d8e6cd7f7a0e01b155657526d693acf97c2b3' 2025-04-21T21:42:45.8465835Z Submodule path 'third_party/ideep/mkl-dnn': checked out '8d263e693366ef8db40acc569cc7d8edf644556d' 2025-04-21T21:42:45.8692962Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2025-04-21T21:42:45.9812338Z Submodule path 'third_party/kineto': checked out '5fa4bd8c8fb581a621f72957d0cb11431cc4714f' 2025-04-21T21:42:46.0934660Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2025-04-21T21:42:46.3355509Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2025-04-21T21:42:46.3635555Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2025-04-21T21:42:46.4135575Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2025-04-21T21:42:46.4334458Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2025-04-21T21:42:46.4459352Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2025-04-21T21:42:46.4717578Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2025-04-21T21:42:46.5279642Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2025-04-21T21:42:46.6717799Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2025-04-21T21:42:46.6958806Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2025-04-21T21:42:46.7391977Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2025-04-21T21:42:46.8002613Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2025-04-21T21:42:46.8539864Z Submodule path 'third_party/kleidiai': checked out 'ef685a13cfbe8d418aa2ed34350e21e4938358b6' 2025-04-21T21:42:46.9082310Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2025-04-21T21:42:47.0652218Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2025-04-21T21:42:47.7143837Z Submodule path 'third_party/onnx': checked out 'b8baa8446686496da4cc8fda09f2b6fe65c2a02c' 2025-04-21T21:42:47.7674503Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '3e9dfa2866941655c56877882565e7577de6fc7b' 2025-04-21T21:42:47.8939052Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2025-04-21T21:42:47.9252757Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2025-04-21T21:42:47.9812087Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2025-04-21T21:42:47.9994231Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2025-04-21T21:42:48.1622605Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2025-04-21T21:42:48.1797293Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2025-04-21T21:42:48.2019940Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2025-04-21T21:42:48.2279305Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2025-04-21T21:42:48.5631153Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2025-04-21T21:42:48.6256202Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2025-04-21T21:42:49.5787386Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2025-04-21T21:42:49.5936178Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2025-04-21T21:42:49.9942862Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2025-04-21T21:42:50.0161704Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2025-04-21T21:42:50.0860966Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2025-04-21T21:42:50.0986611Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2025-04-21T21:42:50.1149668Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2025-04-21T21:42:50.1688319Z Submodule path 'third_party/pybind11': checked out 'a2e59f0e7065404b44dfe92a28aca47ba1378dc4' 2025-04-21T21:42:50.2088892Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2025-04-21T21:42:50.2659825Z Submodule path 'third_party/sleef': checked out '56e1f79cb140fb9326d612d0be06b5250565cade' 2025-04-21T21:42:50.3098695Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2025-04-21T21:42:50.3696398Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2025-04-21T21:42:50.3937051Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2025-04-21T21:42:50.4906397Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2025-04-21T21:42:50.5321849Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2025-04-21T21:42:50.5442798Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2025-04-21T21:42:50.5495311Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2025-04-21T21:42:50.5855326Z Entering 'android/libs/fbjni' 2025-04-21T21:42:50.5908574Z Entering 'third_party/FP16' 2025-04-21T21:42:50.5961502Z Entering 'third_party/FXdiv' 2025-04-21T21:42:50.6014300Z Entering 'third_party/NNPACK' 2025-04-21T21:42:50.6066827Z Entering 'third_party/NVTX' 2025-04-21T21:42:50.6120579Z Entering 'third_party/VulkanMemoryAllocator' 2025-04-21T21:42:50.6173639Z Entering 'third_party/XNNPACK' 2025-04-21T21:42:50.6245344Z Entering 'third_party/benchmark' 2025-04-21T21:42:50.6299431Z Entering 'third_party/composable_kernel' 2025-04-21T21:42:50.6358854Z Entering 'third_party/cpp-httplib' 2025-04-21T21:42:50.6411853Z Entering 'third_party/cpuinfo' 2025-04-21T21:42:50.6464693Z Entering 'third_party/cudnn_frontend' 2025-04-21T21:42:50.6518131Z Entering 'third_party/cutlass' 2025-04-21T21:42:50.6579633Z Entering 'third_party/eigen' 2025-04-21T21:42:50.6633898Z Entering 'third_party/fbgemm' 2025-04-21T21:42:50.6687258Z Entering 'third_party/fbgemm/third_party/asmjit' 2025-04-21T21:42:50.6738012Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2025-04-21T21:42:50.6794055Z Entering 'third_party/fbgemm/third_party/cutlass' 2025-04-21T21:42:50.6851197Z Entering 'third_party/fbgemm/third_party/googletest' 2025-04-21T21:42:50.6902095Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2025-04-21T21:42:50.6956814Z Entering 'third_party/flash-attention' 2025-04-21T21:42:50.7010088Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-04-21T21:42:50.7068077Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-04-21T21:42:50.7130514Z Entering 'third_party/flatbuffers' 2025-04-21T21:42:50.7187227Z Entering 'third_party/fmt' 2025-04-21T21:42:50.7239173Z Entering 'third_party/gemmlowp/gemmlowp' 2025-04-21T21:42:50.7291956Z Entering 'third_party/gloo' 2025-04-21T21:42:50.7344715Z Entering 'third_party/googletest' 2025-04-21T21:42:50.7398128Z Entering 'third_party/ideep' 2025-04-21T21:42:50.7449622Z Entering 'third_party/ideep/mkl-dnn' 2025-04-21T21:42:50.7510267Z Entering 'third_party/ittapi' 2025-04-21T21:42:50.7562069Z Entering 'third_party/kineto' 2025-04-21T21:42:50.7614606Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-04-21T21:42:50.7664594Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-04-21T21:42:50.7721171Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-04-21T21:42:50.7772125Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-04-21T21:42:50.7822836Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-04-21T21:42:50.7871930Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-04-21T21:42:50.7929182Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-04-21T21:42:50.7979527Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-04-21T21:42:50.8030454Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-04-21T21:42:50.8083052Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-04-21T21:42:50.8138466Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-04-21T21:42:50.8190142Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-04-21T21:42:50.8244802Z Entering 'third_party/kleidiai' 2025-04-21T21:42:50.8298824Z Entering 'third_party/mimalloc' 2025-04-21T21:42:50.8351671Z Entering 'third_party/nlohmann' 2025-04-21T21:42:50.8406096Z Entering 'third_party/onnx' 2025-04-21T21:42:50.8474825Z Entering 'third_party/onnx/third_party/pybind11' 2025-04-21T21:42:50.8533482Z Entering 'third_party/opentelemetry-cpp' 2025-04-21T21:42:50.8587846Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-04-21T21:42:50.8642333Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-04-21T21:42:50.8693862Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-04-21T21:42:50.8743350Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-04-21T21:42:50.8795939Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-04-21T21:42:50.8846410Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-04-21T21:42:50.8896661Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-04-21T21:42:50.8945205Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-04-21T21:42:50.8998357Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-04-21T21:42:50.9051240Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-04-21T21:42:50.9125879Z Entering 'third_party/pocketfft' 2025-04-21T21:42:50.9178738Z Entering 'third_party/protobuf' 2025-04-21T21:42:50.9234663Z Entering 'third_party/protobuf/third_party/benchmark' 2025-04-21T21:42:50.9287812Z Entering 'third_party/protobuf/third_party/googletest' 2025-04-21T21:42:50.9343011Z Entering 'third_party/psimd' 2025-04-21T21:42:50.9396636Z Entering 'third_party/pthreadpool' 2025-04-21T21:42:50.9447984Z Entering 'third_party/pybind11' 2025-04-21T21:42:50.9500252Z Entering 'third_party/python-peachpy' 2025-04-21T21:42:50.9551567Z Entering 'third_party/sleef' 2025-04-21T21:42:50.9603764Z Entering 'third_party/tensorpipe' 2025-04-21T21:42:50.9656183Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-04-21T21:42:50.9706415Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-04-21T21:42:50.9756507Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-04-21T21:42:50.9807142Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-04-21T21:42:50.9856337Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-04-21T21:42:50.9930341Z ##[endgroup] 2025-04-21T21:42:50.9930842Z ##[group]Persisting credentials for submodules 2025-04-21T21:42:50.9937178Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2025-04-21T21:42:51.0300003Z Entering 'android/libs/fbjni' 2025-04-21T21:42:51.0343656Z url.https://github.com/.insteadof 2025-04-21T21:42:51.0343972Z url.https://github.com/.insteadof 2025-04-21T21:42:51.0389049Z Entering 'third_party/FP16' 2025-04-21T21:42:51.0432827Z url.https://github.com/.insteadof 2025-04-21T21:42:51.0433443Z url.https://github.com/.insteadof 2025-04-21T21:42:51.0477738Z Entering 'third_party/FXdiv' 2025-04-21T21:42:51.0525087Z url.https://github.com/.insteadof 2025-04-21T21:42:51.0525555Z url.https://github.com/.insteadof 2025-04-21T21:42:51.0568480Z Entering 'third_party/NNPACK' 2025-04-21T21:42:51.0613412Z url.https://github.com/.insteadof 2025-04-21T21:42:51.0613858Z url.https://github.com/.insteadof 2025-04-21T21:42:51.0658124Z Entering 'third_party/NVTX' 2025-04-21T21:42:51.0703967Z url.https://github.com/.insteadof 2025-04-21T21:42:51.0704303Z url.https://github.com/.insteadof 2025-04-21T21:42:51.0748517Z Entering 'third_party/VulkanMemoryAllocator' 2025-04-21T21:42:51.0792734Z url.https://github.com/.insteadof 2025-04-21T21:42:51.0793192Z url.https://github.com/.insteadof 2025-04-21T21:42:51.0838267Z Entering 'third_party/XNNPACK' 2025-04-21T21:42:51.0883351Z url.https://github.com/.insteadof 2025-04-21T21:42:51.0883672Z url.https://github.com/.insteadof 2025-04-21T21:42:51.0944058Z Entering 'third_party/benchmark' 2025-04-21T21:42:51.0988314Z url.https://github.com/.insteadof 2025-04-21T21:42:51.0988640Z url.https://github.com/.insteadof 2025-04-21T21:42:51.1032883Z Entering 'third_party/composable_kernel' 2025-04-21T21:42:51.1076779Z url.https://github.com/.insteadof 2025-04-21T21:42:51.1077082Z url.https://github.com/.insteadof 2025-04-21T21:42:51.1126998Z Entering 'third_party/cpp-httplib' 2025-04-21T21:42:51.1171181Z url.https://github.com/.insteadof 2025-04-21T21:42:51.1171493Z url.https://github.com/.insteadof 2025-04-21T21:42:51.1215522Z Entering 'third_party/cpuinfo' 2025-04-21T21:42:51.1261229Z url.https://github.com/.insteadof 2025-04-21T21:42:51.1261539Z url.https://github.com/.insteadof 2025-04-21T21:42:51.1306257Z Entering 'third_party/cudnn_frontend' 2025-04-21T21:42:51.1350273Z url.https://github.com/.insteadof 2025-04-21T21:42:51.1350604Z url.https://github.com/.insteadof 2025-04-21T21:42:51.1394945Z Entering 'third_party/cutlass' 2025-04-21T21:42:51.1438203Z url.https://github.com/.insteadof 2025-04-21T21:42:51.1438663Z url.https://github.com/.insteadof 2025-04-21T21:42:51.1492413Z Entering 'third_party/eigen' 2025-04-21T21:42:51.1536151Z url.https://github.com/.insteadof 2025-04-21T21:42:51.1536659Z url.https://github.com/.insteadof 2025-04-21T21:42:51.1582993Z Entering 'third_party/fbgemm' 2025-04-21T21:42:51.1627152Z url.https://github.com/.insteadof 2025-04-21T21:42:51.1627492Z url.https://github.com/.insteadof 2025-04-21T21:42:51.1671283Z Entering 'third_party/fbgemm/third_party/asmjit' 2025-04-21T21:42:51.1718818Z url.https://github.com/.insteadof 2025-04-21T21:42:51.1719198Z url.https://github.com/.insteadof 2025-04-21T21:42:51.1761760Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2025-04-21T21:42:51.1805420Z url.https://github.com/.insteadof 2025-04-21T21:42:51.1805763Z url.https://github.com/.insteadof 2025-04-21T21:42:51.1849861Z Entering 'third_party/fbgemm/third_party/cutlass' 2025-04-21T21:42:51.1893637Z url.https://github.com/.insteadof 2025-04-21T21:42:51.1893956Z url.https://github.com/.insteadof 2025-04-21T21:42:51.1944654Z Entering 'third_party/fbgemm/third_party/googletest' 2025-04-21T21:42:51.1987982Z url.https://github.com/.insteadof 2025-04-21T21:42:51.1988329Z url.https://github.com/.insteadof 2025-04-21T21:42:51.2031370Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2025-04-21T21:42:51.2073708Z url.https://github.com/.insteadof 2025-04-21T21:42:51.2074038Z url.https://github.com/.insteadof 2025-04-21T21:42:51.2120819Z Entering 'third_party/flash-attention' 2025-04-21T21:42:51.2165694Z url.https://github.com/.insteadof 2025-04-21T21:42:51.2166017Z url.https://github.com/.insteadof 2025-04-21T21:42:51.2211418Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-04-21T21:42:51.2255149Z url.https://github.com/.insteadof 2025-04-21T21:42:51.2255470Z url.https://github.com/.insteadof 2025-04-21T21:42:51.2306220Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-04-21T21:42:51.2348292Z url.https://github.com/.insteadof 2025-04-21T21:42:51.2348614Z url.https://github.com/.insteadof 2025-04-21T21:42:51.2404123Z Entering 'third_party/flatbuffers' 2025-04-21T21:42:51.2447436Z url.https://github.com/.insteadof 2025-04-21T21:42:51.2447752Z url.https://github.com/.insteadof 2025-04-21T21:42:51.2494766Z Entering 'third_party/fmt' 2025-04-21T21:42:51.2537893Z url.https://github.com/.insteadof 2025-04-21T21:42:51.2538208Z url.https://github.com/.insteadof 2025-04-21T21:42:51.2581951Z Entering 'third_party/gemmlowp/gemmlowp' 2025-04-21T21:42:51.2626003Z url.https://github.com/.insteadof 2025-04-21T21:42:51.2626652Z url.https://github.com/.insteadof 2025-04-21T21:42:51.2671434Z Entering 'third_party/gloo' 2025-04-21T21:42:51.2716032Z url.https://github.com/.insteadof 2025-04-21T21:42:51.2716370Z url.https://github.com/.insteadof 2025-04-21T21:42:51.2760275Z Entering 'third_party/googletest' 2025-04-21T21:42:51.2804411Z url.https://github.com/.insteadof 2025-04-21T21:42:51.2804750Z url.https://github.com/.insteadof 2025-04-21T21:42:51.2848782Z Entering 'third_party/ideep' 2025-04-21T21:42:51.2893187Z url.https://github.com/.insteadof 2025-04-21T21:42:51.2893513Z url.https://github.com/.insteadof 2025-04-21T21:42:51.2935644Z Entering 'third_party/ideep/mkl-dnn' 2025-04-21T21:42:51.2979690Z url.https://github.com/.insteadof 2025-04-21T21:42:51.2980008Z url.https://github.com/.insteadof 2025-04-21T21:42:51.3032816Z Entering 'third_party/ittapi' 2025-04-21T21:42:51.3077754Z url.https://github.com/.insteadof 2025-04-21T21:42:51.3078097Z url.https://github.com/.insteadof 2025-04-21T21:42:51.3121386Z Entering 'third_party/kineto' 2025-04-21T21:42:51.3165355Z url.https://github.com/.insteadof 2025-04-21T21:42:51.3165668Z url.https://github.com/.insteadof 2025-04-21T21:42:51.3209053Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-04-21T21:42:51.3251957Z url.https://github.com/.insteadof 2025-04-21T21:42:51.3252285Z url.https://github.com/.insteadof 2025-04-21T21:42:51.3295583Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-04-21T21:42:51.3340268Z url.https://github.com/.insteadof 2025-04-21T21:42:51.3340919Z url.https://github.com/.insteadof 2025-04-21T21:42:51.3386939Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-04-21T21:42:51.3431424Z url.https://github.com/.insteadof 2025-04-21T21:42:51.3431744Z url.https://github.com/.insteadof 2025-04-21T21:42:51.3476592Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-04-21T21:42:51.3521086Z url.https://github.com/.insteadof 2025-04-21T21:42:51.3521412Z url.https://github.com/.insteadof 2025-04-21T21:42:51.3565767Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-04-21T21:42:51.3608914Z url.https://github.com/.insteadof 2025-04-21T21:42:51.3609269Z url.https://github.com/.insteadof 2025-04-21T21:42:51.3652365Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-04-21T21:42:51.3696502Z url.https://github.com/.insteadof 2025-04-21T21:42:51.3696846Z url.https://github.com/.insteadof 2025-04-21T21:42:51.3748942Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-04-21T21:42:51.3792818Z url.https://github.com/.insteadof 2025-04-21T21:42:51.3793132Z url.https://github.com/.insteadof 2025-04-21T21:42:51.3837836Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-04-21T21:42:51.3884257Z url.https://github.com/.insteadof 2025-04-21T21:42:51.3885358Z url.https://github.com/.insteadof 2025-04-21T21:42:51.3928546Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-04-21T21:42:51.3977368Z url.https://github.com/.insteadof 2025-04-21T21:42:51.3977702Z url.https://github.com/.insteadof 2025-04-21T21:42:51.4022388Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-04-21T21:42:51.4068086Z url.https://github.com/.insteadof 2025-04-21T21:42:51.4068407Z url.https://github.com/.insteadof 2025-04-21T21:42:51.4116121Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-04-21T21:42:51.4159401Z url.https://github.com/.insteadof 2025-04-21T21:42:51.4160017Z url.https://github.com/.insteadof 2025-04-21T21:42:51.4203437Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-04-21T21:42:51.4245675Z url.https://github.com/.insteadof 2025-04-21T21:42:51.4245997Z url.https://github.com/.insteadof 2025-04-21T21:42:51.4292427Z Entering 'third_party/kleidiai' 2025-04-21T21:42:51.4335609Z url.https://github.com/.insteadof 2025-04-21T21:42:51.4335919Z url.https://github.com/.insteadof 2025-04-21T21:42:51.4380099Z Entering 'third_party/mimalloc' 2025-04-21T21:42:51.4423274Z url.https://github.com/.insteadof 2025-04-21T21:42:51.4423746Z url.https://github.com/.insteadof 2025-04-21T21:42:51.4467273Z Entering 'third_party/nlohmann' 2025-04-21T21:42:51.4517204Z url.https://github.com/.insteadof 2025-04-21T21:42:51.4517540Z url.https://github.com/.insteadof 2025-04-21T21:42:51.4562164Z Entering 'third_party/onnx' 2025-04-21T21:42:51.4605996Z url.https://github.com/.insteadof 2025-04-21T21:42:51.4606333Z url.https://github.com/.insteadof 2025-04-21T21:42:51.4664441Z Entering 'third_party/onnx/third_party/pybind11' 2025-04-21T21:42:51.4708516Z url.https://github.com/.insteadof 2025-04-21T21:42:51.4708855Z url.https://github.com/.insteadof 2025-04-21T21:42:51.4758963Z Entering 'third_party/opentelemetry-cpp' 2025-04-21T21:42:51.4803183Z url.https://github.com/.insteadof 2025-04-21T21:42:51.4803503Z url.https://github.com/.insteadof 2025-04-21T21:42:51.4847930Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-04-21T21:42:51.4891516Z url.https://github.com/.insteadof 2025-04-21T21:42:51.4891848Z url.https://github.com/.insteadof 2025-04-21T21:42:51.4935846Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-04-21T21:42:51.4978233Z url.https://github.com/.insteadof 2025-04-21T21:42:51.4978549Z url.https://github.com/.insteadof 2025-04-21T21:42:51.5022036Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-04-21T21:42:51.5064166Z url.https://github.com/.insteadof 2025-04-21T21:42:51.5064560Z url.https://github.com/.insteadof 2025-04-21T21:42:51.5109574Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-04-21T21:42:51.5152698Z url.https://github.com/.insteadof 2025-04-21T21:42:51.5153019Z url.https://github.com/.insteadof 2025-04-21T21:42:51.5200190Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-04-21T21:42:51.5243431Z url.https://github.com/.insteadof 2025-04-21T21:42:51.5243762Z url.https://github.com/.insteadof 2025-04-21T21:42:51.5286620Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-04-21T21:42:51.5329146Z url.https://github.com/.insteadof 2025-04-21T21:42:51.5329919Z url.https://github.com/.insteadof 2025-04-21T21:42:51.5376159Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-04-21T21:42:51.5420291Z url.https://github.com/.insteadof 2025-04-21T21:42:51.5420624Z url.https://github.com/.insteadof 2025-04-21T21:42:51.5461204Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-04-21T21:42:51.5504509Z url.https://github.com/.insteadof 2025-04-21T21:42:51.5504819Z url.https://github.com/.insteadof 2025-04-21T21:42:51.5550971Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-04-21T21:42:51.5595638Z url.https://github.com/.insteadof 2025-04-21T21:42:51.5596056Z url.https://github.com/.insteadof 2025-04-21T21:42:51.5643412Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-04-21T21:42:51.5686540Z url.https://github.com/.insteadof 2025-04-21T21:42:51.5686956Z url.https://github.com/.insteadof 2025-04-21T21:42:51.5752988Z Entering 'third_party/pocketfft' 2025-04-21T21:42:51.5798062Z url.https://github.com/.insteadof 2025-04-21T21:42:51.5798689Z url.https://github.com/.insteadof 2025-04-21T21:42:51.5842588Z Entering 'third_party/protobuf' 2025-04-21T21:42:51.5887697Z url.https://github.com/.insteadof 2025-04-21T21:42:51.5888153Z url.https://github.com/.insteadof 2025-04-21T21:42:51.5934871Z Entering 'third_party/protobuf/third_party/benchmark' 2025-04-21T21:42:51.5979475Z url.https://github.com/.insteadof 2025-04-21T21:42:51.5979862Z url.https://github.com/.insteadof 2025-04-21T21:42:51.6021860Z Entering 'third_party/protobuf/third_party/googletest' 2025-04-21T21:42:51.6064810Z url.https://github.com/.insteadof 2025-04-21T21:42:51.6065139Z url.https://github.com/.insteadof 2025-04-21T21:42:51.6111790Z Entering 'third_party/psimd' 2025-04-21T21:42:51.6156248Z url.https://github.com/.insteadof 2025-04-21T21:42:51.6156569Z url.https://github.com/.insteadof 2025-04-21T21:42:51.6201340Z Entering 'third_party/pthreadpool' 2025-04-21T21:42:51.6245092Z url.https://github.com/.insteadof 2025-04-21T21:42:51.6245394Z url.https://github.com/.insteadof 2025-04-21T21:42:51.6289984Z Entering 'third_party/pybind11' 2025-04-21T21:42:51.6334145Z url.https://github.com/.insteadof 2025-04-21T21:42:51.6334452Z url.https://github.com/.insteadof 2025-04-21T21:42:51.6378806Z Entering 'third_party/python-peachpy' 2025-04-21T21:42:51.6424333Z url.https://github.com/.insteadof 2025-04-21T21:42:51.6424795Z url.https://github.com/.insteadof 2025-04-21T21:42:51.6468344Z Entering 'third_party/sleef' 2025-04-21T21:42:51.6515491Z url.https://github.com/.insteadof 2025-04-21T21:42:51.6515810Z url.https://github.com/.insteadof 2025-04-21T21:42:51.6559639Z Entering 'third_party/tensorpipe' 2025-04-21T21:42:51.6604582Z url.https://github.com/.insteadof 2025-04-21T21:42:51.6604900Z url.https://github.com/.insteadof 2025-04-21T21:42:51.6647865Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-04-21T21:42:51.6691317Z url.https://github.com/.insteadof 2025-04-21T21:42:51.6691633Z url.https://github.com/.insteadof 2025-04-21T21:42:51.6734765Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-04-21T21:42:51.6777608Z url.https://github.com/.insteadof 2025-04-21T21:42:51.6777925Z url.https://github.com/.insteadof 2025-04-21T21:42:51.6820800Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-04-21T21:42:51.6866589Z url.https://github.com/.insteadof 2025-04-21T21:42:51.6866939Z url.https://github.com/.insteadof 2025-04-21T21:42:51.6911721Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-04-21T21:42:51.6954685Z url.https://github.com/.insteadof 2025-04-21T21:42:51.6955025Z url.https://github.com/.insteadof 2025-04-21T21:42:51.6997454Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-04-21T21:42:51.7040856Z url.https://github.com/.insteadof 2025-04-21T21:42:51.7041239Z url.https://github.com/.insteadof 2025-04-21T21:42:51.7114442Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2025-04-21T21:42:51.7473580Z Entering 'android/libs/fbjni' 2025-04-21T21:42:51.7537733Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2025-04-21T21:42:51.7560746Z Entering 'third_party/FP16' 2025-04-21T21:42:51.7623515Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2025-04-21T21:42:51.7646167Z Entering 'third_party/FXdiv' 2025-04-21T21:42:51.7709319Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2025-04-21T21:42:51.7731601Z Entering 'third_party/NNPACK' 2025-04-21T21:42:51.7795265Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2025-04-21T21:42:51.7817802Z Entering 'third_party/NVTX' 2025-04-21T21:42:51.7880871Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NVTX/config remote.origin.url 2025-04-21T21:42:51.7903909Z Entering 'third_party/VulkanMemoryAllocator' 2025-04-21T21:42:51.7965674Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2025-04-21T21:42:51.7989055Z Entering 'third_party/XNNPACK' 2025-04-21T21:42:51.8052851Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2025-04-21T21:42:51.8095346Z Entering 'third_party/benchmark' 2025-04-21T21:42:51.8156009Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2025-04-21T21:42:51.8178136Z Entering 'third_party/composable_kernel' 2025-04-21T21:42:51.8240748Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/composable_kernel/config remote.origin.url 2025-04-21T21:42:51.8269693Z Entering 'third_party/cpp-httplib' 2025-04-21T21:42:51.8332497Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2025-04-21T21:42:51.8354710Z Entering 'third_party/cpuinfo' 2025-04-21T21:42:51.8416658Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2025-04-21T21:42:51.8439818Z Entering 'third_party/cudnn_frontend' 2025-04-21T21:42:51.8502673Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2025-04-21T21:42:51.8525354Z Entering 'third_party/cutlass' 2025-04-21T21:42:51.8587665Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2025-04-21T21:42:51.8618664Z Entering 'third_party/eigen' 2025-04-21T21:42:51.8681417Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2025-04-21T21:42:51.8706101Z Entering 'third_party/fbgemm' 2025-04-21T21:42:51.8769819Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2025-04-21T21:42:51.8791831Z Entering 'third_party/fbgemm/third_party/asmjit' 2025-04-21T21:42:51.8853623Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2025-04-21T21:42:51.8873914Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2025-04-21T21:42:51.8935788Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2025-04-21T21:42:51.8956077Z Entering 'third_party/fbgemm/third_party/cutlass' 2025-04-21T21:42:51.9018748Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2025-04-21T21:42:51.9046559Z Entering 'third_party/fbgemm/third_party/googletest' 2025-04-21T21:42:51.9108143Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2025-04-21T21:42:51.9128644Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2025-04-21T21:42:51.9189236Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2025-04-21T21:42:51.9212582Z Entering 'third_party/flash-attention' 2025-04-21T21:42:51.9277817Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/config remote.origin.url 2025-04-21T21:42:51.9299469Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-04-21T21:42:51.9366222Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/modules/csrc/composable_kernel/config remote.origin.url 2025-04-21T21:42:51.9394183Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-04-21T21:42:51.9456150Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/modules/csrc/cutlass/config remote.origin.url 2025-04-21T21:42:51.9489250Z Entering 'third_party/flatbuffers' 2025-04-21T21:42:51.9551416Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2025-04-21T21:42:51.9576175Z Entering 'third_party/fmt' 2025-04-21T21:42:51.9638115Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2025-04-21T21:42:51.9660348Z Entering 'third_party/gemmlowp/gemmlowp' 2025-04-21T21:42:51.9724732Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2025-04-21T21:42:51.9747196Z Entering 'third_party/gloo' 2025-04-21T21:42:51.9810301Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2025-04-21T21:42:51.9832928Z Entering 'third_party/googletest' 2025-04-21T21:42:51.9897022Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2025-04-21T21:42:51.9919475Z Entering 'third_party/ideep' 2025-04-21T21:42:51.9983468Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2025-04-21T21:42:52.0003333Z Entering 'third_party/ideep/mkl-dnn' 2025-04-21T21:42:52.0064193Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2025-04-21T21:42:52.0095404Z Entering 'third_party/ittapi' 2025-04-21T21:42:52.0158771Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2025-04-21T21:42:52.0181417Z Entering 'third_party/kineto' 2025-04-21T21:42:52.0245431Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2025-04-21T21:42:52.0266178Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-04-21T21:42:52.0329151Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2025-04-21T21:42:52.0350202Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-04-21T21:42:52.0413455Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2025-04-21T21:42:52.0436366Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-04-21T21:42:52.0498749Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2025-04-21T21:42:52.0520467Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-04-21T21:42:52.0583013Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2025-04-21T21:42:52.0604779Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-04-21T21:42:52.0668225Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2025-04-21T21:42:52.0687998Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-04-21T21:42:52.0751878Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2025-04-21T21:42:52.0777185Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-04-21T21:42:52.0840681Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2025-04-21T21:42:52.0862152Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-04-21T21:42:52.0928037Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2025-04-21T21:42:52.0954306Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-04-21T21:42:52.1017209Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2025-04-21T21:42:52.1039707Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-04-21T21:42:52.1103636Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2025-04-21T21:42:52.1128435Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-04-21T21:42:52.1190680Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2025-04-21T21:42:52.1211836Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-04-21T21:42:52.1273828Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2025-04-21T21:42:52.1301764Z Entering 'third_party/kleidiai' 2025-04-21T21:42:52.1364483Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kleidiai/config remote.origin.url 2025-04-21T21:42:52.1387731Z Entering 'third_party/mimalloc' 2025-04-21T21:42:52.1450075Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2025-04-21T21:42:52.1474078Z Entering 'third_party/nlohmann' 2025-04-21T21:42:52.1536223Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2025-04-21T21:42:52.1559984Z Entering 'third_party/onnx' 2025-04-21T21:42:52.1623609Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2025-04-21T21:42:52.1660058Z Entering 'third_party/onnx/third_party/pybind11' 2025-04-21T21:42:52.1727179Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2025-04-21T21:42:52.1754153Z Entering 'third_party/opentelemetry-cpp' 2025-04-21T21:42:52.1819192Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2025-04-21T21:42:52.1843097Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-04-21T21:42:52.1904841Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2025-04-21T21:42:52.1925804Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-04-21T21:42:52.1987600Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2025-04-21T21:42:52.2008664Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-04-21T21:42:52.2074797Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2025-04-21T21:42:52.2096081Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-04-21T21:42:52.2158423Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2025-04-21T21:42:52.2181419Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-04-21T21:42:52.2242737Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2025-04-21T21:42:52.2264233Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-04-21T21:42:52.2327327Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2025-04-21T21:42:52.2347550Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-04-21T21:42:52.2409491Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2025-04-21T21:42:52.2428818Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-04-21T21:42:52.2491837Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2025-04-21T21:42:52.2514514Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-04-21T21:42:52.2580447Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2025-04-21T21:42:52.2604951Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-04-21T21:42:52.2666675Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2025-04-21T21:42:52.2710980Z Entering 'third_party/pocketfft' 2025-04-21T21:42:52.2776038Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2025-04-21T21:42:52.2799247Z Entering 'third_party/protobuf' 2025-04-21T21:42:52.2861447Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2025-04-21T21:42:52.2886098Z Entering 'third_party/protobuf/third_party/benchmark' 2025-04-21T21:42:52.2948817Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2025-04-21T21:42:52.2969002Z Entering 'third_party/protobuf/third_party/googletest' 2025-04-21T21:42:52.3031324Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2025-04-21T21:42:52.3055687Z Entering 'third_party/psimd' 2025-04-21T21:42:52.3119393Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2025-04-21T21:42:52.3141467Z Entering 'third_party/pthreadpool' 2025-04-21T21:42:52.3207749Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2025-04-21T21:42:52.3229828Z Entering 'third_party/pybind11' 2025-04-21T21:42:52.3293632Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2025-04-21T21:42:52.3316803Z Entering 'third_party/python-peachpy' 2025-04-21T21:42:52.3379670Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2025-04-21T21:42:52.3401326Z Entering 'third_party/sleef' 2025-04-21T21:42:52.3463296Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2025-04-21T21:42:52.3486618Z Entering 'third_party/tensorpipe' 2025-04-21T21:42:52.3548797Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2025-04-21T21:42:52.3568149Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-04-21T21:42:52.3629665Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2025-04-21T21:42:52.3650398Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-04-21T21:42:52.3717943Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2025-04-21T21:42:52.3738533Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-04-21T21:42:52.3799848Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2025-04-21T21:42:52.3820677Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-04-21T21:42:52.3887278Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2025-04-21T21:42:52.3906707Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-04-21T21:42:52.3968973Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2025-04-21T21:42:52.5100478Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2025-04-21T21:42:52.5462876Z Entering 'android/libs/fbjni' 2025-04-21T21:42:52.5516185Z Entering 'third_party/FP16' 2025-04-21T21:42:52.5567329Z Entering 'third_party/FXdiv' 2025-04-21T21:42:52.5618446Z Entering 'third_party/NNPACK' 2025-04-21T21:42:52.5674383Z Entering 'third_party/NVTX' 2025-04-21T21:42:52.5725661Z Entering 'third_party/VulkanMemoryAllocator' 2025-04-21T21:42:52.5789938Z Entering 'third_party/XNNPACK' 2025-04-21T21:42:52.5858519Z Entering 'third_party/benchmark' 2025-04-21T21:42:52.5920139Z Entering 'third_party/composable_kernel' 2025-04-21T21:42:52.5980738Z Entering 'third_party/cpp-httplib' 2025-04-21T21:42:52.6032548Z Entering 'third_party/cpuinfo' 2025-04-21T21:42:52.6085253Z Entering 'third_party/cudnn_frontend' 2025-04-21T21:42:52.6136163Z Entering 'third_party/cutlass' 2025-04-21T21:42:52.6199750Z Entering 'third_party/eigen' 2025-04-21T21:42:52.6282372Z Entering 'third_party/fbgemm' 2025-04-21T21:42:52.6360898Z Entering 'third_party/fbgemm/third_party/asmjit' 2025-04-21T21:42:52.6416064Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2025-04-21T21:42:52.6467468Z Entering 'third_party/fbgemm/third_party/cutlass' 2025-04-21T21:42:52.6525437Z Entering 'third_party/fbgemm/third_party/googletest' 2025-04-21T21:42:52.6575549Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2025-04-21T21:42:52.6629417Z Entering 'third_party/flash-attention' 2025-04-21T21:42:52.6683439Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-04-21T21:42:52.6740973Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-04-21T21:42:52.6808004Z Entering 'third_party/flatbuffers' 2025-04-21T21:42:52.6863364Z Entering 'third_party/fmt' 2025-04-21T21:42:52.6917700Z Entering 'third_party/gemmlowp/gemmlowp' 2025-04-21T21:42:52.6971455Z Entering 'third_party/gloo' 2025-04-21T21:42:52.7025165Z Entering 'third_party/googletest' 2025-04-21T21:42:52.7078639Z Entering 'third_party/ideep' 2025-04-21T21:42:52.7130349Z Entering 'third_party/ideep/mkl-dnn' 2025-04-21T21:42:52.7191786Z Entering 'third_party/ittapi' 2025-04-21T21:42:52.7244722Z Entering 'third_party/kineto' 2025-04-21T21:42:52.7297426Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-04-21T21:42:52.7348041Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-04-21T21:42:52.7403392Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-04-21T21:42:52.7454376Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-04-21T21:42:52.7506178Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-04-21T21:42:52.7555794Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-04-21T21:42:52.7611385Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-04-21T21:42:52.7663107Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-04-21T21:42:52.7715350Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-04-21T21:42:52.7767451Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-04-21T21:42:52.7822190Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-04-21T21:42:52.7874263Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-04-21T21:42:52.7928683Z Entering 'third_party/kleidiai' 2025-04-21T21:42:52.7981968Z Entering 'third_party/mimalloc' 2025-04-21T21:42:52.8035029Z Entering 'third_party/nlohmann' 2025-04-21T21:42:52.8090363Z Entering 'third_party/onnx' 2025-04-21T21:42:52.8157757Z Entering 'third_party/onnx/third_party/pybind11' 2025-04-21T21:42:52.8215430Z Entering 'third_party/opentelemetry-cpp' 2025-04-21T21:42:52.8270182Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-04-21T21:42:52.8320886Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-04-21T21:42:52.8371417Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-04-21T21:42:52.8422968Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-04-21T21:42:52.8475227Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-04-21T21:42:52.8525950Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-04-21T21:42:52.8577415Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-04-21T21:42:52.8628001Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-04-21T21:42:52.8681146Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-04-21T21:42:52.8735133Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-04-21T21:42:52.8809092Z Entering 'third_party/pocketfft' 2025-04-21T21:42:52.8863260Z Entering 'third_party/protobuf' 2025-04-21T21:42:52.8918692Z Entering 'third_party/protobuf/third_party/benchmark' 2025-04-21T21:42:52.8968924Z Entering 'third_party/protobuf/third_party/googletest' 2025-04-21T21:42:52.9025547Z Entering 'third_party/psimd' 2025-04-21T21:42:52.9079594Z Entering 'third_party/pthreadpool' 2025-04-21T21:42:52.9131782Z Entering 'third_party/pybind11' 2025-04-21T21:42:52.9184964Z Entering 'third_party/python-peachpy' 2025-04-21T21:42:52.9237385Z Entering 'third_party/sleef' 2025-04-21T21:42:52.9290529Z Entering 'third_party/tensorpipe' 2025-04-21T21:42:52.9343354Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-04-21T21:42:52.9397976Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-04-21T21:42:52.9448896Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-04-21T21:42:52.9500457Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-04-21T21:42:52.9550254Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-04-21T21:42:52.9628797Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2025-04-21T21:42:52.9985720Z Entering 'android/libs/fbjni' 2025-04-21T21:42:53.0037851Z Entering 'third_party/FP16' 2025-04-21T21:42:53.0090763Z Entering 'third_party/FXdiv' 2025-04-21T21:42:53.0143003Z Entering 'third_party/NNPACK' 2025-04-21T21:42:53.0196406Z Entering 'third_party/NVTX' 2025-04-21T21:42:53.0249134Z Entering 'third_party/VulkanMemoryAllocator' 2025-04-21T21:42:53.0302219Z Entering 'third_party/XNNPACK' 2025-04-21T21:42:53.0369459Z Entering 'third_party/benchmark' 2025-04-21T21:42:53.0422881Z Entering 'third_party/composable_kernel' 2025-04-21T21:42:53.0481313Z Entering 'third_party/cpp-httplib' 2025-04-21T21:42:53.0533420Z Entering 'third_party/cpuinfo' 2025-04-21T21:42:53.0586361Z Entering 'third_party/cudnn_frontend' 2025-04-21T21:42:53.0638820Z Entering 'third_party/cutlass' 2025-04-21T21:42:53.0699636Z Entering 'third_party/eigen' 2025-04-21T21:42:53.0752731Z Entering 'third_party/fbgemm' 2025-04-21T21:42:53.0804947Z Entering 'third_party/fbgemm/third_party/asmjit' 2025-04-21T21:42:53.0856490Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2025-04-21T21:42:53.0909519Z Entering 'third_party/fbgemm/third_party/cutlass' 2025-04-21T21:42:53.0966909Z Entering 'third_party/fbgemm/third_party/googletest' 2025-04-21T21:42:53.1018094Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2025-04-21T21:42:53.1071382Z Entering 'third_party/flash-attention' 2025-04-21T21:42:53.1124116Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-04-21T21:42:53.1180210Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-04-21T21:42:53.1243986Z Entering 'third_party/flatbuffers' 2025-04-21T21:42:53.1300712Z Entering 'third_party/fmt' 2025-04-21T21:42:53.1353790Z Entering 'third_party/gemmlowp/gemmlowp' 2025-04-21T21:42:53.1407550Z Entering 'third_party/gloo' 2025-04-21T21:42:53.1459788Z Entering 'third_party/googletest' 2025-04-21T21:42:53.1512486Z Entering 'third_party/ideep' 2025-04-21T21:42:53.1564016Z Entering 'third_party/ideep/mkl-dnn' 2025-04-21T21:42:53.1624013Z Entering 'third_party/ittapi' 2025-04-21T21:42:53.1676776Z Entering 'third_party/kineto' 2025-04-21T21:42:53.1728648Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-04-21T21:42:53.1783496Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-04-21T21:42:53.1836569Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-04-21T21:42:53.1887911Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-04-21T21:42:53.1938953Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-04-21T21:42:53.1990148Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-04-21T21:42:53.2046846Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-04-21T21:42:53.2098533Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-04-21T21:42:53.2150454Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-04-21T21:42:53.2202995Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-04-21T21:42:53.2257246Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-04-21T21:42:53.2308678Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-04-21T21:42:53.2362571Z Entering 'third_party/kleidiai' 2025-04-21T21:42:53.2415511Z Entering 'third_party/mimalloc' 2025-04-21T21:42:53.2467562Z Entering 'third_party/nlohmann' 2025-04-21T21:42:53.2523286Z Entering 'third_party/onnx' 2025-04-21T21:42:53.2589333Z Entering 'third_party/onnx/third_party/pybind11' 2025-04-21T21:42:53.2646721Z Entering 'third_party/opentelemetry-cpp' 2025-04-21T21:42:53.2701895Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-04-21T21:42:53.2751956Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-04-21T21:42:53.2802796Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-04-21T21:42:53.2852647Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-04-21T21:42:53.2904769Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-04-21T21:42:53.2954796Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-04-21T21:42:53.3006199Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-04-21T21:42:53.3055337Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-04-21T21:42:53.3109389Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-04-21T21:42:53.3163013Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-04-21T21:42:53.3235870Z Entering 'third_party/pocketfft' 2025-04-21T21:42:53.3288494Z Entering 'third_party/protobuf' 2025-04-21T21:42:53.3343562Z Entering 'third_party/protobuf/third_party/benchmark' 2025-04-21T21:42:53.3396415Z Entering 'third_party/protobuf/third_party/googletest' 2025-04-21T21:42:53.3451143Z Entering 'third_party/psimd' 2025-04-21T21:42:53.3505008Z Entering 'third_party/pthreadpool' 2025-04-21T21:42:53.3556954Z Entering 'third_party/pybind11' 2025-04-21T21:42:53.3610389Z Entering 'third_party/python-peachpy' 2025-04-21T21:42:53.3661820Z Entering 'third_party/sleef' 2025-04-21T21:42:53.3716022Z Entering 'third_party/tensorpipe' 2025-04-21T21:42:53.3767825Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-04-21T21:42:53.3819050Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-04-21T21:42:53.3869109Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-04-21T21:42:53.3920557Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-04-21T21:42:53.3969265Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-04-21T21:42:53.4045612Z ##[endgroup] 2025-04-21T21:42:53.4096863Z [command]/usr/bin/git log -1 --format=%H 2025-04-21T21:42:53.4126235Z b68c0effe97351b0c24b2f7aef671559be6aa977 2025-04-21T21:42:53.4315448Z Prepare all required actions 2025-04-21T21:42:53.4315960Z Getting action download info 2025-04-21T21:42:53.5529760Z ##[group]Run ./.github/actions/setup-linux 2025-04-21T21:42:53.5530062Z env: 2025-04-21T21:42:53.5530265Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:42:53.5530503Z ##[endgroup] 2025-04-21T21:42:53.5574680Z ##[group]Run set -euo pipefail 2025-04-21T21:42:53.5575364Z set -euo pipefail 2025-04-21T21:42:53.5575649Z function get_ec2_metadata() { 2025-04-21T21:42:53.5575999Z  # Pulled from instance metadata endpoint for EC2 2025-04-21T21:42:53.5576590Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2025-04-21T21:42:53.5577111Z  category=$1 2025-04-21T21:42:53.5577451Z  # If it is GCP runner (runner name contains gcp), do not run this 2025-04-21T21:42:53.5577876Z  runner_name_str=i-09a65d907c0d198ba 2025-04-21T21:42:53.5578231Z  if [[ -f /.inarc ]]; then 2025-04-21T21:42:53.5578556Z  echo "ARC Runner, no info on ec2 metadata" 2025-04-21T21:42:53.5578928Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2025-04-21T21:42:53.5579371Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2025-04-21T21:42:53.5579770Z  else 2025-04-21T21:42:53.5580548Z  curl -H "X-aws-ec2-metadata-token: $(curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 30")" -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2025-04-21T21:42:53.5581374Z  fi 2025-04-21T21:42:53.5581578Z } 2025-04-21T21:42:53.5581824Z echo "ami-id: $(get_ec2_metadata ami-id)" 2025-04-21T21:42:53.5582220Z echo "instance-id: $(get_ec2_metadata instance-id)" 2025-04-21T21:42:53.5582675Z echo "instance-type: $(get_ec2_metadata instance-type)" 2025-04-21T21:42:53.5583070Z echo "system info $(uname -a)" 2025-04-21T21:42:53.5592079Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:42:53.5592428Z env: 2025-04-21T21:42:53.5592633Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:42:53.5592874Z ##[endgroup] 2025-04-21T21:42:53.5743868Z ami-id: ami-071226ecf16aa7d96 2025-04-21T21:42:53.5855967Z instance-id: i-09a65d907c0d198ba 2025-04-21T21:42:53.5969361Z instance-type: g5.4xlarge 2025-04-21T21:42:53.5982790Z system info Linux ip-10-0-6-215.ec2.internal 6.1.130-139.222.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Mar 11 01:10:58 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux 2025-04-21T21:42:53.6013563Z ##[group]Run echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-04-21T21:42:53.6014424Z echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-04-21T21:42:53.6023651Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:42:53.6024017Z env: 2025-04-21T21:42:53.6024230Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:42:53.6024483Z ##[endgroup] 2025-04-21T21:42:53.6089928Z ##[group]Run if systemctl is-active --quiet docker; then 2025-04-21T21:42:53.6090345Z if systemctl is-active --quiet docker; then 2025-04-21T21:42:53.6090721Z  echo "Docker daemon is running..."; 2025-04-21T21:42:53.6091027Z else 2025-04-21T21:42:53.6091367Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2025-04-21T21:42:53.6091755Z fi 2025-04-21T21:42:53.6099574Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:42:53.6099919Z env: 2025-04-21T21:42:53.6100125Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:42:53.6100369Z ##[endgroup] 2025-04-21T21:42:53.6184085Z Docker daemon is running... 2025-04-21T21:42:53.6232354Z ##[group]Run nick-fields/retry@v3.0.0 2025-04-21T21:42:53.6232650Z with: 2025-04-21T21:42:53.6232846Z shell: bash 2025-04-21T21:42:53.6248114Z timeout_minutes: 5 2025-04-21T21:42:53.6248375Z max_attempts: 3 2025-04-21T21:42:53.6248599Z retry_wait_seconds: 30 2025-04-21T21:42:53.6250701Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" # For LF Runners we need to make sure we also login to Meta's ECR docker registry too. META_AWS_ACCOUNT_ID=308535385114 if [ "$AWS_ACCOUNT_ID" != "$META_AWS_ACCOUNT_ID" ] ; then aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$META_AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" fi 2025-04-21T21:42:53.6252764Z polling_interval_seconds: 1 2025-04-21T21:42:53.6253035Z warning_on_retry: true 2025-04-21T21:42:53.6253294Z continue_on_error: false 2025-04-21T21:42:53.6253529Z env: 2025-04-21T21:42:53.6253736Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:42:53.6253992Z AWS_RETRY_MODE: standard 2025-04-21T21:42:53.6254231Z AWS_MAX_ATTEMPTS: 5 2025-04-21T21:42:53.6254476Z AWS_DEFAULT_REGION: us-east-1 2025-04-21T21:42:53.6254733Z ##[endgroup] 2025-04-21T21:42:54.7474805Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2025-04-21T21:42:54.7475544Z Configure a credential helper to remove this warning. See 2025-04-21T21:42:54.7476094Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2025-04-21T21:42:54.7476455Z 2025-04-21T21:42:54.7476558Z Login Succeeded 2025-04-21T21:42:55.7098138Z Command completed after 1 attempt(s). 2025-04-21T21:42:55.7166564Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2025-04-21T21:42:55.7167070Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2025-04-21T21:42:55.7167532Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2025-04-21T21:42:55.7177569Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:42:55.7177917Z env: 2025-04-21T21:42:55.7178133Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:42:55.7178383Z ##[endgroup] 2025-04-21T21:42:55.7270891Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2025-04-21T21:42:55.7271409Z # ignore expansion of "docker ps -q" since it could be empty 2025-04-21T21:42:55.7271823Z # shellcheck disable=SC2046 2025-04-21T21:42:55.7272174Z docker stop $(docker ps -q) || true 2025-04-21T21:42:55.7272513Z # Prune all of the docker images 2025-04-21T21:42:55.7272827Z docker system prune -af 2025-04-21T21:42:55.7281025Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:42:55.7281382Z env: 2025-04-21T21:42:55.7281594Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:42:55.7281837Z ##[endgroup] 2025-04-21T21:42:55.7571668Z "docker stop" requires at least 1 argument. 2025-04-21T21:42:55.7572026Z See 'docker stop --help'. 2025-04-21T21:42:55.7572197Z 2025-04-21T21:42:55.7572370Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2025-04-21T21:42:55.7572617Z 2025-04-21T21:42:55.7572728Z Stop one or more running containers 2025-04-21T21:42:55.7743252Z Total reclaimed space: 0B 2025-04-21T21:42:55.7793727Z ##[group]Run set +e 2025-04-21T21:42:55.7794014Z set +e 2025-04-21T21:42:55.7794242Z set -x 2025-04-21T21:42:55.7794457Z  2025-04-21T21:42:55.7794691Z PT_DOMAIN=download.pytorch.org 2025-04-21T21:42:55.7795233Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2025-04-21T21:42:55.7795938Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2025-04-21T21:42:55.7796437Z # one is returned at random 2025-04-21T21:42:55.7796805Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2025-04-21T21:42:55.7797172Z  2025-04-21T21:42:55.7797608Z if [ -z "${RESOLVED_IP}" ]; then 2025-04-21T21:42:55.7798015Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2025-04-21T21:42:55.7798499Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2025-04-21T21:42:55.7798866Z  2025-04-21T21:42:55.7799094Z  if [ -z "${RESOLVED_IP}" ]; then 2025-04-21T21:42:55.7799451Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2025-04-21T21:42:55.7799796Z  exit 1 2025-04-21T21:42:55.7800019Z  fi 2025-04-21T21:42:55.7800216Z fi 2025-04-21T21:42:55.7800415Z  2025-04-21T21:42:55.7800662Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2025-04-21T21:42:55.7801000Z  # Clean up any old records first 2025-04-21T21:42:55.7801344Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2025-04-21T21:42:55.7801656Z fi 2025-04-21T21:42:55.7801856Z  2025-04-21T21:42:55.7802160Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2025-04-21T21:42:55.7802535Z cat /etc/hosts 2025-04-21T21:42:55.7811156Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:42:55.7811503Z env: 2025-04-21T21:42:55.7811713Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:42:55.7811959Z ##[endgroup] 2025-04-21T21:42:55.7838929Z + PT_DOMAIN=download.pytorch.org 2025-04-21T21:42:55.7845841Z ++ dig -4 +short download.pytorch.org 2025-04-21T21:42:55.7846757Z ++ tail -n1 2025-04-21T21:42:55.8145889Z + RESOLVED_IP=18.160.10.28 2025-04-21T21:42:55.8146160Z + '[' -z 18.160.10.28 ']' 2025-04-21T21:42:55.8146438Z + grep -r download.pytorch.org /etc/hosts 2025-04-21T21:42:55.8159583Z 18.160.10.36 download.pytorch.org 2025-04-21T21:42:55.8161545Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2025-04-21T21:42:55.9376553Z + echo '18.160.10.28 download.pytorch.org' 2025-04-21T21:42:55.9377010Z + sudo tee -a /etc/hosts 2025-04-21T21:42:56.1240994Z 18.160.10.28 download.pytorch.org 2025-04-21T21:42:56.1264763Z + cat /etc/hosts 2025-04-21T21:42:56.1276088Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2025-04-21T21:42:56.1282278Z ::1 localhost6 localhost6.localdomain6 2025-04-21T21:42:56.1282628Z 18.160.10.28 download.pytorch.org 2025-04-21T21:42:56.1450697Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2025-04-21T21:42:56.1451136Z with: 2025-04-21T21:42:56.1451968Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.6-cudnn9-py3-gcc9-inductor-benchmarks:f583c1656c56372ad0e1c57f3463a017fbf03138 2025-04-21T21:42:56.1452839Z docker-build-dir: .ci/docker 2025-04-21T21:42:56.1453128Z docker-build-script: ./build.sh 2025-04-21T21:42:56.1453414Z working-directory: . 2025-04-21T21:42:56.1453744Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-04-21T21:42:56.1454289Z force-push: false 2025-04-21T21:42:56.1454506Z env: 2025-04-21T21:42:56.1454704Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:42:56.1454955Z ##[endgroup] 2025-04-21T21:42:56.1483483Z ##[group]Run set -ex 2025-04-21T21:42:56.1483753Z set -ex 2025-04-21T21:42:56.1483962Z  2025-04-21T21:42:56.1484369Z # If the docker build directory or the build script doesn't exist, the action will 2025-04-21T21:42:56.1484993Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2025-04-21T21:42:56.1485526Z # job could then download the pre-built image as usual 2025-04-21T21:42:56.1486055Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/${DOCKER_BUILD_SCRIPT}" ]]; then 2025-04-21T21:42:56.1486538Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2025-04-21T21:42:56.1486959Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2025-04-21T21:42:56.1487339Z  2025-04-21T21:42:56.1487682Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2025-04-21T21:42:56.1488098Z  exit 0 2025-04-21T21:42:56.1488310Z else 2025-04-21T21:42:56.1488565Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2025-04-21T21:42:56.1488871Z fi 2025-04-21T21:42:56.1489068Z  2025-04-21T21:42:56.1489392Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2025-04-21T21:42:56.1489948Z  # The docker image name already includes the ECR prefix and tag, so we can just 2025-04-21T21:42:56.1490445Z  # use it as it is, but first let's extract the tag 2025-04-21T21:42:56.1490893Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2025-04-21T21:42:56.1491375Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2025-04-21T21:42:56.1491837Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2025-04-21T21:42:56.1492215Z else 2025-04-21T21:42:56.1492640Z  DOCKER_TAG=${CUSTOM_TAG_PREFIX:+${CUSTOM_TAG_PREFIX}-}$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2025-04-21T21:42:56.1493208Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2025-04-21T21:42:56.1493803Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2025-04-21T21:42:56.1494316Z fi 2025-04-21T21:42:56.1504138Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:42:56.1504488Z env: 2025-04-21T21:42:56.1504692Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:42:56.1504945Z REPO_NAME: pytorch 2025-04-21T21:42:56.1505840Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.6-cudnn9-py3-gcc9-inductor-benchmarks:f583c1656c56372ad0e1c57f3463a017fbf03138 2025-04-21T21:42:56.1506673Z DOCKER_BUILD_DIR: .ci/docker 2025-04-21T21:42:56.1506941Z DOCKER_BUILD_SCRIPT: ./build.sh 2025-04-21T21:42:56.1507310Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-04-21T21:42:56.1507675Z CUSTOM_TAG_PREFIX: 2025-04-21T21:42:56.1507897Z ##[endgroup] 2025-04-21T21:42:56.1534747Z + [[ ! -d .ci/docker ]] 2025-04-21T21:42:56.1535153Z + [[ ! -f .ci/docker/./build.sh ]] 2025-04-21T21:42:56.1535507Z + echo skip=false 2025-04-21T21:42:56.1536846Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.6-cudnn9-py3-gcc9-inductor-benchmarks:f583c1656c56372ad0e1c57f3463a017fbf03138 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2025-04-21T21:42:56.1543559Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.6-cudnn9-py3-gcc9-inductor-benchmarks:f583c1656c56372ad0e1c57f3463a017fbf03138 2025-04-21T21:42:56.1544431Z ++ awk -F '[:,]' '{print $2}' 2025-04-21T21:42:56.1567881Z + DOCKER_TAG=f583c1656c56372ad0e1c57f3463a017fbf03138 2025-04-21T21:42:56.1568425Z + echo docker-tag=f583c1656c56372ad0e1c57f3463a017fbf03138 2025-04-21T21:42:56.1569685Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.6-cudnn9-py3-gcc9-inductor-benchmarks:f583c1656c56372ad0e1c57f3463a017fbf03138 2025-04-21T21:42:56.1603809Z ##[group]Run set +e 2025-04-21T21:42:56.1604085Z set +e 2025-04-21T21:42:56.1604313Z set -x 2025-04-21T21:42:56.1604522Z  2025-04-21T21:42:56.1604718Z login() { 2025-04-21T21:42:56.1605161Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2025-04-21T21:42:56.1605641Z } 2025-04-21T21:42:56.1605834Z  2025-04-21T21:42:56.1606033Z retry () { 2025-04-21T21:42:56.1606291Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2025-04-21T21:42:56.1606580Z } 2025-04-21T21:42:56.1606776Z  2025-04-21T21:42:56.1606998Z retry login "${DOCKER_REGISTRY}" 2025-04-21T21:42:56.1607295Z  2025-04-21T21:42:56.1607500Z START_TIME=$(date +%s) 2025-04-21T21:42:56.1607779Z # Wait up to 120 minutes 2025-04-21T21:42:56.1608120Z while [[ $(( $(date +%s) - 7200 )) -lt $START_TIME ]]; do 2025-04-21T21:42:56.1608570Z  # Check if image already exists, if it does then skip building it 2025-04-21T21:42:56.1609033Z  if docker manifest inspect "${DOCKER_IMAGE}"; then 2025-04-21T21:42:56.1609373Z  exit 0 2025-04-21T21:42:56.1609586Z  fi 2025-04-21T21:42:56.1609791Z  2025-04-21T21:42:56.1610154Z  # NB: This flag is used by Docker build workflow to push the image to ECR, so we can 2025-04-21T21:42:56.1610768Z  # use this to differentiate between the Docker build and regular build jobs. For the 2025-04-21T21:42:56.1611368Z  # latter, it will wait for the Docker images to become available before continuing 2025-04-21T21:42:56.1611852Z  if [ "${DOCKER_PUSH:-false}" == "true" ]; then 2025-04-21T21:42:56.1612239Z  # It's a Docker build job, let's build the image 2025-04-21T21:42:56.1612563Z  break 2025-04-21T21:42:56.1612779Z  else 2025-04-21T21:42:56.1613099Z  # It's a regular build job, wait for the image to become available 2025-04-21T21:42:56.1613481Z  sleep 300 2025-04-21T21:42:56.1613712Z  fi 2025-04-21T21:42:56.1613923Z done 2025-04-21T21:42:56.1614130Z  2025-04-21T21:42:56.1614456Z # NB: This part requires a full checkout. Otherwise, the merge base will 2025-04-21T21:42:56.1614988Z # be empty. The default action would be to continue rebuild the image 2025-04-21T21:42:56.1615462Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2025-04-21T21:42:56.1615892Z  # if we're on the base branch then use the parent commit 2025-04-21T21:42:56.1616263Z  MERGE_BASE=$(git rev-parse HEAD~) 2025-04-21T21:42:56.1616562Z else 2025-04-21T21:42:56.1616865Z  # otherwise we're on a PR, so use the most recent base commit 2025-04-21T21:42:56.1617301Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2025-04-21T21:42:56.1617642Z fi 2025-04-21T21:42:56.1617842Z  2025-04-21T21:42:56.1618062Z if [[ -z "${MERGE_BASE}" ]]; then 2025-04-21T21:42:56.1618551Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2025-04-21T21:42:56.1618868Z  2025-04-21T21:42:56.1619310Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2025-04-21T21:42:56.1619803Z  exit 0 2025-04-21T21:42:56.1620014Z fi 2025-04-21T21:42:56.1620213Z  2025-04-21T21:42:56.1620496Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2025-04-21T21:42:56.1621122Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2025-04-21T21:42:56.1621822Z  exit 1 2025-04-21T21:42:56.1622038Z fi 2025-04-21T21:42:56.1622232Z  2025-04-21T21:42:56.1622566Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2025-04-21T21:42:56.1623188Z # If no image exists but the hash is the same as the previous hash then we should error out here 2025-04-21T21:42:56.1623739Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2025-04-21T21:42:56.1624355Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2025-04-21T21:42:56.1625051Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2025-04-21T21:42:56.1625626Z fi 2025-04-21T21:42:56.1625820Z  2025-04-21T21:42:56.1626069Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2025-04-21T21:42:56.1634277Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:42:56.1634634Z env: 2025-04-21T21:42:56.1634833Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:42:56.1635113Z DOCKER_BUILD_DIR: .ci/docker 2025-04-21T21:42:56.1635441Z BASE_REVISION: b68c0effe97351b0c24b2f7aef671559be6aa977 2025-04-21T21:42:56.1636322Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.6-cudnn9-py3-gcc9-inductor-benchmarks:f583c1656c56372ad0e1c57f3463a017fbf03138 2025-04-21T21:42:56.1637185Z DOCKER_TAG: f583c1656c56372ad0e1c57f3463a017fbf03138 2025-04-21T21:42:56.1637587Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-04-21T21:42:56.1637943Z DOCKER_PUSH: 2025-04-21T21:42:56.1638156Z ##[endgroup] 2025-04-21T21:42:56.1664297Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-04-21T21:42:56.1664733Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-04-21T21:42:56.1667149Z + aws ecr get-login-password --region us-east-1 2025-04-21T21:42:56.1669040Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-04-21T21:42:56.6817589Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2025-04-21T21:42:56.6818143Z Configure a credential helper to remove this warning. See 2025-04-21T21:42:56.6818673Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2025-04-21T21:42:56.6819041Z 2025-04-21T21:42:56.6819161Z Login Succeeded 2025-04-21T21:42:56.6844788Z ++ date +%s 2025-04-21T21:42:56.6856243Z + START_TIME=1745271776 2025-04-21T21:42:56.6860592Z ++ date +%s 2025-04-21T21:42:56.6872113Z + [[ 1745264576 -lt 1745271776 ]] 2025-04-21T21:42:56.6873009Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.6-cudnn9-py3-gcc9-inductor-benchmarks:f583c1656c56372ad0e1c57f3463a017fbf03138 2025-04-21T21:42:56.9342042Z { 2025-04-21T21:42:56.9342690Z "schemaVersion": 2, 2025-04-21T21:42:56.9343253Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2025-04-21T21:42:56.9343695Z "config": { 2025-04-21T21:42:56.9344033Z "mediaType": "application/vnd.docker.container.image.v1+json", 2025-04-21T21:42:56.9344398Z "size": 46736, 2025-04-21T21:42:56.9344800Z "digest": "sha256:7b29eacbf93f5bdd9e6b666f27edb34c0d2f289aa58ceaf3bdfcf4e06f505e55" 2025-04-21T21:42:56.9345569Z }, 2025-04-21T21:42:56.9345823Z "layers": [ 2025-04-21T21:42:56.9346291Z { 2025-04-21T21:42:56.9346733Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9347277Z "size": 28583948, 2025-04-21T21:42:56.9347823Z "digest": "sha256:86e5016c269355b382c9cabab4f6646d56d75914f20d545289970436dae431b1" 2025-04-21T21:42:56.9348488Z }, 2025-04-21T21:42:56.9348754Z { 2025-04-21T21:42:56.9349197Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9349612Z "size": 7964619, 2025-04-21T21:42:56.9349990Z "digest": "sha256:49e139a3d6c2f1801aa0cea1eb34e57c5314065b679325df205026fb175383b8" 2025-04-21T21:42:56.9350602Z }, 2025-04-21T21:42:56.9350779Z { 2025-04-21T21:42:56.9351086Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9351549Z "size": 57379226, 2025-04-21T21:42:56.9352105Z "digest": "sha256:a14844c8c51f98ebee2b5eda8bff8742dd804385e1dbbeb56928946113000293" 2025-04-21T21:42:56.9352651Z }, 2025-04-21T21:42:56.9352899Z { 2025-04-21T21:42:56.9366008Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9366424Z "size": 187, 2025-04-21T21:42:56.9366825Z "digest": "sha256:18fb524087fbed3307d23afde734ef1df452f55e7a22e28b56ce1e6eb9c6b3d9" 2025-04-21T21:42:56.9367261Z }, 2025-04-21T21:42:56.9367445Z { 2025-04-21T21:42:56.9367763Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9368143Z "size": 6885, 2025-04-21T21:42:56.9368540Z "digest": "sha256:efd686a7b2c8bb052a3bb919973c4bee33c2bbcff08fd431362b90c9469fbc4d" 2025-04-21T21:42:56.9368988Z }, 2025-04-21T21:42:56.9369176Z { 2025-04-21T21:42:56.9369490Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9369875Z "size": 1438282733, 2025-04-21T21:42:56.9370268Z "digest": "sha256:52c648e213347a29001a45b2c4f834cc60621a3d6ab68fa06ebd2075f6487ec5" 2025-04-21T21:42:56.9370686Z }, 2025-04-21T21:42:56.9370898Z { 2025-04-21T21:42:56.9371411Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9371891Z "size": 62694, 2025-04-21T21:42:56.9372484Z "digest": "sha256:56e384e4e5aa7ebddf31686fc6ae62ef19fdc5311bbea8186563fc2aeca2fb52" 2025-04-21T21:42:56.9373110Z }, 2025-04-21T21:42:56.9373287Z { 2025-04-21T21:42:56.9373596Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9373979Z "size": 1684, 2025-04-21T21:42:56.9374357Z "digest": "sha256:fb71b792ec6c458c338cace5ce42f653e4ae4b24442a49b7c47af3a38342b5bc" 2025-04-21T21:42:56.9374793Z }, 2025-04-21T21:42:56.9375307Z { 2025-04-21T21:42:56.9375634Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9376011Z "size": 1523, 2025-04-21T21:42:56.9376387Z "digest": "sha256:5509576f2693abf94a50a70a59bbf8b519f20705f147d495ddde85981c8189fa" 2025-04-21T21:42:56.9376809Z }, 2025-04-21T21:42:56.9376983Z { 2025-04-21T21:42:56.9377299Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9377684Z "size": 2515216700, 2025-04-21T21:42:56.9378071Z "digest": "sha256:1e6c6f2d245956001435a5df27fd4defea08affa7f6c22252464f758447db401" 2025-04-21T21:42:56.9378498Z }, 2025-04-21T21:42:56.9378678Z { 2025-04-21T21:42:56.9378975Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9379358Z "size": 86619, 2025-04-21T21:42:56.9379733Z "digest": "sha256:a1fe8922734d371e0c0fd5b04183758331e08f45553720ecff4b062824e0c8e1" 2025-04-21T21:42:56.9380153Z }, 2025-04-21T21:42:56.9380331Z { 2025-04-21T21:42:56.9380635Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9381022Z "size": 1893, 2025-04-21T21:42:56.9381389Z "digest": "sha256:84e5433913c60a912877721ebf3a4d115b8bd55fb1c5a298b58fdc1f3ef945b1" 2025-04-21T21:42:56.9381814Z }, 2025-04-21T21:42:56.9381994Z { 2025-04-21T21:42:56.9382236Z + exit 0 2025-04-21T21:42:56.9382796Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9383193Z "size": 248991124, 2025-04-21T21:42:56.9383579Z "digest": "sha256:c68e78d48b95f6272003f9de9b80c623c4b361ed6e860801cbe1bd2db4d97263" 2025-04-21T21:42:56.9384009Z }, 2025-04-21T21:42:56.9384196Z { 2025-04-21T21:42:56.9384496Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9384881Z "size": 703, 2025-04-21T21:42:56.9385370Z "digest": "sha256:b14b320c459cd13b3ef4a06575f1b29513d8b0e6c1672ea68ae5b34a977cd975" 2025-04-21T21:42:56.9385801Z }, 2025-04-21T21:42:56.9385968Z { 2025-04-21T21:42:56.9386271Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9386828Z "size": 1262, 2025-04-21T21:42:56.9387199Z "digest": "sha256:0d929bfaefee4bb307c42773c6f04675af8b94398a51b1c8575447099bbf8607" 2025-04-21T21:42:56.9387627Z }, 2025-04-21T21:42:56.9387802Z { 2025-04-21T21:42:56.9388099Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9388488Z "size": 484, 2025-04-21T21:42:56.9388876Z "digest": "sha256:a5a8ea5a3eabb6655fad5c54e28c86000dca6ac1c3a5a485c718f87b60e28711" 2025-04-21T21:42:56.9389309Z }, 2025-04-21T21:42:56.9389492Z { 2025-04-21T21:42:56.9389885Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9390270Z "size": 91731730, 2025-04-21T21:42:56.9390645Z "digest": "sha256:281f13ef6929e0c7c8259f29b3dc71103ec70018f2711ec9f8a8f04093099171" 2025-04-21T21:42:56.9391063Z }, 2025-04-21T21:42:56.9391236Z { 2025-04-21T21:42:56.9391529Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9391933Z "size": 3767, 2025-04-21T21:42:56.9392339Z "digest": "sha256:a8695f1c94c34953b85e2090d713494e7fc84adc148495937b7a7c3deed10275" 2025-04-21T21:42:56.9392748Z }, 2025-04-21T21:42:56.9392927Z { 2025-04-21T21:42:56.9393231Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9393600Z "size": 1868, 2025-04-21T21:42:56.9393988Z "digest": "sha256:c72f323be8018695bb0b260acf986dd615ac7ce067ad92b0eb1ac2e72925a39a" 2025-04-21T21:42:56.9394418Z }, 2025-04-21T21:42:56.9394586Z { 2025-04-21T21:42:56.9394885Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9395261Z "size": 701, 2025-04-21T21:42:56.9395638Z "digest": "sha256:4afa91cd0662787b5dfd97c4724a659e2b0dad7e1c058307ee96067d064b7e03" 2025-04-21T21:42:56.9396059Z }, 2025-04-21T21:42:56.9396236Z { 2025-04-21T21:42:56.9396536Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9396901Z "size": 477, 2025-04-21T21:42:56.9397296Z "digest": "sha256:a9bdca78eb50cb709cb6b39f9f2bcd879bc2bb990b944eca5bc0b41d8ac75114" 2025-04-21T21:42:56.9397735Z }, 2025-04-21T21:42:56.9397902Z { 2025-04-21T21:42:56.9398203Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9398580Z "size": 3018885224, 2025-04-21T21:42:56.9398959Z "digest": "sha256:87b13320574046b541e9fdaa0186880e7082d07d823639a45f77e9da1e325fda" 2025-04-21T21:42:56.9399375Z }, 2025-04-21T21:42:56.9399557Z { 2025-04-21T21:42:56.9399855Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9400232Z "size": 380, 2025-04-21T21:42:56.9400605Z "digest": "sha256:1a1f3a6b6e2805950beb3918166ea519e8237d7b6493d050bce0ecb65b3750e6" 2025-04-21T21:42:56.9401027Z }, 2025-04-21T21:42:56.9401192Z { 2025-04-21T21:42:56.9401492Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9401870Z "size": 12918, 2025-04-21T21:42:56.9402239Z "digest": "sha256:1e726e5b69dff0c7b1285406bf76144f96e624de579ac59e252da98586616399" 2025-04-21T21:42:56.9402713Z }, 2025-04-21T21:42:56.9402887Z { 2025-04-21T21:42:56.9403180Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9403563Z "size": 791, 2025-04-21T21:42:56.9404056Z "digest": "sha256:e55ee61cc4aaf6da32e72efeef162eda299bb6a632e8d8dcb487521c151204a0" 2025-04-21T21:42:56.9404514Z }, 2025-04-21T21:42:56.9404764Z { 2025-04-21T21:42:56.9405179Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9405696Z "size": 106, 2025-04-21T21:42:56.9406108Z "digest": "sha256:d2c9e5f6d068cc6a329295a862e67a4110accba3b0c35efaab96121dc83e2c79" 2025-04-21T21:42:56.9406533Z }, 2025-04-21T21:42:56.9406713Z { 2025-04-21T21:42:56.9407010Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9407390Z "size": 503, 2025-04-21T21:42:56.9407773Z "digest": "sha256:a9e23e3c0a187a40d8af9500ce12468894283cc71e0ba2fb1fcb6624dc40ddbf" 2025-04-21T21:42:56.9408335Z }, 2025-04-21T21:42:56.9408522Z { 2025-04-21T21:42:56.9408877Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9409267Z "size": 121477387, 2025-04-21T21:42:56.9409655Z "digest": "sha256:956077b157237c043dfffd54a6aee7e4f7d3e03584e329d427ab36bcf285b595" 2025-04-21T21:42:56.9410082Z }, 2025-04-21T21:42:56.9410259Z { 2025-04-21T21:42:56.9410562Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9410930Z "size": 109, 2025-04-21T21:42:56.9411384Z "digest": "sha256:b840d58dab14ffab9220e7e5dba7751d9809c049d77a61b93924e4947527bb65" 2025-04-21T21:42:56.9411857Z }, 2025-04-21T21:42:56.9412030Z { 2025-04-21T21:42:56.9412396Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9412806Z "size": 1472, 2025-04-21T21:42:56.9413198Z "digest": "sha256:135c1f91f3df5257432cc2d37486ab3cf2ff585d3a2ec4ac2304cb0d25e5fac1" 2025-04-21T21:42:56.9413635Z }, 2025-04-21T21:42:56.9413812Z { 2025-04-21T21:42:56.9414105Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9414487Z "size": 429635236, 2025-04-21T21:42:56.9414874Z "digest": "sha256:2fba85b4232347228dbfdef797160ace15ad71face213ee558c1180161dfe7c6" 2025-04-21T21:42:56.9415301Z }, 2025-04-21T21:42:56.9415476Z { 2025-04-21T21:42:56.9415775Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9416154Z "size": 162, 2025-04-21T21:42:56.9416526Z "digest": "sha256:7e377528c596bece391a67c4dfc9bab633557a7bd77e7a650595f6f4ef162b1a" 2025-04-21T21:42:56.9416954Z }, 2025-04-21T21:42:56.9417128Z { 2025-04-21T21:42:56.9417423Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9417804Z "size": 801, 2025-04-21T21:42:56.9418186Z "digest": "sha256:7c1fe59ec87a4d7e8d2d9c14b3aa7254a4dbeb2566473fceaa3d6ef086a29e5a" 2025-04-21T21:42:56.9418616Z }, 2025-04-21T21:42:56.9418794Z { 2025-04-21T21:42:56.9419093Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9419468Z "size": 35860969, 2025-04-21T21:42:56.9419860Z "digest": "sha256:bb83f375ed7d52c4af28b418a77af53bdb24511aad990c9e8f017c3b6145d401" 2025-04-21T21:42:56.9420293Z }, 2025-04-21T21:42:56.9420466Z { 2025-04-21T21:42:56.9420766Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9421143Z "size": 104, 2025-04-21T21:42:56.9421513Z "digest": "sha256:d1ee3dcc43c51d791d948668680e601c3394068ce014a7395b2565facd8e1554" 2025-04-21T21:42:56.9421931Z }, 2025-04-21T21:42:56.9422110Z { 2025-04-21T21:42:56.9422413Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9422784Z "size": 424, 2025-04-21T21:42:56.9423150Z "digest": "sha256:54687ddf5aab346564b45b9340d55679e17969e1a9290a068a71acd75e445a68" 2025-04-21T21:42:56.9423571Z }, 2025-04-21T21:42:56.9423741Z { 2025-04-21T21:42:56.9424047Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9424423Z "size": 20262168, 2025-04-21T21:42:56.9424806Z "digest": "sha256:6b673e167d0fd859f7f81837936c87feb0f3540e9d83ccb685e01c2d2b1a105c" 2025-04-21T21:42:56.9425334Z }, 2025-04-21T21:42:56.9425516Z { 2025-04-21T21:42:56.9425924Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9426297Z "size": 643, 2025-04-21T21:42:56.9426661Z "digest": "sha256:a4f475d6d4d727b2776f8de8fb347830ba056f3485b53102d7502bf63a1c08b4" 2025-04-21T21:42:56.9427079Z }, 2025-04-21T21:42:56.9427247Z { 2025-04-21T21:42:56.9427543Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9427920Z "size": 701, 2025-04-21T21:42:56.9428283Z "digest": "sha256:4afa91cd0662787b5dfd97c4724a659e2b0dad7e1c058307ee96067d064b7e03" 2025-04-21T21:42:56.9428707Z }, 2025-04-21T21:42:56.9428879Z { 2025-04-21T21:42:56.9429175Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9429627Z "size": 141, 2025-04-21T21:42:56.9429991Z "digest": "sha256:48e60e8ea738df82c6a874ed4b7e587451261d7675d166a68e4b882ce5a932e7" 2025-04-21T21:42:56.9430414Z }, 2025-04-21T21:42:56.9430580Z { 2025-04-21T21:42:56.9430877Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9431262Z "size": 134, 2025-04-21T21:42:56.9431628Z "digest": "sha256:d52c79949221ec29bacb5c73e1630c76a77c93a6dcd0a36570ef858cb7ff2491" 2025-04-21T21:42:56.9432061Z }, 2025-04-21T21:42:56.9432237Z { 2025-04-21T21:42:56.9432531Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9432911Z "size": 5221737836, 2025-04-21T21:42:56.9433288Z "digest": "sha256:846d8aa2652360437687998e7786934ac4a2680471da233060f0e42acca36db8" 2025-04-21T21:42:56.9433693Z }, 2025-04-21T21:42:56.9433870Z { 2025-04-21T21:42:56.9434171Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9434548Z "size": 194, 2025-04-21T21:42:56.9434921Z "digest": "sha256:595e9954171617ada6037b7869515bf0e2d1dff54b868e3ffba1f652ef8c56f5" 2025-04-21T21:42:56.9435341Z }, 2025-04-21T21:42:56.9435517Z { 2025-04-21T21:42:56.9435814Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9436193Z "size": 563, 2025-04-21T21:42:56.9436575Z "digest": "sha256:013dfd09e68f59d76a79ddcc8b47d99f10ad62529abcd1725885de25818509ae" 2025-04-21T21:42:56.9436996Z }, 2025-04-21T21:42:56.9437169Z { 2025-04-21T21:42:56.9437467Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9437838Z "size": 43164291, 2025-04-21T21:42:56.9438231Z "digest": "sha256:4402edece585e892e3dcd04507c4e9315cd4537bacd16ed748aebfec5e8af5e8" 2025-04-21T21:42:56.9438662Z }, 2025-04-21T21:42:56.9438833Z { 2025-04-21T21:42:56.9439139Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9439515Z "size": 106, 2025-04-21T21:42:56.9439896Z "digest": "sha256:82aa68866e535166ba9028ae8d5fe7bf3d13ea4c69a43469aaefdffccc00b8a9" 2025-04-21T21:42:56.9440320Z }, 2025-04-21T21:42:56.9440489Z { 2025-04-21T21:42:56.9440787Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9441159Z "size": 307403395, 2025-04-21T21:42:56.9441542Z "digest": "sha256:e624d917de213d9b06921538cf6d41f55585fc181f15c727920ca878e0d432c1" 2025-04-21T21:42:56.9441959Z }, 2025-04-21T21:42:56.9442123Z { 2025-04-21T21:42:56.9442424Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9442806Z "size": 5761507135, 2025-04-21T21:42:56.9443192Z "digest": "sha256:79d2c9f506eccd71c934a8639db0b5fb46a4c03b1e4c68535c972001a0b3a1f6" 2025-04-21T21:42:56.9443615Z }, 2025-04-21T21:42:56.9443797Z { 2025-04-21T21:42:56.9444096Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9444474Z "size": 127, 2025-04-21T21:42:56.9444848Z "digest": "sha256:3b25d713acb3675cc21cfaf3e73036b34d21dd039d0339ebd5874e41f29d4c73" 2025-04-21T21:42:56.9445284Z }, 2025-04-21T21:42:56.9445453Z { 2025-04-21T21:42:56.9445757Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9446154Z "size": 1012, 2025-04-21T21:42:56.9446637Z "digest": "sha256:c0ad696e18d34210a240d661b9a6994f1fcf548fb2eeaeccef011fd9fbb01d79" 2025-04-21T21:42:56.9447074Z }, 2025-04-21T21:42:56.9447255Z { 2025-04-21T21:42:56.9447552Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9447934Z "size": 701, 2025-04-21T21:42:56.9448315Z "digest": "sha256:4afa91cd0662787b5dfd97c4724a659e2b0dad7e1c058307ee96067d064b7e03" 2025-04-21T21:42:56.9448738Z }, 2025-04-21T21:42:56.9448912Z { 2025-04-21T21:42:56.9449214Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9449591Z "size": 135, 2025-04-21T21:42:56.9449951Z "digest": "sha256:767566b693177188e1a3b60bd320a787e121ffd7dd9f0cd79f75f94f17d2197b" 2025-04-21T21:42:56.9450479Z }, 2025-04-21T21:42:56.9450656Z { 2025-04-21T21:42:56.9450948Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9451324Z "size": 32, 2025-04-21T21:42:56.9451704Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-04-21T21:42:56.9452130Z }, 2025-04-21T21:42:56.9452311Z { 2025-04-21T21:42:56.9452612Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9452980Z "size": 159, 2025-04-21T21:42:56.9453352Z "digest": "sha256:d83ab53d6ab47f0836a1d834f483ea68f6bb74717f9b5f435958be607a1e25de" 2025-04-21T21:42:56.9453776Z }, 2025-04-21T21:42:56.9453947Z { 2025-04-21T21:42:56.9454249Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9454626Z "size": 1898, 2025-04-21T21:42:56.9455018Z "digest": "sha256:e9ae24bb3b5b9eaace2576d0bc4ce0ccdea8f446743644ae2f5d99fa4d2704ed" 2025-04-21T21:42:56.9455457Z }, 2025-04-21T21:42:56.9455633Z { 2025-04-21T21:42:56.9455932Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9456302Z "size": 237571532, 2025-04-21T21:42:56.9456685Z "digest": "sha256:8c11aa345fff3b36368b1fc7fc8268149038691d5e7a11054e8c029d0b9a6675" 2025-04-21T21:42:56.9457111Z }, 2025-04-21T21:42:56.9457285Z { 2025-04-21T21:42:56.9457585Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9457963Z "size": 162, 2025-04-21T21:42:56.9458336Z "digest": "sha256:cc9ed3f492faeb8df7ef7ca746a2c85d1189181e3a2516a19c0126bd224ad5d9" 2025-04-21T21:42:56.9458771Z }, 2025-04-21T21:42:56.9458947Z { 2025-04-21T21:42:56.9459240Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9459617Z "size": 7944, 2025-04-21T21:42:56.9459989Z "digest": "sha256:6133da88e6692508c12cae6643582782b9a424aa73892cf6e4c0befebe516269" 2025-04-21T21:42:56.9460408Z }, 2025-04-21T21:42:56.9460585Z { 2025-04-21T21:42:56.9460885Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9461264Z "size": 8071, 2025-04-21T21:42:56.9461625Z "digest": "sha256:f55162ff6427c5c71e1737cb77e4c9913c679981ec16b241d92084fe621b74c9" 2025-04-21T21:42:56.9462045Z }, 2025-04-21T21:42:56.9462221Z { 2025-04-21T21:42:56.9462546Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9462945Z "size": 304, 2025-04-21T21:42:56.9463321Z "digest": "sha256:f2fb53f519b30fceaf1192ac5221e9c1130032af58e6dd7312a7d82f6be9e66e" 2025-04-21T21:42:56.9463743Z }, 2025-04-21T21:42:56.9463922Z { 2025-04-21T21:42:56.9464224Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9464600Z "size": 7631978, 2025-04-21T21:42:56.9464973Z "digest": "sha256:d93b86c61192d758035dbbef6102f4c29339a3bc439c06bde06e3e7efb73c071" 2025-04-21T21:42:56.9465517Z }, 2025-04-21T21:42:56.9465694Z { 2025-04-21T21:42:56.9465996Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9466373Z "size": 108, 2025-04-21T21:42:56.9466754Z "digest": "sha256:2d75b00aef3ef33cd0a6aca03e1b81df01b8d5c874850e5ee0a580fdaeffafda" 2025-04-21T21:42:56.9467181Z }, 2025-04-21T21:42:56.9467353Z { 2025-04-21T21:42:56.9467756Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9468133Z "size": 54145661, 2025-04-21T21:42:56.9468513Z "digest": "sha256:165cb697180f8d5f9e2677a69ad5431ab294be16a39bc423f0d8593513c041f0" 2025-04-21T21:42:56.9468926Z }, 2025-04-21T21:42:56.9469095Z { 2025-04-21T21:42:56.9469401Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9469779Z "size": 495, 2025-04-21T21:42:56.9470142Z "digest": "sha256:952482b7353e3317938e0c2a2c86c92dc2301bec5f1683ae1fc4f72ef2e8ac4b" 2025-04-21T21:42:56.9470562Z }, 2025-04-21T21:42:56.9470737Z { 2025-04-21T21:42:56.9471032Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9471493Z "size": 1179582084, 2025-04-21T21:42:56.9471878Z "digest": "sha256:f4fb942e12736de733008a26bdd886b81205f8a80595e32db7584540ac193776" 2025-04-21T21:42:56.9472295Z }, 2025-04-21T21:42:56.9472487Z { 2025-04-21T21:42:56.9472840Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9473223Z "size": 106, 2025-04-21T21:42:56.9473599Z "digest": "sha256:22d7ad62e40c8f69aaedce849745730ac752c1b564e4a7d39bbae1f12e7f03c6" 2025-04-21T21:42:56.9474035Z }, 2025-04-21T21:42:56.9474210Z { 2025-04-21T21:42:56.9474503Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9475169Z "size": 612, 2025-04-21T21:42:56.9475546Z "digest": "sha256:340ddaf5579e22644b795f56a17c3697533215e26d5761d5cca8bae92205e078" 2025-04-21T21:42:56.9475957Z }, 2025-04-21T21:42:56.9476131Z { 2025-04-21T21:42:56.9476430Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9476812Z "size": 317359363, 2025-04-21T21:42:56.9477199Z "digest": "sha256:abb8b598f42085f2a06e7d8095a6ddb1407e5611c878d0ec1c605c6b62565e39" 2025-04-21T21:42:56.9477616Z }, 2025-04-21T21:42:56.9477780Z { 2025-04-21T21:42:56.9478077Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9478447Z "size": 111, 2025-04-21T21:42:56.9478821Z "digest": "sha256:410b8ad7da03358202afe62f6f95a90ffc13b6c289c59f77d52078e0cfb3b427" 2025-04-21T21:42:56.9479237Z }, 2025-04-21T21:42:56.9479408Z { 2025-04-21T21:42:56.9479707Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9480067Z "size": 475, 2025-04-21T21:42:56.9480435Z "digest": "sha256:79d2e9034f2b1b76f47e712bb3ab767c7b0a8aa2a9bdfc69526ea55345939d3c" 2025-04-21T21:42:56.9480862Z }, 2025-04-21T21:42:56.9481029Z { 2025-04-21T21:42:56.9481335Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9481708Z "size": 180, 2025-04-21T21:42:56.9482068Z "digest": "sha256:c2128146ebc3a8a388e3ae687d3a8509470f980106006e2c07f01878a39cb1e2" 2025-04-21T21:42:56.9482484Z }, 2025-04-21T21:42:56.9482657Z { 2025-04-21T21:42:56.9482955Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9483325Z "size": 529690496, 2025-04-21T21:42:56.9483719Z "digest": "sha256:4b1e8ee3de3882b3d9b74a17aa0895307d6b7f5aa4bde3fe3338dc7599d38fcf" 2025-04-21T21:42:56.9484148Z }, 2025-04-21T21:42:56.9484315Z { 2025-04-21T21:42:56.9484613Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9484982Z "size": 183, 2025-04-21T21:42:56.9485346Z "digest": "sha256:02d23db28b3d929d4f9be01edc8371656c3215d2b4c08df0f24fb897e8b9c4eb" 2025-04-21T21:42:56.9485767Z }, 2025-04-21T21:42:56.9485934Z { 2025-04-21T21:42:56.9486227Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9486597Z "size": 529, 2025-04-21T21:42:56.9486971Z "digest": "sha256:24d4ae1408767f7ceda609b9b1eef8ff3b3f7ee876a8a92ba1de0800db9328d1" 2025-04-21T21:42:56.9487404Z }, 2025-04-21T21:42:56.9487576Z { 2025-04-21T21:42:56.9487872Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9488246Z "size": 26134, 2025-04-21T21:42:56.9488761Z "digest": "sha256:2604f3be2ad176777da15e3755dea31789b0e584891d00680961ee3095a8b9e0" 2025-04-21T21:42:56.9489181Z }, 2025-04-21T21:42:56.9489354Z { 2025-04-21T21:42:56.9489648Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9490021Z "size": 106, 2025-04-21T21:42:56.9490392Z "digest": "sha256:ba564700c169e37253f9d67c6224e478ab8ee66a1906b540eaa06a77e3c5b75a" 2025-04-21T21:42:56.9490809Z }, 2025-04-21T21:42:56.9490981Z { 2025-04-21T21:42:56.9491275Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9491653Z "size": 32, 2025-04-21T21:42:56.9492036Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-04-21T21:42:56.9492621Z }, 2025-04-21T21:42:56.9492793Z { 2025-04-21T21:42:56.9493093Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9493468Z "size": 32, 2025-04-21T21:42:56.9493839Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-04-21T21:42:56.9515067Z }, 2025-04-21T21:42:56.9515278Z { 2025-04-21T21:42:56.9515641Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9516072Z "size": 32, 2025-04-21T21:42:56.9516449Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-04-21T21:42:56.9516881Z }, 2025-04-21T21:42:56.9517055Z { 2025-04-21T21:42:56.9517353Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-04-21T21:42:56.9517733Z "size": 32, 2025-04-21T21:42:56.9518120Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-04-21T21:42:56.9518554Z } 2025-04-21T21:42:56.9518721Z ] 2025-04-21T21:42:56.9518899Z } 2025-04-21T21:42:56.9553177Z ##[group]Run set -eux 2025-04-21T21:42:56.9553439Z set -eux 2025-04-21T21:42:56.9554236Z aws secretsmanager get-secret-value --secret-id docker_hub_readonly_token | jq --raw-output '.SecretString' | jq -r .docker_hub_readonly_token | docker login --username pytorchbot --password-stdin 2025-04-21T21:42:56.9564206Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:42:56.9564553Z env: 2025-04-21T21:42:56.9564762Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:42:56.9565005Z ##[endgroup] 2025-04-21T21:42:56.9598282Z + aws secretsmanager get-secret-value --secret-id docker_hub_readonly_token 2025-04-21T21:42:56.9598823Z + jq --raw-output .SecretString 2025-04-21T21:42:56.9599866Z + jq -r .docker_hub_readonly_token 2025-04-21T21:42:56.9601743Z + docker login --username pytorchbot --password-stdin 2025-04-21T21:42:57.5317116Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2025-04-21T21:42:57.5317737Z Configure a credential helper to remove this warning. See 2025-04-21T21:42:57.5318380Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2025-04-21T21:42:57.5318842Z 2025-04-21T21:42:57.5318937Z Login Succeeded 2025-04-21T21:42:57.5412070Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2025-04-21T21:42:57.5412472Z tag=${ECR_DOCKER_IMAGE##*/} 2025-04-21T21:42:57.5412845Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2025-04-21T21:42:57.5421575Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:42:57.5421961Z env: 2025-04-21T21:42:57.5422176Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:42:57.5423036Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.6-cudnn9-py3-gcc9-inductor-benchmarks:f583c1656c56372ad0e1c57f3463a017fbf03138 2025-04-21T21:42:57.5423857Z ##[endgroup] 2025-04-21T21:42:57.5453899Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.6-cudnn9-py3-gcc9-inductor-benchmarks-f583c1656c56372ad0e1c57f3463a017fbf03138 2025-04-21T21:42:57.5510597Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2025-04-21T21:42:57.5511009Z with: 2025-04-21T21:42:57.5511760Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.6-cudnn9-py3-gcc9-inductor-benchmarks:f583c1656c56372ad0e1c57f3463a017fbf03138 2025-04-21T21:42:57.5512674Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-04-21T21:42:57.5513039Z env: 2025-04-21T21:42:57.5513247Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:42:57.5513490Z ##[endgroup] 2025-04-21T21:42:57.5538061Z ##[group]Run set -x 2025-04-21T21:42:57.5538317Z set -x 2025-04-21T21:42:57.5538531Z set +e 2025-04-21T21:42:57.5538741Z  2025-04-21T21:42:57.5538934Z login() { 2025-04-21T21:42:57.5539377Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2025-04-21T21:42:57.5540017Z } 2025-04-21T21:42:57.5540211Z  2025-04-21T21:42:57.5540449Z retry () { 2025-04-21T21:42:57.5540702Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2025-04-21T21:42:57.5541000Z } 2025-04-21T21:42:57.5541197Z  2025-04-21T21:42:57.5541421Z retry login "${DOCKER_REGISTRY}" 2025-04-21T21:42:57.5541703Z  2025-04-21T21:42:57.5542161Z IMAGE_SIZE=$(docker manifest inspect "${DOCKER_IMAGE}" | jq '[.layers[].size, .config.size] | add / 1024 / 1024') 2025-04-21T21:42:57.5542775Z echo "Compressed size of image in MB: ${IMAGE_SIZE}" 2025-04-21T21:42:57.5543169Z  2025-04-21T21:42:57.5543368Z set -e 2025-04-21T21:42:57.5543689Z # ignore output since only exit code is used for conditional 2025-04-21T21:42:57.5544150Z # only pull docker image if it's not available locally 2025-04-21T21:42:57.5544645Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2025-04-21T21:42:57.5545116Z  retry docker pull "${DOCKER_IMAGE}" 2025-04-21T21:42:57.5545526Z fi 2025-04-21T21:42:57.5554071Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:42:57.5554419Z env: 2025-04-21T21:42:57.5554622Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:42:57.5555405Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.6-cudnn9-py3-gcc9-inductor-benchmarks:f583c1656c56372ad0e1c57f3463a017fbf03138 2025-04-21T21:42:57.5556309Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-04-21T21:42:57.5556666Z ##[endgroup] 2025-04-21T21:42:57.5595608Z + set +e 2025-04-21T21:42:57.5596411Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-04-21T21:42:57.5597005Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-04-21T21:42:57.5600244Z + aws ecr get-login-password --region us-east-1 2025-04-21T21:42:57.5601236Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-04-21T21:42:58.0712205Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2025-04-21T21:42:58.0712808Z Configure a credential helper to remove this warning. See 2025-04-21T21:42:58.0713335Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2025-04-21T21:42:58.0713898Z 2025-04-21T21:42:58.0713999Z Login Succeeded 2025-04-21T21:42:58.0743557Z ++ docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.6-cudnn9-py3-gcc9-inductor-benchmarks:f583c1656c56372ad0e1c57f3463a017fbf03138 2025-04-21T21:42:58.0744513Z ++ jq '[.layers[].size, .config.size] | add / 1024 / 1024' 2025-04-21T21:42:58.3061539Z + IMAGE_SIZE=20670.273096084595 2025-04-21T21:42:58.3061909Z + echo 'Compressed size of image in MB: 20670.273096084595' 2025-04-21T21:42:58.3062261Z + set -e 2025-04-21T21:42:58.3063519Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.6-cudnn9-py3-gcc9-inductor-benchmarks:f583c1656c56372ad0e1c57f3463a017fbf03138 2025-04-21T21:42:58.3064457Z Compressed size of image in MB: 20670.273096084595 2025-04-21T21:42:58.3216867Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.6-cudnn9-py3-gcc9-inductor-benchmarks:f583c1656c56372ad0e1c57f3463a017fbf03138 2025-04-21T21:42:58.3218239Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.6-cudnn9-py3-gcc9-inductor-benchmarks:f583c1656c56372ad0e1c57f3463a017fbf03138 2025-04-21T21:42:58.5693343Z f583c1656c56372ad0e1c57f3463a017fbf03138: Pulling from pytorch/pytorch-linux-focal-cuda12.6-cudnn9-py3-gcc9-inductor-benchmarks 2025-04-21T21:42:58.5699778Z 86e5016c2693: Pulling fs layer 2025-04-21T21:42:58.5700379Z 49e139a3d6c2: Pulling fs layer 2025-04-21T21:42:58.5700828Z a14844c8c51f: Pulling fs layer 2025-04-21T21:42:58.5701255Z 18fb524087fb: Pulling fs layer 2025-04-21T21:42:58.5701926Z efd686a7b2c8: Pulling fs layer 2025-04-21T21:42:58.5702315Z 52c648e21334: Pulling fs layer 2025-04-21T21:42:58.5702675Z 56e384e4e5aa: Pulling fs layer 2025-04-21T21:42:58.5702988Z fb71b792ec6c: Pulling fs layer 2025-04-21T21:42:58.5703250Z 5509576f2693: Pulling fs layer 2025-04-21T21:42:58.5703513Z 1e6c6f2d2459: Pulling fs layer 2025-04-21T21:42:58.5703771Z a1fe8922734d: Pulling fs layer 2025-04-21T21:42:58.5704092Z 84e5433913c6: Pulling fs layer 2025-04-21T21:42:58.5704360Z c68e78d48b95: Pulling fs layer 2025-04-21T21:42:58.5704706Z b14b320c459c: Pulling fs layer 2025-04-21T21:42:58.5705081Z 0d929bfaefee: Pulling fs layer 2025-04-21T21:42:58.5705644Z a5a8ea5a3eab: Pulling fs layer 2025-04-21T21:42:58.5706013Z 281f13ef6929: Pulling fs layer 2025-04-21T21:42:58.5706514Z a8695f1c94c3: Pulling fs layer 2025-04-21T21:42:58.5707002Z 18fb524087fb: Waiting 2025-04-21T21:42:58.5707390Z c72f323be801: Pulling fs layer 2025-04-21T21:42:58.5707770Z 4afa91cd0662: Pulling fs layer 2025-04-21T21:42:58.5708176Z a9bdca78eb50: Pulling fs layer 2025-04-21T21:42:58.5708549Z efd686a7b2c8: Waiting 2025-04-21T21:42:58.5708821Z 52c648e21334: Waiting 2025-04-21T21:42:58.5709098Z 87b133205740: Pulling fs layer 2025-04-21T21:42:58.5709477Z 1a1f3a6b6e28: Pulling fs layer 2025-04-21T21:42:58.5709811Z 1e726e5b69df: Pulling fs layer 2025-04-21T21:42:58.5710066Z 5509576f2693: Waiting 2025-04-21T21:42:58.5710378Z 56e384e4e5aa: Waiting 2025-04-21T21:42:58.5710681Z e55ee61cc4aa: Pulling fs layer 2025-04-21T21:42:58.5711010Z 281f13ef6929: Waiting 2025-04-21T21:42:58.5711337Z 1e6c6f2d2459: Waiting 2025-04-21T21:42:58.5711694Z d2c9e5f6d068: Pulling fs layer 2025-04-21T21:42:58.5712053Z a8695f1c94c3: Waiting 2025-04-21T21:42:58.5712311Z a9e23e3c0a18: Pulling fs layer 2025-04-21T21:42:58.5712566Z a5a8ea5a3eab: Waiting 2025-04-21T21:42:58.5712844Z 956077b15723: Pulling fs layer 2025-04-21T21:42:58.5713102Z b840d58dab14: Pulling fs layer 2025-04-21T21:42:58.5713365Z 135c1f91f3df: Pulling fs layer 2025-04-21T21:42:58.5713627Z 1a1f3a6b6e28: Waiting 2025-04-21T21:42:58.5713863Z 2fba85b42323: Pulling fs layer 2025-04-21T21:42:58.5714135Z 7e377528c596: Pulling fs layer 2025-04-21T21:42:58.5714399Z 7c1fe59ec87a: Pulling fs layer 2025-04-21T21:42:58.5714645Z c68e78d48b95: Waiting 2025-04-21T21:42:58.5714887Z bb83f375ed7d: Pulling fs layer 2025-04-21T21:42:58.5715142Z 1e726e5b69df: Waiting 2025-04-21T21:42:58.5715381Z d1ee3dcc43c5: Pulling fs layer 2025-04-21T21:42:58.5715648Z 54687ddf5aab: Pulling fs layer 2025-04-21T21:42:58.5715912Z 4afa91cd0662: Waiting 2025-04-21T21:42:58.5716133Z 87b133205740: Waiting 2025-04-21T21:42:58.5716358Z d2c9e5f6d068: Waiting 2025-04-21T21:42:58.5716584Z 84e5433913c6: Waiting 2025-04-21T21:42:58.5716820Z 6b673e167d0f: Pulling fs layer 2025-04-21T21:42:58.5717069Z a9e23e3c0a18: Waiting 2025-04-21T21:42:58.5717308Z a4f475d6d4d7: Pulling fs layer 2025-04-21T21:42:58.5717563Z 135c1f91f3df: Waiting 2025-04-21T21:42:58.5717792Z 48e60e8ea738: Pulling fs layer 2025-04-21T21:42:58.5718044Z c72f323be801: Waiting 2025-04-21T21:42:58.5718288Z d52c79949221: Pulling fs layer 2025-04-21T21:42:58.5718751Z 7c1fe59ec87a: Waiting 2025-04-21T21:42:58.5718987Z 7e377528c596: Waiting 2025-04-21T21:42:58.5719217Z b840d58dab14: Waiting 2025-04-21T21:42:58.5719434Z 2fba85b42323: Waiting 2025-04-21T21:42:58.5719666Z 846d8aa26523: Pulling fs layer 2025-04-21T21:42:58.5719923Z 595e99541716: Pulling fs layer 2025-04-21T21:42:58.5720167Z 0d929bfaefee: Waiting 2025-04-21T21:42:58.5720391Z 54687ddf5aab: Waiting 2025-04-21T21:42:58.5720618Z 48e60e8ea738: Waiting 2025-04-21T21:42:58.5720847Z 013dfd09e68f: Pulling fs layer 2025-04-21T21:42:58.5721110Z 4402edece585: Pulling fs layer 2025-04-21T21:42:58.5721372Z 82aa68866e53: Pulling fs layer 2025-04-21T21:42:58.5721611Z d52c79949221: Waiting 2025-04-21T21:42:58.5721842Z e624d917de21: Pulling fs layer 2025-04-21T21:42:58.5722105Z 79d2c9f506ec: Pulling fs layer 2025-04-21T21:42:58.5722352Z 013dfd09e68f: Waiting 2025-04-21T21:42:58.5722580Z 3b25d713acb3: Pulling fs layer 2025-04-21T21:42:58.5722938Z 4402edece585: Waiting 2025-04-21T21:42:58.5723201Z c0ad696e18d3: Pulling fs layer 2025-04-21T21:42:58.5723461Z 767566b69317: Pulling fs layer 2025-04-21T21:42:58.5723708Z 79d2c9f506ec: Waiting 2025-04-21T21:42:58.5723940Z 4f4fb700ef54: Pulling fs layer 2025-04-21T21:42:58.5724185Z bb83f375ed7d: Waiting 2025-04-21T21:42:58.5724422Z d83ab53d6ab4: Pulling fs layer 2025-04-21T21:42:58.5724688Z e9ae24bb3b5b: Pulling fs layer 2025-04-21T21:42:58.5724932Z c0ad696e18d3: Waiting 2025-04-21T21:42:58.5725149Z 767566b69317: Waiting 2025-04-21T21:42:58.5725367Z 3b25d713acb3: Waiting 2025-04-21T21:42:58.5725593Z 8c11aa345fff: Pulling fs layer 2025-04-21T21:42:58.5725858Z cc9ed3f492fa: Pulling fs layer 2025-04-21T21:42:58.5726111Z 6b673e167d0f: Waiting 2025-04-21T21:42:58.5726336Z 6133da88e669: Pulling fs layer 2025-04-21T21:42:58.5726596Z f55162ff6427: Pulling fs layer 2025-04-21T21:42:58.5726844Z d83ab53d6ab4: Waiting 2025-04-21T21:42:58.5727062Z e9ae24bb3b5b: Waiting 2025-04-21T21:42:58.5727316Z f2fb53f519b3: Pulling fs layer 2025-04-21T21:42:58.5727585Z d93b86c61192: Pulling fs layer 2025-04-21T21:42:58.5727842Z 8c11aa345fff: Waiting 2025-04-21T21:42:58.5728075Z 2d75b00aef3e: Pulling fs layer 2025-04-21T21:42:58.5728330Z cc9ed3f492fa: Waiting 2025-04-21T21:42:58.5728566Z 165cb697180f: Pulling fs layer 2025-04-21T21:42:58.5728820Z 952482b7353e: Pulling fs layer 2025-04-21T21:42:58.5729077Z a9bdca78eb50: Waiting 2025-04-21T21:42:58.5729319Z f4fb942e1273: Pulling fs layer 2025-04-21T21:42:58.5729564Z 165cb697180f: Waiting 2025-04-21T21:42:58.5729798Z 22d7ad62e40c: Pulling fs layer 2025-04-21T21:42:58.5730056Z 2d75b00aef3e: Waiting 2025-04-21T21:42:58.5730291Z 340ddaf5579e: Pulling fs layer 2025-04-21T21:42:58.5730559Z abb8b598f420: Pulling fs layer 2025-04-21T21:42:58.5730826Z 410b8ad7da03: Pulling fs layer 2025-04-21T21:42:58.5731072Z f2fb53f519b3: Waiting 2025-04-21T21:42:58.5731299Z 82aa68866e53: Waiting 2025-04-21T21:42:58.5731536Z 79d2e9034f2b: Pulling fs layer 2025-04-21T21:42:58.5731793Z c2128146ebc3: Pulling fs layer 2025-04-21T21:42:58.5732066Z 4b1e8ee3de38: Pulling fs layer 2025-04-21T21:42:58.5732338Z 02d23db28b3d: Pulling fs layer 2025-04-21T21:42:58.5732602Z 24d4ae140876: Pulling fs layer 2025-04-21T21:42:58.5732875Z 2604f3be2ad1: Pulling fs layer 2025-04-21T21:42:58.5733170Z f4fb942e1273: Waiting 2025-04-21T21:42:58.5733410Z ba564700c169: Pulling fs layer 2025-04-21T21:42:58.5733657Z 4f4fb700ef54: Waiting 2025-04-21T21:42:58.5733885Z 22d7ad62e40c: Waiting 2025-04-21T21:42:58.5734112Z 340ddaf5579e: Waiting 2025-04-21T21:42:58.5734340Z 4b1e8ee3de38: Waiting 2025-04-21T21:42:58.5734566Z 02d23db28b3d: Waiting 2025-04-21T21:42:58.5734788Z e624d917de21: Waiting 2025-04-21T21:42:58.5735026Z d1ee3dcc43c5: Waiting 2025-04-21T21:42:58.5735247Z c2128146ebc3: Waiting 2025-04-21T21:42:58.5735467Z 79d2e9034f2b: Waiting 2025-04-21T21:42:58.5735691Z 2604f3be2ad1: Waiting 2025-04-21T21:42:58.5735910Z d93b86c61192: Waiting 2025-04-21T21:42:58.5736131Z 952482b7353e: Waiting 2025-04-21T21:42:58.5736356Z abb8b598f420: Waiting 2025-04-21T21:42:58.5736584Z 24d4ae140876: Waiting 2025-04-21T21:42:58.5736808Z 410b8ad7da03: Waiting 2025-04-21T21:42:58.8830985Z 49e139a3d6c2: Verifying Checksum 2025-04-21T21:42:58.8831332Z 49e139a3d6c2: Download complete 2025-04-21T21:42:58.9405339Z 86e5016c2693: Verifying Checksum 2025-04-21T21:42:58.9405803Z 86e5016c2693: Download complete 2025-04-21T21:42:58.9727326Z 18fb524087fb: Verifying Checksum 2025-04-21T21:42:58.9727631Z 18fb524087fb: Download complete 2025-04-21T21:42:59.0884943Z efd686a7b2c8: Verifying Checksum 2025-04-21T21:42:59.0885409Z efd686a7b2c8: Download complete 2025-04-21T21:42:59.1905569Z 56e384e4e5aa: Verifying Checksum 2025-04-21T21:42:59.1906034Z 56e384e4e5aa: Download complete 2025-04-21T21:42:59.2772605Z a14844c8c51f: Verifying Checksum 2025-04-21T21:42:59.2773050Z a14844c8c51f: Download complete 2025-04-21T21:42:59.2965935Z fb71b792ec6c: Verifying Checksum 2025-04-21T21:42:59.2966393Z fb71b792ec6c: Download complete 2025-04-21T21:42:59.3594245Z 5509576f2693: Verifying Checksum 2025-04-21T21:42:59.3594846Z 5509576f2693: Download complete 2025-04-21T21:42:59.4909290Z a1fe8922734d: Verifying Checksum 2025-04-21T21:42:59.4909703Z a1fe8922734d: Download complete 2025-04-21T21:42:59.6285614Z 84e5433913c6: Verifying Checksum 2025-04-21T21:42:59.6285945Z 84e5433913c6: Download complete 2025-04-21T21:42:59.9940734Z 86e5016c2693: Pull complete 2025-04-21T21:43:00.2864767Z 49e139a3d6c2: Pull complete 2025-04-21T21:43:01.0420481Z a14844c8c51f: Pull complete 2025-04-21T21:43:01.0660499Z 18fb524087fb: Pull complete 2025-04-21T21:43:01.0891386Z efd686a7b2c8: Pull complete 2025-04-21T21:43:03.9428415Z c68e78d48b95: Verifying Checksum 2025-04-21T21:43:03.9428803Z c68e78d48b95: Download complete 2025-04-21T21:43:04.0931072Z b14b320c459c: Verifying Checksum 2025-04-21T21:43:04.0931557Z b14b320c459c: Download complete 2025-04-21T21:43:04.2022646Z 0d929bfaefee: Verifying Checksum 2025-04-21T21:43:04.2820578Z a5a8ea5a3eab: Download complete 2025-04-21T21:43:06.5470589Z 281f13ef6929: Verifying Checksum 2025-04-21T21:43:06.5470938Z 281f13ef6929: Download complete 2025-04-21T21:43:06.6772430Z a8695f1c94c3: Download complete 2025-04-21T21:43:06.7551236Z c72f323be801: Download complete 2025-04-21T21:43:06.8607803Z 4afa91cd0662: Verifying Checksum 2025-04-21T21:43:06.8608196Z 4afa91cd0662: Download complete 2025-04-21T21:43:06.9486685Z a9bdca78eb50: Verifying Checksum 2025-04-21T21:43:06.9486994Z a9bdca78eb50: Download complete 2025-04-21T21:43:17.3263278Z 52c648e21334: Verifying Checksum 2025-04-21T21:43:17.3263708Z 52c648e21334: Download complete 2025-04-21T21:43:17.4276852Z 1a1f3a6b6e28: Verifying Checksum 2025-04-21T21:43:17.4277273Z 1a1f3a6b6e28: Download complete 2025-04-21T21:43:17.5250647Z 1e726e5b69df: Verifying Checksum 2025-04-21T21:43:17.5251051Z 1e726e5b69df: Download complete 2025-04-21T21:43:17.6035419Z e55ee61cc4aa: Verifying Checksum 2025-04-21T21:43:17.6035830Z e55ee61cc4aa: Download complete 2025-04-21T21:43:17.6988078Z d2c9e5f6d068: Verifying Checksum 2025-04-21T21:43:17.6988625Z d2c9e5f6d068: Download complete 2025-04-21T21:43:17.8068264Z a9e23e3c0a18: Verifying Checksum 2025-04-21T21:43:17.8068819Z a9e23e3c0a18: Download complete 2025-04-21T21:43:20.2134906Z 956077b15723: Verifying Checksum 2025-04-21T21:43:20.2135367Z 956077b15723: Download complete 2025-04-21T21:43:20.3310843Z b840d58dab14: Verifying Checksum 2025-04-21T21:43:20.3311334Z b840d58dab14: Download complete 2025-04-21T21:43:20.4247060Z 135c1f91f3df: Download complete 2025-04-21T21:43:27.2161723Z 2fba85b42323: Verifying Checksum 2025-04-21T21:43:27.2162172Z 2fba85b42323: Download complete 2025-04-21T21:43:27.3133529Z 7e377528c596: Verifying Checksum 2025-04-21T21:43:27.3133908Z 7e377528c596: Download complete 2025-04-21T21:43:27.4264684Z 7c1fe59ec87a: Download complete 2025-04-21T21:43:28.4479721Z bb83f375ed7d: Verifying Checksum 2025-04-21T21:43:28.4480193Z bb83f375ed7d: Download complete 2025-04-21T21:43:28.5445676Z d1ee3dcc43c5: Verifying Checksum 2025-04-21T21:43:28.5446148Z d1ee3dcc43c5: Download complete 2025-04-21T21:43:28.6560445Z 54687ddf5aab: Verifying Checksum 2025-04-21T21:43:28.6560908Z 54687ddf5aab: Download complete 2025-04-21T21:43:28.9655645Z 6b673e167d0f: Verifying Checksum 2025-04-21T21:43:28.9655994Z 6b673e167d0f: Download complete 2025-04-21T21:43:28.9689349Z 1e6c6f2d2459: Verifying Checksum 2025-04-21T21:43:28.9689661Z 1e6c6f2d2459: Download complete 2025-04-21T21:43:29.0457807Z a4f475d6d4d7: Verifying Checksum 2025-04-21T21:43:29.0458213Z a4f475d6d4d7: Download complete 2025-04-21T21:43:29.0796387Z 48e60e8ea738: Download complete 2025-04-21T21:43:29.1325897Z d52c79949221: Verifying Checksum 2025-04-21T21:43:29.1326278Z d52c79949221: Download complete 2025-04-21T21:43:29.2289297Z 595e99541716: Download complete 2025-04-21T21:43:29.3078933Z 013dfd09e68f: Verifying Checksum 2025-04-21T21:43:29.3079341Z 013dfd09e68f: Download complete 2025-04-21T21:43:29.9086163Z 52c648e21334: Pull complete 2025-04-21T21:43:30.0295692Z 4402edece585: Verifying Checksum 2025-04-21T21:43:30.0296031Z 4402edece585: Download complete 2025-04-21T21:43:30.0460844Z 56e384e4e5aa: Pull complete 2025-04-21T21:43:30.1328950Z 82aa68866e53: Verifying Checksum 2025-04-21T21:43:30.1329413Z 82aa68866e53: Download complete 2025-04-21T21:43:30.1481580Z fb71b792ec6c: Pull complete 2025-04-21T21:43:30.1844597Z 5509576f2693: Pull complete 2025-04-21T21:43:35.3079906Z e624d917de21: Verifying Checksum 2025-04-21T21:43:35.3080258Z e624d917de21: Download complete 2025-04-21T21:43:53.0885362Z 87b133205740: Verifying Checksum 2025-04-21T21:43:53.0885794Z 87b133205740: Download complete 2025-04-21T21:43:53.1829907Z 3b25d713acb3: Verifying Checksum 2025-04-21T21:43:53.1830381Z 3b25d713acb3: Download complete 2025-04-21T21:43:53.2899164Z c0ad696e18d3: Download complete 2025-04-21T21:43:53.4213430Z 767566b69317: Download complete 2025-04-21T21:43:53.4296321Z 4f4fb700ef54: Verifying Checksum 2025-04-21T21:43:53.4296719Z 4f4fb700ef54: Download complete 2025-04-21T21:43:53.5120316Z d83ab53d6ab4: Download complete 2025-04-21T21:43:53.5816122Z e9ae24bb3b5b: Download complete 2025-04-21T21:43:58.9185832Z 8c11aa345fff: Verifying Checksum 2025-04-21T21:43:58.9186330Z 8c11aa345fff: Download complete 2025-04-21T21:43:59.0275584Z cc9ed3f492fa: Verifying Checksum 2025-04-21T21:43:59.0276028Z cc9ed3f492fa: Download complete 2025-04-21T21:43:59.0982505Z 6133da88e669: Verifying Checksum 2025-04-21T21:43:59.0982993Z 6133da88e669: Download complete 2025-04-21T21:43:59.2026409Z f55162ff6427: Verifying Checksum 2025-04-21T21:43:59.2026727Z f55162ff6427: Download complete 2025-04-21T21:43:59.2879778Z f2fb53f519b3: Download complete 2025-04-21T21:43:59.6555986Z d93b86c61192: Verifying Checksum 2025-04-21T21:43:59.6556429Z d93b86c61192: Download complete 2025-04-21T21:43:59.7622910Z 2d75b00aef3e: Verifying Checksum 2025-04-21T21:43:59.7623353Z 2d75b00aef3e: Download complete 2025-04-21T21:44:01.7493951Z 165cb697180f: Verifying Checksum 2025-04-21T21:44:01.7494299Z 165cb697180f: Download complete 2025-04-21T21:44:01.8469123Z 952482b7353e: Download complete 2025-04-21T21:44:35.8682438Z f4fb942e1273: Verifying Checksum 2025-04-21T21:44:35.8682823Z f4fb942e1273: Download complete 2025-04-21T21:44:35.9584960Z 22d7ad62e40c: Verifying Checksum 2025-04-21T21:44:35.9585277Z 22d7ad62e40c: Download complete 2025-04-21T21:44:36.0626995Z 340ddaf5579e: Verifying Checksum 2025-04-21T21:44:36.0627939Z 340ddaf5579e: Download complete 2025-04-21T21:44:45.3196271Z abb8b598f420: Verifying Checksum 2025-04-21T21:44:45.3196719Z abb8b598f420: Download complete 2025-04-21T21:44:45.4551582Z 410b8ad7da03: Download complete 2025-04-21T21:44:45.5383029Z 79d2e9034f2b: Verifying Checksum 2025-04-21T21:44:45.5383520Z 79d2e9034f2b: Download complete 2025-04-21T21:44:45.6068809Z c2128146ebc3: Download complete 2025-04-21T21:44:59.9213610Z 4b1e8ee3de38: Verifying Checksum 2025-04-21T21:44:59.9213959Z 4b1e8ee3de38: Download complete 2025-04-21T21:45:00.0074560Z 02d23db28b3d: Verifying Checksum 2025-04-21T21:45:00.0075049Z 02d23db28b3d: Download complete 2025-04-21T21:45:00.0901329Z 24d4ae140876: Verifying Checksum 2025-04-21T21:45:00.0901657Z 24d4ae140876: Download complete 2025-04-21T21:45:00.1803286Z 2604f3be2ad1: Download complete 2025-04-21T21:45:00.2562275Z ba564700c169: Download complete 2025-04-21T21:45:20.9206066Z 846d8aa26523: Verifying Checksum 2025-04-21T21:45:20.9207856Z 846d8aa26523: Download complete 2025-04-21T21:45:22.4134598Z 1e6c6f2d2459: Pull complete 2025-04-21T21:45:22.5909515Z a1fe8922734d: Pull complete 2025-04-21T21:45:22.6775890Z 84e5433913c6: Pull complete 2025-04-21T21:45:33.9256645Z 79d2c9f506ec: Verifying Checksum 2025-04-21T21:45:33.9256998Z 79d2c9f506ec: Download complete 2025-04-21T21:45:35.1220173Z c68e78d48b95: Pull complete 2025-04-21T21:45:35.2677619Z b14b320c459c: Pull complete 2025-04-21T21:45:35.3431336Z 0d929bfaefee: Pull complete 2025-04-21T21:45:35.4589143Z a5a8ea5a3eab: Pull complete 2025-04-21T21:45:37.6469358Z 281f13ef6929: Pull complete 2025-04-21T21:45:37.7999709Z a8695f1c94c3: Pull complete 2025-04-21T21:45:37.9433546Z c72f323be801: Pull complete 2025-04-21T21:45:38.0147363Z 4afa91cd0662: Pull complete 2025-04-21T21:45:38.0437734Z a9bdca78eb50: Pull complete 2025-04-21T21:46:34.3849834Z 87b133205740: Pull complete 2025-04-21T21:46:34.4835273Z 1a1f3a6b6e28: Pull complete 2025-04-21T21:46:34.5881036Z 1e726e5b69df: Pull complete 2025-04-21T21:46:34.7354478Z e55ee61cc4aa: Pull complete 2025-04-21T21:46:34.8291185Z d2c9e5f6d068: Pull complete 2025-04-21T21:46:34.8800849Z a9e23e3c0a18: Pull complete 2025-04-21T21:46:37.2327254Z 956077b15723: Pull complete 2025-04-21T21:46:37.3999581Z b840d58dab14: Pull complete 2025-04-21T21:46:37.4890501Z 135c1f91f3df: Pull complete 2025-04-21T21:46:45.8568943Z 2fba85b42323: Pull complete 2025-04-21T21:46:46.0115686Z 7e377528c596: Pull complete 2025-04-21T21:46:46.1914410Z 7c1fe59ec87a: Pull complete 2025-04-21T21:46:46.9395969Z bb83f375ed7d: Pull complete 2025-04-21T21:46:47.0949398Z d1ee3dcc43c5: Pull complete 2025-04-21T21:46:47.1690269Z 54687ddf5aab: Pull complete 2025-04-21T21:46:47.5236387Z 6b673e167d0f: Pull complete 2025-04-21T21:46:47.6482552Z a4f475d6d4d7: Pull complete 2025-04-21T21:46:47.9543499Z 48e60e8ea738: Pull complete 2025-04-21T21:46:48.0978476Z d52c79949221: Pull complete 2025-04-21T21:47:27.8282024Z 846d8aa26523: Pull complete 2025-04-21T21:47:27.9711437Z 595e99541716: Pull complete 2025-04-21T21:47:28.1624306Z 013dfd09e68f: Pull complete 2025-04-21T21:47:30.6195591Z 4402edece585: Pull complete 2025-04-21T21:47:30.7812107Z 82aa68866e53: Pull complete 2025-04-21T21:47:32.1635182Z e624d917de21: Pull complete 2025-04-21T21:49:02.1705056Z 79d2c9f506ec: Pull complete 2025-04-21T21:49:02.1924684Z 3b25d713acb3: Pull complete 2025-04-21T21:49:02.2145606Z c0ad696e18d3: Pull complete 2025-04-21T21:49:02.2598688Z 767566b69317: Pull complete 2025-04-21T21:49:02.2826369Z 4f4fb700ef54: Pull complete 2025-04-21T21:49:02.3058764Z d83ab53d6ab4: Pull complete 2025-04-21T21:49:02.3287381Z e9ae24bb3b5b: Pull complete 2025-04-21T21:49:09.4360211Z 8c11aa345fff: Pull complete 2025-04-21T21:49:09.6116639Z cc9ed3f492fa: Pull complete 2025-04-21T21:49:09.7053927Z 6133da88e669: Pull complete 2025-04-21T21:49:09.7588069Z f55162ff6427: Pull complete 2025-04-21T21:49:09.7920210Z f2fb53f519b3: Pull complete 2025-04-21T21:49:11.0365709Z d93b86c61192: Pull complete 2025-04-21T21:49:11.1939234Z 2d75b00aef3e: Pull complete 2025-04-21T21:49:12.7619028Z 165cb697180f: Pull complete 2025-04-21T21:49:12.9473248Z 952482b7353e: Pull complete 2025-04-21T21:49:23.5443311Z f4fb942e1273: Pull complete 2025-04-21T21:49:23.6250854Z 22d7ad62e40c: Pull complete 2025-04-21T21:49:23.7936317Z 340ddaf5579e: Pull complete 2025-04-21T21:49:26.8016044Z abb8b598f420: Pull complete 2025-04-21T21:49:26.9579180Z 410b8ad7da03: Pull complete 2025-04-21T21:49:27.0445200Z 79d2e9034f2b: Pull complete 2025-04-21T21:49:27.0890419Z c2128146ebc3: Pull complete 2025-04-21T21:49:31.2390320Z 4b1e8ee3de38: Pull complete 2025-04-21T21:49:31.3464405Z 02d23db28b3d: Pull complete 2025-04-21T21:49:31.4385997Z 24d4ae140876: Pull complete 2025-04-21T21:49:31.4879505Z 2604f3be2ad1: Pull complete 2025-04-21T21:49:31.5549922Z ba564700c169: Pull complete 2025-04-21T21:49:31.9321568Z Digest: sha256:745e08885687216ce5adf3196dea7f49a9467042491210addb990693d7fef90a 2025-04-21T21:49:31.9491428Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.6-cudnn9-py3-gcc9-inductor-benchmarks:f583c1656c56372ad0e1c57f3463a017fbf03138 2025-04-21T21:49:31.9643965Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.6-cudnn9-py3-gcc9-inductor-benchmarks:f583c1656c56372ad0e1c57f3463a017fbf03138 2025-04-21T21:49:31.9703205Z ##[group]Run echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-04-21T21:49:31.9704077Z echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-04-21T21:49:31.9715714Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:49:31.9716104Z env: 2025-04-21T21:49:31.9716538Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:49:31.9716784Z ##[endgroup] 2025-04-21T21:49:31.9890050Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2025-04-21T21:49:31.9890437Z with: 2025-04-21T21:49:31.9890649Z driver-version: 570.133.07 2025-04-21T21:49:31.9890888Z env: 2025-04-21T21:49:31.9891096Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:49:31.9891342Z ##[endgroup] 2025-04-21T21:49:31.9997338Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2025-04-21T21:49:31.9997729Z with: 2025-04-21T21:49:31.9997935Z timeout_minutes: 10 2025-04-21T21:49:31.9998165Z max_attempts: 3 2025-04-21T21:49:32.0021933Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 nvidia-container-toolkit-1.16.2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 nvidia-container-toolkit-1.16.2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" # Turn off persistent mode so that the installation script can unload the kernel module sudo killall nvidia-persistenced || true else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" # Fix https://github.com/NVIDIA/nvidia-docker/issues/1648 on runners with # more than one GPUs. This just needs to be run once. The command fails # on subsequent runs and complains that the mode is already on, but that's # ok sudo nvidia-persistenced || true # This should show persistence mode ON nvidia-smi 2025-04-21T21:49:32.0045821Z retry_wait_seconds: 10 2025-04-21T21:49:32.0046088Z polling_interval_seconds: 1 2025-04-21T21:49:32.0046349Z warning_on_retry: true 2025-04-21T21:49:32.0046603Z continue_on_error: false 2025-04-21T21:49:32.0046843Z env: 2025-04-21T21:49:32.0047041Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:49:32.0047297Z DRIVER_VERSION: 570.133.07 2025-04-21T21:49:32.0047545Z ##[endgroup] 2025-04-21T21:49:32.0917898Z == Installing nvidia driver NVIDIA-Linux-x86_64-570.133.07.run == 2025-04-21T21:49:32.0919328Z + pre_install_nvidia_driver_amzn2 2025-04-21T21:49:32.0921500Z + sudo yum remove -y nvidia-driver-latest-dkms 2025-04-21T21:49:32.6745360Z No match for argument: nvidia-driver-latest-dkms 2025-04-21T21:49:32.6746368Z No packages marked for removal. 2025-04-21T21:49:32.7155940Z Dependencies resolved. 2025-04-21T21:49:32.7165636Z Nothing to do. 2025-04-21T21:49:32.7166083Z Complete! 2025-04-21T21:49:32.7981646Z + install_nvidia_driver_common 2025-04-21T21:49:32.7985613Z + echo 'Before installing NVIDIA driver' 2025-04-21T21:49:32.7987269Z Before installing NVIDIA driver 2025-04-21T21:49:32.7989060Z + lspci 2025-04-21T21:49:32.8110739Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2025-04-21T21:49:32.8111239Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2025-04-21T21:49:32.8111781Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2025-04-21T21:49:32.8112295Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2025-04-21T21:49:32.8112767Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe EBS Controller 2025-04-21T21:49:32.8113282Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2025-04-21T21:49:32.8114188Z 00:1e.0 3D controller: NVIDIA Corporation GA102GL [A10G] (rev a1) 2025-04-21T21:49:32.8114667Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2025-04-21T21:49:32.8115053Z + lsmod 2025-04-21T21:49:32.8157648Z Module Size Used by 2025-04-21T21:49:32.8157984Z veth 36864 0 2025-04-21T21:49:32.8158348Z nvidia_modeset 1716224 0 2025-04-21T21:49:32.8158629Z video 65536 1 nvidia_modeset 2025-04-21T21:49:32.8158929Z wmi 36864 1 video 2025-04-21T21:49:32.8159201Z nvidia_uvm 1884160 0 2025-04-21T21:49:32.8159493Z nvidia 11583488 7 nvidia_uvm,nvidia_modeset 2025-04-21T21:49:32.8159841Z drm 602112 1 nvidia 2025-04-21T21:49:32.8160216Z drm_panel_orientation_quirks 32768 1 drm 2025-04-21T21:49:32.8160601Z backlight 24576 3 video,drm,nvidia_modeset 2025-04-21T21:49:32.8161032Z i2c_core 110592 2 nvidia,drm 2025-04-21T21:49:32.8161332Z xt_conntrack 16384 1 2025-04-21T21:49:32.8161595Z nft_chain_nat 16384 3 2025-04-21T21:49:32.8161853Z xt_MASQUERADE 20480 1 2025-04-21T21:49:32.8162141Z nf_nat 57344 2 nft_chain_nat,xt_MASQUERADE 2025-04-21T21:49:32.8162472Z nf_conntrack_netlink 57344 0 2025-04-21T21:49:32.8162872Z nf_conntrack 184320 4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE 2025-04-21T21:49:32.8163297Z nf_defrag_ipv6 24576 1 nf_conntrack 2025-04-21T21:49:32.8163609Z nf_defrag_ipv4 16384 1 nf_conntrack 2025-04-21T21:49:32.8163899Z xfrm_user 57344 1 2025-04-21T21:49:32.8164152Z xfrm_algo 16384 1 xfrm_user 2025-04-21T21:49:32.8164436Z xt_addrtype 16384 2 2025-04-21T21:49:32.8164691Z nft_compat 20480 4 2025-04-21T21:49:32.8164989Z nf_tables 311296 57 nft_compat,nft_chain_nat 2025-04-21T21:49:32.8165387Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2025-04-21T21:49:32.8165768Z br_netfilter 36864 0 2025-04-21T21:49:32.8166040Z bridge 323584 1 br_netfilter 2025-04-21T21:49:32.8166323Z stp 16384 1 bridge 2025-04-21T21:49:32.8166603Z llc 16384 2 bridge,stp 2025-04-21T21:49:32.8166878Z overlay 167936 0 2025-04-21T21:49:32.8167114Z tls 135168 0 2025-04-21T21:49:32.8167358Z nls_ascii 16384 1 2025-04-21T21:49:32.8167610Z nls_cp437 20480 1 2025-04-21T21:49:32.8167847Z vfat 24576 1 2025-04-21T21:49:32.8168089Z fat 86016 1 vfat 2025-04-21T21:49:32.8168353Z sunrpc 696320 1 2025-04-21T21:49:32.8168594Z ena 180224 0 2025-04-21T21:49:32.8168828Z i8042 45056 0 2025-04-21T21:49:32.8169072Z serio 28672 3 i8042 2025-04-21T21:49:32.8169352Z ghash_clmulni_intel 16384 0 2025-04-21T21:49:32.8169600Z button 24576 0 2025-04-21T21:49:32.8169858Z sch_fq_codel 20480 17 2025-04-21T21:49:32.8170111Z dm_mod 188416 0 2025-04-21T21:49:32.8170346Z fuse 163840 1 2025-04-21T21:49:32.8170585Z loop 36864 0 2025-04-21T21:49:32.8170827Z configfs 57344 1 2025-04-21T21:49:32.8171068Z dax 45056 1 dm_mod 2025-04-21T21:49:32.8171335Z dmi_sysfs 20480 0 2025-04-21T21:49:32.8171583Z crc32_pclmul 16384 0 2025-04-21T21:49:32.8171824Z crc32c_intel 24576 0 2025-04-21T21:49:32.8172068Z efivarfs 24576 1 2025-04-21T21:49:32.8172313Z + modinfo nvidia 2025-04-21T21:49:32.8176256Z filename: /lib/modules/6.1.130-139.222.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2025-04-21T21:49:32.8176724Z import_ns: DMA_BUF 2025-04-21T21:49:32.8176973Z alias: char-major-195-* 2025-04-21T21:49:32.8177239Z version: 570.133.07 2025-04-21T21:49:32.8177480Z supported: external 2025-04-21T21:49:32.8177908Z license: Dual MIT/GPL 2025-04-21T21:49:32.8178325Z firmware: nvidia/570.133.07/gsp_tu10x.bin 2025-04-21T21:49:32.8178674Z firmware: nvidia/570.133.07/gsp_ga10x.bin 2025-04-21T21:49:32.8178993Z srcversion: 49515739FD8F721A3F2F714 2025-04-21T21:49:32.8179306Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2025-04-21T21:49:32.8179658Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2025-04-21T21:49:32.8180020Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2025-04-21T21:49:32.8180328Z depends: i2c-core,drm 2025-04-21T21:49:32.8180582Z retpoline: Y 2025-04-21T21:49:32.8180795Z name: nvidia 2025-04-21T21:49:32.8181150Z vermagic: 6.1.130-139.222.amzn2023.x86_64 SMP preempt mod_unload modversions 2025-04-21T21:49:32.8181613Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2025-04-21T21:49:32.8182056Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2025-04-21T21:49:32.8182472Z parm: NVreg_ResmanDebugLevel:int 2025-04-21T21:49:32.8182787Z parm: NVreg_RmLogonRC:int 2025-04-21T21:49:32.8183087Z parm: NVreg_ModifyDeviceFiles:int 2025-04-21T21:49:32.8183398Z parm: NVreg_DeviceFileUID:int 2025-04-21T21:49:32.8183699Z parm: NVreg_DeviceFileGID:int 2025-04-21T21:49:32.8184006Z parm: NVreg_DeviceFileMode:int 2025-04-21T21:49:32.8184356Z parm: NVreg_InitializeSystemMemoryAllocations:int 2025-04-21T21:49:32.8184746Z parm: NVreg_UsePageAttributeTable:int 2025-04-21T21:49:32.8185082Z parm: NVreg_EnablePCIeGen3:int 2025-04-21T21:49:32.8185377Z parm: NVreg_EnableMSI:int 2025-04-21T21:49:32.8185793Z parm: NVreg_EnableStreamMemOPs:int 2025-04-21T21:49:32.8186156Z parm: NVreg_RestrictProfilingToAdminUsers:int 2025-04-21T21:49:32.8186555Z parm: NVreg_PreserveVideoMemoryAllocations:int 2025-04-21T21:49:32.8186932Z parm: NVreg_EnableS0ixPowerManagement:int 2025-04-21T21:49:32.8187354Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2025-04-21T21:49:32.8187768Z parm: NVreg_DynamicPowerManagement:int 2025-04-21T21:49:32.8188182Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2025-04-21T21:49:32.8188596Z parm: NVreg_EnableGpuFirmware:int 2025-04-21T21:49:32.8189102Z parm: NVreg_EnableGpuFirmwareLogs:int 2025-04-21T21:49:32.8189467Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2025-04-21T21:49:32.8189869Z parm: NVreg_EnableUserNUMAManagement:int 2025-04-21T21:49:32.8190234Z parm: NVreg_MemoryPoolSize:int 2025-04-21T21:49:32.8190552Z parm: NVreg_KMallocHeapMaxSize:int 2025-04-21T21:49:32.8190924Z parm: NVreg_VMallocHeapMaxSize:int 2025-04-21T21:49:32.8191354Z parm: NVreg_IgnoreMMIOCheck:int 2025-04-21T21:49:32.8191658Z parm: NVreg_NvLinkDisable:int 2025-04-21T21:49:32.8191992Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2025-04-21T21:49:32.8192356Z parm: NVreg_RegisterPCIDriver:int 2025-04-21T21:49:32.8192689Z parm: NVreg_EnableResizableBar:int 2025-04-21T21:49:32.8193012Z parm: NVreg_EnableDbgBreakpoint:int 2025-04-21T21:49:32.8193353Z parm: NVreg_EnableNonblockingOpen:int 2025-04-21T21:49:32.8193685Z parm: NVreg_RegistryDwords:charp 2025-04-21T21:49:32.8194019Z parm: NVreg_RegistryDwordsPerDevice:charp 2025-04-21T21:49:32.8194342Z parm: NVreg_RmMsg:charp 2025-04-21T21:49:32.8194626Z parm: NVreg_GpuBlacklist:charp 2025-04-21T21:49:32.8194946Z parm: NVreg_TemporaryFilePath:charp 2025-04-21T21:49:32.8195263Z parm: NVreg_ExcludedGpus:charp 2025-04-21T21:49:32.8195570Z parm: NVreg_DmaRemapPeerMmio:int 2025-04-21T21:49:32.8195895Z parm: NVreg_RmNvlinkBandwidth:charp 2025-04-21T21:49:32.8196238Z parm: NVreg_RmNvlinkBandwidthLinkCount:int 2025-04-21T21:49:32.8196587Z parm: NVreg_ImexChannelCount:int 2025-04-21T21:49:32.8197014Z parm: NVreg_CreateImexChannel0:int 2025-04-21T21:49:32.8197437Z parm: NVreg_GrdmaPciTopoCheckOverride:int 2025-04-21T21:49:32.8197769Z parm: rm_firmware_active:charp 2025-04-21T21:49:32.8198059Z + HAS_NVIDIA_DRIVER=0 2025-04-21T21:49:32.8198301Z ++ command -v nvidia-smi 2025-04-21T21:49:32.8198550Z + '[' -x /usr/bin/nvidia-smi ']' 2025-04-21T21:49:32.8198798Z + set +e 2025-04-21T21:49:32.8199101Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2025-04-21T21:49:32.8420172Z + INSTALLED_DRIVER_VERSION=570.133.07 2025-04-21T21:49:32.8420499Z + NVIDIA_SMI_STATUS=0 2025-04-21T21:49:32.8420995Z + '[' 0 -ne 0 ']' 2025-04-21T21:49:32.8421213Z + '[' 570.133.07 '!=' 570.133.07 ']' 2025-04-21T21:49:32.8421487Z + HAS_NVIDIA_DRIVER=1 2025-04-21T21:49:32.8421920Z + echo 'NVIDIA driver (570.133.07) has already been installed. Skipping NVIDIA driver installation' 2025-04-21T21:49:32.8422384Z + set -e 2025-04-21T21:49:32.8422571Z + '[' 1 -eq 0 ']' 2025-04-21T21:49:32.8422965Z NVIDIA driver (570.133.07) has already been installed. Skipping NVIDIA driver installation 2025-04-21T21:49:32.8423431Z + post_install_nvidia_driver_common 2025-04-21T21:49:32.8426876Z + sudo modprobe nvidia 2025-04-21T21:49:32.9271948Z + echo 'After installing NVIDIA driver' 2025-04-21T21:49:32.9272268Z + lspci 2025-04-21T21:49:32.9272496Z After installing NVIDIA driver 2025-04-21T21:49:32.9391251Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2025-04-21T21:49:32.9391774Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2025-04-21T21:49:32.9392317Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2025-04-21T21:49:32.9392829Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2025-04-21T21:49:32.9393298Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe EBS Controller 2025-04-21T21:49:32.9393819Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2025-04-21T21:49:32.9394328Z 00:1e.0 3D controller: NVIDIA Corporation GA102GL [A10G] (rev a1) 2025-04-21T21:49:32.9394795Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2025-04-21T21:49:32.9395191Z + lsmod 2025-04-21T21:49:32.9424554Z Module Size Used by 2025-04-21T21:49:32.9424876Z veth 36864 0 2025-04-21T21:49:32.9425219Z nvidia_modeset 1716224 0 2025-04-21T21:49:32.9425616Z video 65536 1 nvidia_modeset 2025-04-21T21:49:32.9425910Z wmi 36864 1 video 2025-04-21T21:49:32.9426174Z nvidia_uvm 1884160 0 2025-04-21T21:49:32.9426463Z nvidia 11583488 7 nvidia_uvm,nvidia_modeset 2025-04-21T21:49:32.9426788Z drm 602112 1 nvidia 2025-04-21T21:49:32.9427096Z drm_panel_orientation_quirks 32768 1 drm 2025-04-21T21:49:32.9427442Z backlight 24576 3 video,drm,nvidia_modeset 2025-04-21T21:49:32.9427784Z i2c_core 110592 2 nvidia,drm 2025-04-21T21:49:32.9428078Z xt_conntrack 16384 1 2025-04-21T21:49:32.9428330Z nft_chain_nat 16384 3 2025-04-21T21:49:32.9428591Z xt_MASQUERADE 20480 1 2025-04-21T21:49:32.9428884Z nf_nat 57344 2 nft_chain_nat,xt_MASQUERADE 2025-04-21T21:49:32.9429209Z nf_conntrack_netlink 57344 0 2025-04-21T21:49:32.9429594Z nf_conntrack 184320 4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE 2025-04-21T21:49:32.9430027Z nf_defrag_ipv6 24576 1 nf_conntrack 2025-04-21T21:49:32.9430333Z nf_defrag_ipv4 16384 1 nf_conntrack 2025-04-21T21:49:32.9430613Z xfrm_user 57344 1 2025-04-21T21:49:32.9430872Z xfrm_algo 16384 1 xfrm_user 2025-04-21T21:49:32.9431156Z xt_addrtype 16384 2 2025-04-21T21:49:32.9431399Z nft_compat 20480 4 2025-04-21T21:49:32.9431700Z nf_tables 311296 57 nft_compat,nft_chain_nat 2025-04-21T21:49:32.9432105Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2025-04-21T21:49:32.9432749Z br_netfilter 36864 0 2025-04-21T21:49:32.9433162Z bridge 323584 1 br_netfilter 2025-04-21T21:49:32.9433454Z stp 16384 1 bridge 2025-04-21T21:49:32.9433733Z llc 16384 2 bridge,stp 2025-04-21T21:49:32.9434007Z overlay 167936 0 2025-04-21T21:49:32.9434255Z tls 135168 0 2025-04-21T21:49:32.9434544Z nls_ascii 16384 1 2025-04-21T21:49:32.9434821Z nls_cp437 20480 1 2025-04-21T21:49:32.9435068Z vfat 24576 1 2025-04-21T21:49:32.9435312Z fat 86016 1 vfat 2025-04-21T21:49:32.9435568Z sunrpc 696320 1 2025-04-21T21:49:32.9435828Z ena 180224 0 2025-04-21T21:49:32.9436065Z i8042 45056 0 2025-04-21T21:49:32.9436306Z serio 28672 3 i8042 2025-04-21T21:49:32.9436577Z ghash_clmulni_intel 16384 0 2025-04-21T21:49:32.9436832Z button 24576 0 2025-04-21T21:49:32.9437088Z sch_fq_codel 20480 17 2025-04-21T21:49:32.9437344Z dm_mod 188416 0 2025-04-21T21:49:32.9437587Z fuse 163840 1 2025-04-21T21:49:32.9437829Z loop 36864 0 2025-04-21T21:49:32.9438067Z configfs 57344 1 2025-04-21T21:49:32.9438320Z dax 45056 1 dm_mod 2025-04-21T21:49:32.9438588Z dmi_sysfs 20480 0 2025-04-21T21:49:32.9438835Z crc32_pclmul 16384 0 2025-04-21T21:49:32.9439086Z crc32c_intel 24576 0 2025-04-21T21:49:32.9439337Z efivarfs 24576 1 2025-04-21T21:49:32.9439592Z + modinfo nvidia 2025-04-21T21:49:32.9443440Z filename: /lib/modules/6.1.130-139.222.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2025-04-21T21:49:32.9443944Z import_ns: DMA_BUF 2025-04-21T21:49:32.9444199Z alias: char-major-195-* 2025-04-21T21:49:32.9444464Z version: 570.133.07 2025-04-21T21:49:32.9444714Z supported: external 2025-04-21T21:49:32.9444972Z license: Dual MIT/GPL 2025-04-21T21:49:32.9445265Z firmware: nvidia/570.133.07/gsp_tu10x.bin 2025-04-21T21:49:32.9445598Z firmware: nvidia/570.133.07/gsp_ga10x.bin 2025-04-21T21:49:32.9455802Z srcversion: 49515739FD8F721A3F2F714 2025-04-21T21:49:32.9456146Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2025-04-21T21:49:32.9456483Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2025-04-21T21:49:32.9456829Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2025-04-21T21:49:32.9457145Z depends: i2c-core,drm 2025-04-21T21:49:32.9457406Z retpoline: Y 2025-04-21T21:49:32.9457628Z name: nvidia 2025-04-21T21:49:32.9457995Z vermagic: 6.1.130-139.222.amzn2023.x86_64 SMP preempt mod_unload modversions 2025-04-21T21:49:32.9458471Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2025-04-21T21:49:32.9458914Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2025-04-21T21:49:32.9459333Z parm: NVreg_ResmanDebugLevel:int 2025-04-21T21:49:32.9459662Z parm: NVreg_RmLogonRC:int 2025-04-21T21:49:32.9459963Z parm: NVreg_ModifyDeviceFiles:int 2025-04-21T21:49:32.9460278Z parm: NVreg_DeviceFileUID:int 2025-04-21T21:49:32.9460585Z parm: NVreg_DeviceFileGID:int 2025-04-21T21:49:32.9460890Z parm: NVreg_DeviceFileMode:int 2025-04-21T21:49:32.9461246Z parm: NVreg_InitializeSystemMemoryAllocations:int 2025-04-21T21:49:32.9461644Z parm: NVreg_UsePageAttributeTable:int 2025-04-21T21:49:32.9461978Z parm: NVreg_EnablePCIeGen3:int 2025-04-21T21:49:32.9462271Z parm: NVreg_EnableMSI:int 2025-04-21T21:49:32.9462572Z parm: NVreg_EnableStreamMemOPs:int 2025-04-21T21:49:32.9462926Z parm: NVreg_RestrictProfilingToAdminUsers:int 2025-04-21T21:49:32.9463312Z parm: NVreg_PreserveVideoMemoryAllocations:int 2025-04-21T21:49:32.9463698Z parm: NVreg_EnableS0ixPowerManagement:int 2025-04-21T21:49:32.9464247Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2025-04-21T21:49:32.9464755Z parm: NVreg_DynamicPowerManagement:int 2025-04-21T21:49:32.9465174Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2025-04-21T21:49:32.9465710Z parm: NVreg_EnableGpuFirmware:int 2025-04-21T21:49:32.9466052Z parm: NVreg_EnableGpuFirmwareLogs:int 2025-04-21T21:49:32.9466424Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2025-04-21T21:49:32.9466804Z parm: NVreg_EnableUserNUMAManagement:int 2025-04-21T21:49:32.9467146Z parm: NVreg_MemoryPoolSize:int 2025-04-21T21:49:32.9467463Z parm: NVreg_KMallocHeapMaxSize:int 2025-04-21T21:49:32.9467803Z parm: NVreg_VMallocHeapMaxSize:int 2025-04-21T21:49:32.9468140Z parm: NVreg_IgnoreMMIOCheck:int 2025-04-21T21:49:32.9468448Z parm: NVreg_NvLinkDisable:int 2025-04-21T21:49:32.9468785Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2025-04-21T21:49:32.9469308Z parm: NVreg_RegisterPCIDriver:int 2025-04-21T21:49:32.9469664Z parm: NVreg_EnableResizableBar:int 2025-04-21T21:49:32.9469992Z parm: NVreg_EnableDbgBreakpoint:int 2025-04-21T21:49:32.9470342Z parm: NVreg_EnableNonblockingOpen:int 2025-04-21T21:49:32.9470684Z parm: NVreg_RegistryDwords:charp 2025-04-21T21:49:32.9471020Z parm: NVreg_RegistryDwordsPerDevice:charp 2025-04-21T21:49:32.9471355Z parm: NVreg_RmMsg:charp 2025-04-21T21:49:32.9471648Z parm: NVreg_GpuBlacklist:charp 2025-04-21T21:49:32.9471972Z parm: NVreg_TemporaryFilePath:charp 2025-04-21T21:49:32.9472303Z parm: NVreg_ExcludedGpus:charp 2025-04-21T21:49:32.9472620Z parm: NVreg_DmaRemapPeerMmio:int 2025-04-21T21:49:32.9472951Z parm: NVreg_RmNvlinkBandwidth:charp 2025-04-21T21:49:32.9473305Z parm: NVreg_RmNvlinkBandwidthLinkCount:int 2025-04-21T21:49:32.9473658Z parm: NVreg_ImexChannelCount:int 2025-04-21T21:49:32.9473987Z parm: NVreg_CreateImexChannel0:int 2025-04-21T21:49:32.9474333Z parm: NVreg_GrdmaPciTopoCheckOverride:int 2025-04-21T21:49:32.9474676Z parm: rm_firmware_active:charp 2025-04-21T21:49:32.9475240Z + set +e 2025-04-21T21:49:32.9475433Z + nvidia-smi 2025-04-21T21:49:32.9619329Z Mon Apr 21 21:49:32 2025 2025-04-21T21:49:32.9619721Z +-----------------------------------------------------------------------------------------+ 2025-04-21T21:49:32.9620300Z | NVIDIA-SMI 570.133.07 Driver Version: 570.133.07 CUDA Version: 12.8 | 2025-04-21T21:49:32.9620860Z |-----------------------------------------+------------------------+----------------------+ 2025-04-21T21:49:32.9621419Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2025-04-21T21:49:32.9622027Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2025-04-21T21:49:32.9622524Z | | | MIG M. | 2025-04-21T21:49:32.9622910Z |=========================================+========================+======================| 2025-04-21T21:49:32.9776557Z | 0 NVIDIA A10G On | 00000000:00:1E.0 Off | 0 | 2025-04-21T21:49:32.9777005Z | 0% 28C P8 15W / 300W | 0MiB / 23028MiB | 0% Default | 2025-04-21T21:49:32.9777386Z | | | N/A | 2025-04-21T21:49:32.9777768Z +-----------------------------------------+------------------------+----------------------+ 2025-04-21T21:49:32.9781078Z 2025-04-21T21:49:32.9781473Z +-----------------------------------------------------------------------------------------+ 2025-04-21T21:49:32.9781891Z | Processes: | 2025-04-21T21:49:32.9782627Z | GPU GI CI PID Type Process name GPU Memory | 2025-04-21T21:49:32.9783034Z | ID ID Usage | 2025-04-21T21:49:32.9783373Z |=========================================================================================| 2025-04-21T21:49:32.9786484Z | No running processes found | 2025-04-21T21:49:32.9786956Z +-----------------------------------------------------------------------------------------+ 2025-04-21T21:49:33.2464376Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2025-04-21T21:49:33.2632497Z NVIDIA A10G 2025-04-21T21:49:33.2672716Z + NVIDIA_SMI_STATUS=0 2025-04-21T21:49:33.2673006Z + '[' 0 -eq 0 ']' 2025-04-21T21:49:33.2673243Z + echo 'INFO: Ignoring allowed status 0' 2025-04-21T21:49:33.2673532Z + set -e 2025-04-21T21:49:33.2673736Z INFO: Ignoring allowed status 0 2025-04-21T21:49:33.2683008Z == Installing nvidia container toolkit for amzn2023 == 2025-04-21T21:49:33.2686875Z + sudo yum install -y yum-utils 2025-04-21T21:49:33.8152050Z Last metadata expiration check: 1:52:56 ago on Mon Apr 21 19:56:37 2025. 2025-04-21T21:49:33.8397636Z Package dnf-utils-4.3.0-13.amzn2023.0.5.noarch is already installed. 2025-04-21T21:49:33.8796769Z Dependencies resolved. 2025-04-21T21:49:33.8978714Z Nothing to do. 2025-04-21T21:49:33.8979063Z Complete! 2025-04-21T21:49:33.9937510Z + [[ amzn2023 == \a\m\z\n\2\0\2\3 ]] 2025-04-21T21:49:33.9938121Z + YUM_REPO_URL=https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2025-04-21T21:49:33.9938976Z + sudo yum-config-manager --add-repo https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2025-04-21T21:49:34.3174679Z Adding repo from: https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2025-04-21T21:49:34.3850603Z + sudo yum install -y nvidia-docker2 nvidia-container-toolkit-1.16.2 2025-04-21T21:49:34.9849851Z nvidia-container-toolkit 14 kB/s | 833 B 00:00 2025-04-21T21:49:35.0111720Z Package nvidia-docker2-2.14.0-1.noarch is already installed. 2025-04-21T21:49:35.0117212Z Package nvidia-container-toolkit-1.16.2-1.x86_64 is already installed. 2025-04-21T21:49:35.0513622Z Dependencies resolved. 2025-04-21T21:49:35.0695481Z Nothing to do. 2025-04-21T21:49:35.0696265Z Complete! 2025-04-21T21:49:35.1782987Z + sudo systemctl restart docker 2025-04-21T21:50:24.0649323Z nvidia-persistenced failed to initialize. Check syslog for more details. 2025-04-21T21:50:24.0853645Z Mon Apr 21 21:50:24 2025 2025-04-21T21:50:24.0854072Z +-----------------------------------------------------------------------------------------+ 2025-04-21T21:50:24.0854578Z | NVIDIA-SMI 570.133.07 Driver Version: 570.133.07 CUDA Version: 12.8 | 2025-04-21T21:50:24.0855068Z |-----------------------------------------+------------------------+----------------------+ 2025-04-21T21:50:24.0855609Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2025-04-21T21:50:24.0856144Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2025-04-21T21:50:24.0856581Z | | | MIG M. | 2025-04-21T21:50:24.0856909Z |=========================================+========================+======================| 2025-04-21T21:50:24.0987387Z | 0 NVIDIA A10G On | 00000000:00:1E.0 Off | 0 | 2025-04-21T21:50:24.0987862Z | 0% 28C P8 16W / 300W | 0MiB / 23028MiB | 0% Default | 2025-04-21T21:50:24.0988243Z | | | N/A | 2025-04-21T21:50:24.0988654Z +-----------------------------------------+------------------------+----------------------+ 2025-04-21T21:50:24.0992027Z 2025-04-21T21:50:24.0993039Z +-----------------------------------------------------------------------------------------+ 2025-04-21T21:50:24.0993465Z | Processes: | 2025-04-21T21:50:24.0993909Z | GPU GI CI PID Type Process name GPU Memory | 2025-04-21T21:50:24.0994322Z | ID ID Usage | 2025-04-21T21:50:24.0994666Z |=========================================================================================| 2025-04-21T21:50:24.0998332Z | No running processes found | 2025-04-21T21:50:24.0998816Z +-----------------------------------------------------------------------------------------+ 2025-04-21T21:50:25.1208616Z Command completed after 1 attempt(s). 2025-04-21T21:50:25.1308479Z Prepare all required actions 2025-04-21T21:50:25.1337364Z ##[group]Run ./.github/actions/get-workflow-job-id 2025-04-21T21:50:25.1337700Z with: 2025-04-21T21:50:25.1338143Z github-token: *** 2025-04-21T21:50:25.1338363Z env: 2025-04-21T21:50:25.1338580Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:50:25.1338901Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T21:50:25.1339244Z ##[endgroup] 2025-04-21T21:50:25.1363855Z ##[group]Run set -eux 2025-04-21T21:50:25.1364113Z set -eux 2025-04-21T21:50:25.1364529Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2025-04-21T21:50:25.1379378Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:50:25.1379736Z env: 2025-04-21T21:50:25.1379946Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:50:25.1380258Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T21:50:25.1380751Z GITHUB_TOKEN: *** 2025-04-21T21:50:25.1380982Z ##[endgroup] 2025-04-21T21:50:25.1413043Z + python3 .github/scripts/get_workflow_job_id.py 14580756816 i-09a65d907c0d198ba 2025-04-21T21:50:26.2239885Z setting job-id=40900324325 2025-04-21T21:50:26.2240417Z setting job-name=cuda12.6-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2025-04-21T21:50:26.2371524Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 dataclasses_json==0.6.7 2025-04-21T21:50:26.2372205Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 dataclasses_json==0.6.7 2025-04-21T21:50:26.2372745Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2025-04-21T21:50:26.2373196Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2025-04-21T21:50:26.2382442Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:50:26.2382794Z env: 2025-04-21T21:50:26.2383003Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:50:26.2383310Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T21:50:26.2383644Z JOB_ID: 40900324325 2025-04-21T21:50:26.2384077Z JOB_NAME: cuda12.6-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2025-04-21T21:50:26.2384554Z WORKFLOW_NAME: inductor 2025-04-21T21:50:26.2384807Z WORKFLOW_RUN_ID: 14580756816 2025-04-21T21:50:26.2385096Z ##[endgroup] 2025-04-21T21:50:26.4784917Z Defaulting to user installation because normal site-packages is not writeable 2025-04-21T21:50:26.4976651Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.9/site-packages (5.9.1) 2025-04-21T21:50:26.4981056Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.9/site-packages (11.525.84) 2025-04-21T21:50:26.4985906Z Requirement already satisfied: dataclasses_json==0.6.7 in /home/ec2-user/.local/lib/python3.9/site-packages (0.6.7) 2025-04-21T21:50:26.5090256Z Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from dataclasses_json==0.6.7) (3.26.1) 2025-04-21T21:50:26.5094147Z Requirement already satisfied: typing-inspect<1,>=0.4.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from dataclasses_json==0.6.7) (0.9.0) 2025-04-21T21:50:26.5181087Z Requirement already satisfied: packaging>=17.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from marshmallow<4.0.0,>=3.18.0->dataclasses_json==0.6.7) (25.0) 2025-04-21T21:50:26.5209550Z Requirement already satisfied: typing-extensions>=3.7.4 in /home/ec2-user/.local/lib/python3.9/site-packages (from typing-inspect<1,>=0.4.0->dataclasses_json==0.6.7) (4.13.2) 2025-04-21T21:50:26.5216681Z Requirement already satisfied: mypy-extensions>=0.3.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from typing-inspect<1,>=0.4.0->dataclasses_json==0.6.7) (1.0.0) 2025-04-21T21:50:26.7120749Z Prepare all required actions 2025-04-21T21:50:26.7121124Z Getting action download info 2025-04-21T21:50:26.8330047Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2025-04-21T21:50:27.1147085Z Download action repository 'actions/download-artifact@v4' (SHA:95815c38cf2ff2164869cbab79da8d1f422bc89e) 2025-04-21T21:50:27.4427811Z ##[group]Run ./.github/actions/download-build-artifacts 2025-04-21T21:50:27.4428150Z with: 2025-04-21T21:50:27.4428390Z name: linux-focal-cuda12.6-py3.10-gcc9-sm86 2025-04-21T21:50:27.4428712Z s3-bucket: gha-artifacts 2025-04-21T21:50:27.4428955Z env: 2025-04-21T21:50:27.4429153Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:50:27.4429456Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T21:50:27.4429790Z ##[endgroup] 2025-04-21T21:50:27.4476105Z ##[group]Run seemethere/download-artifact-s3@v4 2025-04-21T21:50:27.4476419Z with: 2025-04-21T21:50:27.4476684Z name: linux-focal-cuda12.6-py3.10-gcc9-sm86 2025-04-21T21:50:27.4476996Z s3-bucket: gha-artifacts 2025-04-21T21:50:27.4477244Z region: us-east-1 2025-04-21T21:50:27.4477450Z env: 2025-04-21T21:50:27.4477643Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:50:27.4477952Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T21:50:27.4478292Z ##[endgroup] 2025-04-21T21:50:27.9379131Z (node:370999) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2025-04-21T21:50:27.9379604Z 2025-04-21T21:50:27.9379786Z Please migrate your code to use AWS SDK for JavaScript (v3). 2025-04-21T21:50:27.9380287Z For more information, check the migration guide at https://a.co/7PzMCcy 2025-04-21T21:50:27.9380804Z (Use `node --trace-warnings ...` to show where the warning was created) 2025-04-21T21:50:28.0390873Z Found 1 objects with prefix pytorch/pytorch/14580756816/linux-focal-cuda12.6-py3.10-gcc9-sm86/ 2025-04-21T21:50:28.0391605Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2025-04-21T21:50:37.5248586Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2025-04-21T21:50:37.5253876Z Artifact download has finished successfully 2025-04-21T21:50:37.5606722Z ##[group]Run unzip -o artifacts.zip 2025-04-21T21:50:37.5607073Z unzip -o artifacts.zip 2025-04-21T21:50:37.5616907Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:50:37.5617270Z env: 2025-04-21T21:50:37.5617482Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:50:37.5617794Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T21:50:37.5618132Z ##[endgroup] 2025-04-21T21:50:37.5674127Z Archive: artifacts.zip 2025-04-21T21:50:37.5675943Z creating: dist/ 2025-04-21T21:50:39.7406798Z inflating: dist/torch-2.8.0a0+gitb68c0ef-cp310-cp310-linux_x86_64.whl 2025-04-21T21:50:39.7540281Z inflating: dist/.ninja_log 2025-04-21T21:50:39.7541297Z creating: build/custom_test_artifacts/ 2025-04-21T21:50:39.7541799Z creating: build/custom_test_artifacts/custom-op-build/ 2025-04-21T21:50:39.7542271Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2025-04-21T21:50:39.7542833Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2025-04-21T21:50:39.7550962Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2025-04-21T21:50:39.7551590Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/ 2025-04-21T21:50:39.7552193Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CMakeSystem.cmake 2025-04-21T21:50:39.7552848Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdC/ 2025-04-21T21:50:39.7553473Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdC/tmp/ 2025-04-21T21:50:39.7555852Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdC/CMakeCCompilerId.c 2025-04-21T21:50:39.7557436Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdC/a.out 2025-04-21T21:50:39.7558154Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CMakeCCompiler.cmake 2025-04-21T21:50:39.7558825Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCXX/ 2025-04-21T21:50:39.7559490Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCXX/tmp/ 2025-04-21T21:50:39.7562461Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCXX/CMakeCXXCompilerId.cpp 2025-04-21T21:50:39.7563765Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCXX/a.out 2025-04-21T21:50:39.7565008Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CMakeCXXCompiler.cmake 2025-04-21T21:50:39.7567273Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_C.bin 2025-04-21T21:50:39.7569333Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_CXX.bin 2025-04-21T21:50:39.7570080Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/ 2025-04-21T21:50:39.7570738Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/ 2025-04-21T21:50:39.7612433Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2025-04-21T21:50:39.7653818Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2025-04-21T21:50:39.7654796Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2025-04-21T21:50:39.7702647Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2025-04-21T21:50:39.7704377Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2025-04-21T21:50:39.7706250Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2025-04-21T21:50:39.7708047Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2025-04-21T21:50:39.7709765Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2025-04-21T21:50:39.7710863Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2025-04-21T21:50:39.7711789Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2025-04-21T21:50:39.7712719Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2025-04-21T21:50:39.7713619Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2025-04-21T21:50:39.7714468Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2025-04-21T21:50:39.7715462Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.reg.c 2025-04-21T21:50:39.7716275Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.fatbin 2025-04-21T21:50:39.7717090Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2025-04-21T21:50:39.7717879Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.o 2025-04-21T21:50:39.7718682Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/CMakeCUDACompilerId.cu 2025-04-21T21:50:39.7793164Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCUDA/a.out 2025-04-21T21:50:39.7794181Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CMakeCUDACompiler.cmake 2025-04-21T21:50:39.7868316Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_CUDA.bin 2025-04-21T21:50:39.7869051Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2025-04-21T21:50:39.7869626Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2025-04-21T21:50:39.7870220Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2025-04-21T21:50:39.7870828Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2025-04-21T21:50:39.7871500Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2025-04-21T21:50:39.7872273Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2025-04-21T21:50:39.7873013Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2025-04-21T21:50:39.7873710Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2025-04-21T21:50:39.7874416Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2025-04-21T21:50:39.7875373Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2025-04-21T21:50:39.7876121Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2025-04-21T21:50:39.7876984Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2025-04-21T21:50:39.7878045Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2025-04-21T21:50:39.7899613Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2025-04-21T21:50:39.8098308Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2025-04-21T21:50:39.8099098Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2025-04-21T21:50:39.8100103Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2025-04-21T21:50:39.8101919Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2025-04-21T21:50:39.8103624Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2025-04-21T21:50:39.8105215Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2025-04-21T21:50:39.8106742Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2025-04-21T21:50:39.8108106Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2025-04-21T21:50:39.8109471Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2025-04-21T21:50:39.8110531Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2025-04-21T21:50:39.8111427Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2025-04-21T21:50:39.8128397Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2025-04-21T21:50:39.8212439Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2025-04-21T21:50:39.8214036Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2025-04-21T21:50:39.8215450Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2025-04-21T21:50:39.8217112Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2025-04-21T21:50:39.8218291Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2025-04-21T21:50:39.8219453Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2025-04-21T21:50:39.8220240Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2025-04-21T21:50:39.8220793Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2025-04-21T21:50:39.8221536Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2025-04-21T21:50:39.8222104Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2025-04-21T21:50:39.8386612Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2025-04-21T21:50:39.8443783Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2025-04-21T21:50:39.8444749Z creating: build/custom_test_artifacts/jit-hook-build/ 2025-04-21T21:50:39.8445610Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2025-04-21T21:50:39.8446627Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2025-04-21T21:50:39.8452591Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2025-04-21T21:50:39.8453205Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/ 2025-04-21T21:50:39.8453805Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CMakeSystem.cmake 2025-04-21T21:50:39.8454443Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdC/ 2025-04-21T21:50:39.8455063Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdC/tmp/ 2025-04-21T21:50:39.8457510Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdC/CMakeCCompilerId.c 2025-04-21T21:50:39.8458878Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdC/a.out 2025-04-21T21:50:39.8460048Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CMakeCCompiler.cmake 2025-04-21T21:50:39.8460702Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCXX/ 2025-04-21T21:50:39.8461361Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCXX/tmp/ 2025-04-21T21:50:39.8464075Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCXX/CMakeCXXCompilerId.cpp 2025-04-21T21:50:39.8465564Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCXX/a.out 2025-04-21T21:50:39.8466746Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CMakeCXXCompiler.cmake 2025-04-21T21:50:39.8468886Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_C.bin 2025-04-21T21:50:39.8470960Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_CXX.bin 2025-04-21T21:50:39.8471692Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/ 2025-04-21T21:50:39.8472328Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/ 2025-04-21T21:50:39.8515189Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2025-04-21T21:50:39.8556325Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2025-04-21T21:50:39.8557293Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2025-04-21T21:50:39.8604786Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2025-04-21T21:50:39.8605886Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2025-04-21T21:50:39.8606854Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2025-04-21T21:50:39.8607821Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2025-04-21T21:50:39.8608768Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2025-04-21T21:50:39.8609679Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2025-04-21T21:50:39.8610593Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2025-04-21T21:50:39.8611523Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2025-04-21T21:50:39.8613149Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2025-04-21T21:50:39.8614145Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2025-04-21T21:50:39.8614998Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.reg.c 2025-04-21T21:50:39.8615854Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.fatbin 2025-04-21T21:50:39.8617053Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2025-04-21T21:50:39.8618142Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.o 2025-04-21T21:50:39.8621108Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/CMakeCUDACompilerId.cu 2025-04-21T21:50:39.8695222Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCUDA/a.out 2025-04-21T21:50:39.8696136Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CMakeCUDACompiler.cmake 2025-04-21T21:50:39.8770804Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_CUDA.bin 2025-04-21T21:50:39.8771538Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2025-04-21T21:50:39.8772098Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2025-04-21T21:50:39.8772677Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2025-04-21T21:50:39.8773298Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2025-04-21T21:50:39.8774000Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2025-04-21T21:50:39.8774796Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2025-04-21T21:50:39.8775695Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2025-04-21T21:50:39.8776692Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2025-04-21T21:50:39.8777460Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2025-04-21T21:50:39.8778593Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2025-04-21T21:50:39.8779337Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2025-04-21T21:50:39.8780124Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2025-04-21T21:50:39.8781012Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2025-04-21T21:50:39.8802698Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2025-04-21T21:50:39.8866716Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2025-04-21T21:50:39.8868088Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2025-04-21T21:50:39.8869533Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2025-04-21T21:50:39.8870385Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2025-04-21T21:50:39.8870967Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2025-04-21T21:50:39.8871562Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2025-04-21T21:50:39.8872265Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2025-04-21T21:50:39.8875475Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2025-04-21T21:50:39.8876482Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2025-04-21T21:50:39.8877391Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2025-04-21T21:50:39.8915925Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2025-04-21T21:50:39.8916438Z creating: build/custom_test_artifacts/custom-backend-build/ 2025-04-21T21:50:39.8916941Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2025-04-21T21:50:39.8917525Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2025-04-21T21:50:39.8925394Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2025-04-21T21:50:39.8926057Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/ 2025-04-21T21:50:39.8926702Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CMakeSystem.cmake 2025-04-21T21:50:39.8927410Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdC/ 2025-04-21T21:50:39.8928091Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdC/tmp/ 2025-04-21T21:50:39.8930190Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdC/CMakeCCompilerId.c 2025-04-21T21:50:39.8931980Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdC/a.out 2025-04-21T21:50:39.8932726Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CMakeCCompiler.cmake 2025-04-21T21:50:39.8933453Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCXX/ 2025-04-21T21:50:39.8934154Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCXX/tmp/ 2025-04-21T21:50:39.8936811Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCXX/CMakeCXXCompilerId.cpp 2025-04-21T21:50:39.8938248Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCXX/a.out 2025-04-21T21:50:39.8939461Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CMakeCXXCompiler.cmake 2025-04-21T21:50:39.8941466Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_C.bin 2025-04-21T21:50:39.8943640Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_CXX.bin 2025-04-21T21:50:39.8944426Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/ 2025-04-21T21:50:39.8945120Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/ 2025-04-21T21:50:39.8986715Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2025-04-21T21:50:39.9028217Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2025-04-21T21:50:39.9029242Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2025-04-21T21:50:39.9076617Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2025-04-21T21:50:39.9077938Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2025-04-21T21:50:39.9078970Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2025-04-21T21:50:39.9080007Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2025-04-21T21:50:39.9081141Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2025-04-21T21:50:39.9082138Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2025-04-21T21:50:39.9083131Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2025-04-21T21:50:39.9084122Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2025-04-21T21:50:39.9085174Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2025-04-21T21:50:39.9086089Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2025-04-21T21:50:39.9086975Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.reg.c 2025-04-21T21:50:39.9087852Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.fatbin 2025-04-21T21:50:39.9088756Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2025-04-21T21:50:39.9089945Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/tmp/a_dlink.o 2025-04-21T21:50:39.9092964Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/CMakeCUDACompilerId.cu 2025-04-21T21:50:39.9167678Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCUDA/a.out 2025-04-21T21:50:39.9168458Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CMakeCUDACompiler.cmake 2025-04-21T21:50:39.9243686Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_CUDA.bin 2025-04-21T21:50:39.9244442Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2025-04-21T21:50:39.9245054Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2025-04-21T21:50:39.9245684Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2025-04-21T21:50:39.9246348Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2025-04-21T21:50:39.9247286Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2025-04-21T21:50:39.9248132Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2025-04-21T21:50:39.9248946Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2025-04-21T21:50:39.9249700Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2025-04-21T21:50:39.9250484Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2025-04-21T21:50:39.9251405Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2025-04-21T21:50:39.9252210Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2025-04-21T21:50:39.9252998Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2025-04-21T21:50:39.9253786Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2025-04-21T21:50:39.9258130Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2025-04-21T21:50:39.9380873Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2025-04-21T21:50:39.9381683Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2025-04-21T21:50:39.9382499Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2025-04-21T21:50:39.9383540Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2025-04-21T21:50:39.9384419Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2025-04-21T21:50:39.9385358Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2025-04-21T21:50:39.9386303Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2025-04-21T21:50:39.9387168Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2025-04-21T21:50:39.9388025Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2025-04-21T21:50:39.9388890Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2025-04-21T21:50:39.9389730Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2025-04-21T21:50:39.9410893Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2025-04-21T21:50:39.9466410Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2025-04-21T21:50:39.9478100Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2025-04-21T21:50:39.9478910Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2025-04-21T21:50:39.9479607Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2025-04-21T21:50:39.9480252Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2025-04-21T21:50:39.9480892Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2025-04-21T21:50:39.9481526Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2025-04-21T21:50:39.9482117Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2025-04-21T21:50:39.9482857Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2025-04-21T21:50:39.9483429Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2025-04-21T21:50:39.9577541Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2025-04-21T21:50:39.9616471Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2025-04-21T21:50:39.9616927Z creating: build/lib/ 2025-04-21T21:50:39.9706108Z inflating: build/lib/libprotobuf-lite.a 2025-04-21T21:50:40.0161773Z inflating: build/lib/libprotobuf.a 2025-04-21T21:50:40.0665107Z inflating: build/lib/libprotoc.a 2025-04-21T21:50:40.0675687Z inflating: build/lib/libpthreadpool.a 2025-04-21T21:50:40.0683687Z inflating: build/lib/libcpuinfo.a 2025-04-21T21:50:40.0691724Z inflating: build/lib/libcpuinfo_internals.a 2025-04-21T21:50:40.0692630Z inflating: build/lib/libclog.a 2025-04-21T21:50:40.0712940Z inflating: build/lib/libpytorch_qnnpack.a 2025-04-21T21:50:40.0715571Z inflating: build/lib/libnnpack_reference_layers.a 2025-04-21T21:50:40.0733882Z inflating: build/lib/libnnpack.a 2025-04-21T21:50:40.0920109Z inflating: build/lib/libmicrokernels-prod.a 2025-04-21T21:50:40.1725322Z inflating: build/lib/libmicrokernels-all.a 2025-04-21T21:50:40.1816489Z inflating: build/lib/libXNNPACK.a 2025-04-21T21:50:40.1884898Z inflating: build/lib/libgtest.a 2025-04-21T21:50:40.1902444Z inflating: build/lib/libgmock.a 2025-04-21T21:50:40.1903136Z inflating: build/lib/libgmock_main.a 2025-04-21T21:50:40.1904301Z inflating: build/lib/libgtest_main.a 2025-04-21T21:50:40.1979906Z inflating: build/lib/libbenchmark.a 2025-04-21T21:50:40.1980394Z inflating: build/lib/libbenchmark_main.a 2025-04-21T21:50:40.2044498Z inflating: build/lib/libasmjit.a 2025-04-21T21:50:40.3304480Z inflating: build/lib/libfbgemm.a 2025-04-21T21:50:40.3312243Z inflating: build/lib/libittnotify.a 2025-04-21T21:50:40.3340412Z inflating: build/lib/libtensorpipe_uv.a 2025-04-21T21:50:40.3912784Z inflating: build/lib/libtensorpipe.a 2025-04-21T21:50:40.4168363Z inflating: build/lib/libtensorpipe_cuda.a 2025-04-21T21:50:40.4304243Z inflating: build/lib/libgloo.a 2025-04-21T21:50:40.4692749Z inflating: build/lib/libgloo_cuda.a 2025-04-21T21:50:40.4735411Z inflating: build/lib/libonnx_proto.a 2025-04-21T21:50:40.5465908Z inflating: build/lib/libonnx.a 2025-04-21T21:50:41.6045970Z inflating: build/lib/libdnnl.a 2025-04-21T21:50:41.6065171Z inflating: build/lib/libfmt.a 2025-04-21T21:50:41.6535316Z inflating: build/lib/libkineto.a 2025-04-21T21:50:41.6645345Z inflating: build/lib/libc10.so 2025-04-21T21:50:41.6704037Z inflating: build/lib/libc10_cuda.so 2025-04-21T21:50:41.6705905Z inflating: build/lib/libcaffe2_nvrtc.so 2025-04-21T21:50:41.6707560Z inflating: build/lib/libtorch_global_deps.so 2025-04-21T21:50:44.2325330Z inflating: build/lib/libtorch_cpu.so 2025-04-21T21:50:46.2653068Z inflating: build/lib/libtorch_cuda.so 2025-04-21T21:50:46.2657807Z inflating: build/lib/libunbox_lib.a 2025-04-21T21:50:46.2659352Z inflating: build/lib/libtorch.so 2025-04-21T21:50:47.1421208Z inflating: build/lib/libtorch_cuda_linalg.so 2025-04-21T21:50:47.1491063Z inflating: build/lib/libtorchbind_test.so 2025-04-21T21:50:47.1511188Z inflating: build/lib/libjitbackend_test.so 2025-04-21T21:50:47.1536593Z inflating: build/lib/libbackend_with_compiler.so 2025-04-21T21:50:47.1560932Z inflating: build/lib/libaoti_custom_ops.so 2025-04-21T21:50:47.1563640Z inflating: build/lib/libc10d_cuda_test.so 2025-04-21T21:50:47.1567890Z inflating: build/lib/libshm.so 2025-04-21T21:50:47.3640312Z inflating: build/lib/libtorch_python.so 2025-04-21T21:50:47.3675093Z inflating: build/lib/libnnapi_backend.so 2025-04-21T21:50:47.3675542Z creating: build/bin/ 2025-04-21T21:50:47.4123432Z inflating: build/bin/protoc-3.13.0.0 2025-04-21T21:50:47.4573061Z inflating: build/bin/protoc 2025-04-21T21:50:47.4626321Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2025-04-21T21:50:47.4681555Z inflating: build/bin/c10_DeviceGuard_test 2025-04-21T21:50:47.4737250Z inflating: build/bin/c10_Device_test 2025-04-21T21:50:47.4800519Z inflating: build/bin/c10_DispatchKeySet_test 2025-04-21T21:50:47.4857216Z inflating: build/bin/c10_Scalar_test 2025-04-21T21:50:47.4910109Z inflating: build/bin/c10_StreamGuard_test 2025-04-21T21:50:47.4964578Z inflating: build/bin/c10_SymInt_test 2025-04-21T21:50:47.5022359Z inflating: build/bin/c10_InlineDeviceGuard_test 2025-04-21T21:50:47.5082156Z inflating: build/bin/c10_InlineStreamGuard_test 2025-04-21T21:50:47.5141631Z inflating: build/bin/c10_SizesAndStrides_test 2025-04-21T21:50:47.5216398Z inflating: build/bin/c10_cow_test 2025-04-21T21:50:47.5269329Z inflating: build/bin/c10_ArrayRef_test 2025-04-21T21:50:47.5325840Z inflating: build/bin/c10_Bitset_test 2025-04-21T21:50:47.5378284Z inflating: build/bin/c10_ConstexprCrc_test 2025-04-21T21:50:47.5432142Z inflating: build/bin/c10_DeadlockDetection_test 2025-04-21T21:50:47.5486023Z inflating: build/bin/c10_Half_test 2025-04-21T21:50:47.5545805Z inflating: build/bin/c10_LeftRight_test 2025-04-21T21:50:47.5604187Z inflating: build/bin/c10_Metaprogramming_test 2025-04-21T21:50:47.5658168Z inflating: build/bin/c10_Synchronized_test 2025-04-21T21:50:47.5714781Z inflating: build/bin/c10_NetworkFlow_test 2025-04-21T21:50:47.5774207Z inflating: build/bin/c10_ThreadLocal_test 2025-04-21T21:50:47.5830205Z inflating: build/bin/c10_TypeIndex_test 2025-04-21T21:50:47.5885169Z inflating: build/bin/c10_TypeList_test 2025-04-21T21:50:47.5937695Z inflating: build/bin/c10_TypeTraits_test 2025-04-21T21:50:47.5992854Z inflating: build/bin/c10_accumulate_test 2025-04-21T21:50:47.6052593Z inflating: build/bin/c10_bfloat16_test 2025-04-21T21:50:47.6106610Z inflating: build/bin/c10_bit_cast_test 2025-04-21T21:50:47.6166881Z inflating: build/bin/c10_complex_math_test 2025-04-21T21:50:47.6225885Z inflating: build/bin/c10_complex_test 2025-04-21T21:50:47.6278777Z inflating: build/bin/c10_error_test 2025-04-21T21:50:47.6335129Z inflating: build/bin/c10_exception_test 2025-04-21T21:50:47.6389240Z inflating: build/bin/c10_flags_test 2025-04-21T21:50:47.6560112Z inflating: build/bin/c10_intrusive_ptr_test 2025-04-21T21:50:47.6614158Z inflating: build/bin/c10_generic_math_test 2025-04-21T21:50:47.6669258Z inflating: build/bin/c10_irange_test 2025-04-21T21:50:47.6726399Z inflating: build/bin/c10_lazy_test 2025-04-21T21:50:47.6787046Z inflating: build/bin/c10_logging_test 2025-04-21T21:50:47.6865893Z inflating: build/bin/c10_optional_test 2025-04-21T21:50:47.6932584Z inflating: build/bin/c10_ordered_preserving_dict_test 2025-04-21T21:50:47.6989400Z inflating: build/bin/c10_registry_test 2025-04-21T21:50:47.7149093Z inflating: build/bin/c10_small_vector_test 2025-04-21T21:50:47.7204500Z inflating: build/bin/c10_ssize_test 2025-04-21T21:50:47.7259620Z inflating: build/bin/c10_string_util_test 2025-04-21T21:50:47.7312034Z inflating: build/bin/c10_string_view_test 2025-04-21T21:50:47.7365905Z inflating: build/bin/c10_tempfile_test 2025-04-21T21:50:47.7425593Z inflating: build/bin/c10_typeid_test 2025-04-21T21:50:47.7473858Z inflating: build/bin/c10_intrusive_ptr_benchmark 2025-04-21T21:50:47.7530454Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2025-04-21T21:50:47.7586936Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2025-04-21T21:50:47.7642700Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2025-04-21T21:50:47.7698226Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2025-04-21T21:50:47.7754186Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2025-04-21T21:50:47.7810508Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2025-04-21T21:50:47.7863115Z inflating: build/bin/c10_cuda_CUDATest 2025-04-21T21:50:47.7919210Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2025-04-21T21:50:47.8342192Z inflating: build/bin/vec_test_all_types_DEFAULT 2025-04-21T21:50:47.8771415Z inflating: build/bin/vec_test_all_types_AVX512 2025-04-21T21:50:47.9216781Z inflating: build/bin/vec_test_all_types_AVX2 2025-04-21T21:50:47.9274086Z inflating: build/bin/test_edge_op_registration 2025-04-21T21:50:47.9351678Z inflating: build/bin/Dict_test 2025-04-21T21:50:47.9408039Z inflating: build/bin/Dimname_test 2025-04-21T21:50:47.9475938Z inflating: build/bin/MaybeOwned_test 2025-04-21T21:50:47.9537062Z inflating: build/bin/NamedTensor_test 2025-04-21T21:50:47.9600618Z inflating: build/bin/apply_utils_test 2025-04-21T21:50:47.9664490Z inflating: build/bin/atest 2025-04-21T21:50:47.9730731Z inflating: build/bin/basic 2025-04-21T21:50:47.9789506Z inflating: build/bin/broadcast_test 2025-04-21T21:50:47.9843738Z inflating: build/bin/cpu_allocator_test 2025-04-21T21:50:47.9906807Z inflating: build/bin/cpu_generator_test 2025-04-21T21:50:47.9962783Z inflating: build/bin/cpu_profiling_allocator_test 2025-04-21T21:50:48.0059675Z inflating: build/bin/cpu_rng_test 2025-04-21T21:50:48.0113379Z inflating: build/bin/dlconvertor_test 2025-04-21T21:50:48.0175151Z inflating: build/bin/extension_backend_test 2025-04-21T21:50:48.0233743Z inflating: build/bin/half_test 2025-04-21T21:50:48.0335795Z inflating: build/bin/ivalue_test 2025-04-21T21:50:48.0388933Z inflating: build/bin/lazy_tensor_test 2025-04-21T21:50:48.0446220Z inflating: build/bin/math_kernel_test 2025-04-21T21:50:48.0504280Z inflating: build/bin/memory_format_test 2025-04-21T21:50:48.0560140Z inflating: build/bin/memory_overlapping_test 2025-04-21T21:50:48.0616811Z inflating: build/bin/mobile_memory_cleanup 2025-04-21T21:50:48.0678205Z inflating: build/bin/native_test 2025-04-21T21:50:48.0732326Z inflating: build/bin/operator_name_test 2025-04-21T21:50:48.0787409Z inflating: build/bin/operators_test 2025-04-21T21:50:48.0842742Z inflating: build/bin/packedtensoraccessor_test 2025-04-21T21:50:48.0914612Z inflating: build/bin/pow_test 2025-04-21T21:50:48.0976343Z inflating: build/bin/quantized_test 2025-04-21T21:50:48.1029702Z inflating: build/bin/reduce_ops_test 2025-04-21T21:50:48.1084578Z inflating: build/bin/reportMemoryUsage_test 2025-04-21T21:50:48.1145140Z inflating: build/bin/scalar_tensor_test 2025-04-21T21:50:48.1208129Z inflating: build/bin/scalar_test 2025-04-21T21:50:48.1263522Z inflating: build/bin/StorageUtils_test 2025-04-21T21:50:48.1319141Z inflating: build/bin/stride_properties_test 2025-04-21T21:50:48.1402685Z inflating: build/bin/tensor_iterator_test 2025-04-21T21:50:48.1460796Z inflating: build/bin/test_parallel 2025-04-21T21:50:48.1463970Z inflating: build/bin/thread_init_test 2025-04-21T21:50:48.1523007Z inflating: build/bin/type_ptr_test 2025-04-21T21:50:48.1586713Z inflating: build/bin/type_test 2025-04-21T21:50:48.1642227Z inflating: build/bin/undefined_tensor_test 2025-04-21T21:50:48.1644015Z inflating: build/bin/verify_api_visibility 2025-04-21T21:50:48.1718124Z inflating: build/bin/legacy_vmap_test 2025-04-21T21:50:48.1772929Z inflating: build/bin/weakref_test 2025-04-21T21:50:48.1828304Z inflating: build/bin/wrapdim_test 2025-04-21T21:50:48.1883777Z inflating: build/bin/xla_tensor_test 2025-04-21T21:50:48.1946848Z inflating: build/bin/IListRef_test 2025-04-21T21:50:48.2058438Z inflating: build/bin/List_test 2025-04-21T21:50:48.2129395Z inflating: build/bin/KernelFunction_test 2025-04-21T21:50:48.2256035Z inflating: build/bin/kernel_function_legacy_test 2025-04-21T21:50:48.2356632Z inflating: build/bin/kernel_function_test 2025-04-21T21:50:48.2488703Z inflating: build/bin/kernel_lambda_legacy_test 2025-04-21T21:50:48.2596221Z inflating: build/bin/kernel_lambda_test 2025-04-21T21:50:48.2660925Z inflating: build/bin/kernel_stackbased_test 2025-04-21T21:50:48.2761161Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2025-04-21T21:50:48.2816090Z inflating: build/bin/CppSignature_test 2025-04-21T21:50:48.2875175Z inflating: build/bin/backend_fallback_test 2025-04-21T21:50:48.2927508Z inflating: build/bin/op_allowlist_test 2025-04-21T21:50:48.3234234Z inflating: build/bin/op_registration_test 2025-04-21T21:50:48.3304633Z inflating: build/bin/inline_container_test 2025-04-21T21:50:48.3360073Z inflating: build/bin/cuda_allocator_test 2025-04-21T21:50:48.3416406Z inflating: build/bin/cuda_apply_test 2025-04-21T21:50:48.3480332Z inflating: build/bin/cuda_atomic_ops_test 2025-04-21T21:50:48.3540762Z inflating: build/bin/cuda_caching_host_allocator_test 2025-04-21T21:50:48.3614935Z inflating: build/bin/cuda_complex_math_test 2025-04-21T21:50:48.3678128Z inflating: build/bin/cuda_complex_test 2025-04-21T21:50:48.3743473Z inflating: build/bin/cuda_cub_test 2025-04-21T21:50:48.3796892Z inflating: build/bin/cuda_device_test 2025-04-21T21:50:48.3865824Z inflating: build/bin/cuda_distributions_test 2025-04-21T21:50:48.3920255Z inflating: build/bin/cuda_dlconvertor_test 2025-04-21T21:50:48.3981382Z inflating: build/bin/cuda_generator_test 2025-04-21T21:50:48.4034896Z inflating: build/bin/cuda_half_test 2025-04-21T21:50:48.4089868Z inflating: build/bin/cuda_integer_divider_test 2025-04-21T21:50:48.4142678Z inflating: build/bin/cuda_optional_test 2025-04-21T21:50:48.4198677Z inflating: build/bin/cuda_packedtensoraccessor_test 2025-04-21T21:50:48.4254521Z inflating: build/bin/cuda_reportMemoryUsage_test 2025-04-21T21:50:48.4308505Z inflating: build/bin/cuda_allocatorTraceTracker_test 2025-04-21T21:50:48.4372164Z inflating: build/bin/cuda_stream_test 2025-04-21T21:50:48.4428333Z inflating: build/bin/cuda_vectorized_test 2025-04-21T21:50:48.4481306Z inflating: build/bin/cuda_cudnn_test 2025-04-21T21:50:48.5065818Z inflating: build/bin/test_jit 2025-04-21T21:50:48.5904265Z inflating: build/bin/test_tensorexpr 2025-04-21T21:50:48.5918889Z inflating: build/bin/tutorial_tensorexpr 2025-04-21T21:50:48.5976571Z inflating: build/bin/BackoffTest 2025-04-21T21:50:48.6032992Z inflating: build/bin/FileStoreTest 2025-04-21T21:50:48.6092370Z inflating: build/bin/TCPStoreTest 2025-04-21T21:50:48.6149339Z inflating: build/bin/HashStoreTest 2025-04-21T21:50:48.6219314Z inflating: build/bin/ProcessGroupGlooTest 2025-04-21T21:50:48.6281270Z inflating: build/bin/ProcessGroupGlooAsyncTest 2025-04-21T21:50:48.6349016Z inflating: build/bin/ProcessGroupNCCLTest 2025-04-21T21:50:48.6363552Z inflating: build/bin/ProcessGroupMPITest 2025-04-21T21:50:48.6429890Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2025-04-21T21:50:48.6433114Z inflating: build/bin/example_allreduce 2025-04-21T21:50:48.6491762Z inflating: build/bin/test_dist_autograd 2025-04-21T21:50:48.6564687Z inflating: build/bin/test_cpp_rpc 2025-04-21T21:50:48.7767324Z inflating: build/bin/test_api 2025-04-21T21:50:48.7770135Z inflating: build/bin/parallel_benchmark 2025-04-21T21:50:48.7840678Z inflating: build/bin/test_mobile_nnc 2025-04-21T21:50:48.7849774Z inflating: build/bin/aot_model_compiler_test 2025-04-21T21:50:48.8206020Z inflating: build/bin/test_lazy 2025-04-21T21:50:48.8210386Z inflating: build/bin/torch_shm_manager 2025-04-21T21:50:48.8211075Z creating: .additional_ci_files/ 2025-04-21T21:50:48.8326155Z inflating: .additional_ci_files/test-times.json 2025-04-21T21:50:48.8775693Z inflating: .additional_ci_files/test-class-times.json 2025-04-21T21:50:48.8811992Z ##[group]Run rm artifacts.zip 2025-04-21T21:50:48.8812283Z rm artifacts.zip 2025-04-21T21:50:48.8821237Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:50:48.8821581Z env: 2025-04-21T21:50:48.8821784Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:50:48.8822094Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T21:50:48.8822420Z ##[endgroup] 2025-04-21T21:50:48.9980458Z ##[group]Run df -H 2025-04-21T21:50:48.9980707Z df -H 2025-04-21T21:50:48.9989465Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:50:48.9989816Z env: 2025-04-21T21:50:48.9990027Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:50:48.9990345Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T21:50:48.9990677Z ##[endgroup] 2025-04-21T21:50:49.0042598Z Filesystem Size Used Avail Use% Mounted on 2025-04-21T21:50:49.0043071Z devtmpfs 4.2M 0 4.2M 0% /dev 2025-04-21T21:50:49.0043523Z tmpfs 34G 0 34G 0% /dev/shm 2025-04-21T21:50:49.0043956Z tmpfs 14G 553k 14G 1% /run 2025-04-21T21:50:49.0044394Z /dev/nvme0n1p1 161G 59G 103G 37% / 2025-04-21T21:50:49.0044815Z tmpfs 34G 29k 34G 1% /tmp 2025-04-21T21:50:49.0045269Z /dev/nvme0n1p128 11M 1.4M 9.2M 13% /boot/efi 2025-04-21T21:50:49.0045744Z tmpfs 6.7G 0 6.7G 0% /run/user/0 2025-04-21T21:50:49.0088095Z Prepare all required actions 2025-04-21T21:50:49.0088651Z Getting action download info 2025-04-21T21:50:49.1159511Z ##[group]Run ./.github/actions/download-td-artifacts 2025-04-21T21:50:49.1159836Z with: 2025-04-21T21:50:49.1160034Z env: 2025-04-21T21:50:49.1160245Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:50:49.1160562Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T21:50:49.1160895Z ##[endgroup] 2025-04-21T21:50:49.1197247Z ##[group]Run seemethere/download-artifact-s3@v4 2025-04-21T21:50:49.1197561Z with: 2025-04-21T21:50:49.1197757Z name: td_results 2025-04-21T21:50:49.1197998Z s3-bucket: gha-artifacts 2025-04-21T21:50:49.1198243Z region: us-east-1 2025-04-21T21:50:49.1198468Z env: 2025-04-21T21:50:49.1198671Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:50:49.1198986Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T21:50:49.1199314Z ##[endgroup] 2025-04-21T21:50:49.5743426Z (node:371019) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2025-04-21T21:50:49.5743993Z 2025-04-21T21:50:49.5744184Z Please migrate your code to use AWS SDK for JavaScript (v3). 2025-04-21T21:50:49.5744669Z For more information, check the migration guide at https://a.co/7PzMCcy 2025-04-21T21:50:49.5745180Z (Use `node --trace-warnings ...` to show where the warning was created) 2025-04-21T21:50:49.6780213Z Found 0 objects with prefix pytorch/pytorch/14580756816/td_results/ 2025-04-21T21:50:49.6786589Z Artifact download has finished successfully 2025-04-21T21:50:49.7136483Z ##[group]Run mkdir -p .additional_ci_files 2025-04-21T21:50:49.7136837Z mkdir -p .additional_ci_files 2025-04-21T21:50:49.7137254Z mv td_results.json .additional_ci_files/td_results.json || true 2025-04-21T21:50:49.7146255Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:50:49.7146609Z env: 2025-04-21T21:50:49.7146817Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:50:49.7147140Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T21:50:49.7147484Z ##[endgroup] 2025-04-21T21:50:49.7215979Z mv: cannot stat 'td_results.json': No such file or directory 2025-04-21T21:50:49.7257365Z ##[group]Run .github/scripts/parse_ref.py 2025-04-21T21:50:49.7257723Z .github/scripts/parse_ref.py 2025-04-21T21:50:49.7265683Z shell: /usr/bin/bash -e {0} 2025-04-21T21:50:49.7265954Z env: 2025-04-21T21:50:49.7266165Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:50:49.7266480Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T21:50:49.7266821Z ##[endgroup] 2025-04-21T21:50:49.7609646Z Prepare all required actions 2025-04-21T21:50:49.7610027Z Getting action download info 2025-04-21T21:50:49.8675487Z ##[group]Run ./.github/actions/filter-test-configs 2025-04-21T21:50:49.8676020Z with: 2025-04-21T21:50:49.8676418Z github-token: *** 2025-04-21T21:50:49.8678021Z test-matrix: {"include": [{"config": "inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]} 2025-04-21T21:50:49.8679866Z job-name: cuda12.6-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2025-04-21T21:50:49.8680332Z env: 2025-04-21T21:50:49.8680709Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:50:49.8681027Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T21:50:49.8681363Z ##[endgroup] 2025-04-21T21:50:49.8725168Z ##[group]Run nick-fields/retry@v3.0.0 2025-04-21T21:50:49.8725456Z with: 2025-04-21T21:50:49.8725653Z shell: bash 2025-04-21T21:50:49.8725862Z timeout_minutes: 10 2025-04-21T21:50:49.8726098Z max_attempts: 5 2025-04-21T21:50:49.8726323Z retry_wait_seconds: 30 2025-04-21T21:50:49.8727055Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2025-04-21T21:50:49.8727802Z polling_interval_seconds: 1 2025-04-21T21:50:49.8728073Z warning_on_retry: true 2025-04-21T21:50:49.8728325Z continue_on_error: false 2025-04-21T21:50:49.8728562Z env: 2025-04-21T21:50:49.8728765Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:50:49.8729084Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T21:50:49.8729557Z GITHUB_TOKEN: *** 2025-04-21T21:50:49.8729775Z ##[endgroup] 2025-04-21T21:50:49.9715345Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2025-04-21T21:50:50.1998994Z Defaulting to user installation because normal site-packages is not writeable 2025-04-21T21:50:50.2173192Z Requirement already satisfied: requests==2.27.1 in /home/ec2-user/.local/lib/python3.9/site-packages (2.27.1) 2025-04-21T21:50:50.2176183Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.9/site-packages (6.0.1) 2025-04-21T21:50:50.2284173Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (1.25.10) 2025-04-21T21:50:50.2292636Z Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (2.10) 2025-04-21T21:50:50.2298072Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2025.1.31) 2025-04-21T21:50:50.2306297Z Requirement already satisfied: charset-normalizer~=2.0.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2.0.12) 2025-04-21T21:50:50.9506707Z Command completed after 1 attempt(s). 2025-04-21T21:50:50.9583793Z ##[group]Run set -x 2025-04-21T21:50:50.9584200Z set -x 2025-04-21T21:50:50.9584417Z  2025-04-21T21:50:50.9584776Z # Use relative path here as this could be checked out anywhere, not necessarily 2025-04-21T21:50:50.9585239Z # in runner workspace 2025-04-21T21:50:50.9585693Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2025-04-21T21:50:50.9594621Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:50:50.9594981Z env: 2025-04-21T21:50:50.9595181Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:50:50.9595500Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T21:50:50.9595838Z ##[endgroup] 2025-04-21T21:50:50.9635893Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2025-04-21T21:50:50.9876944Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2025-04-21T21:50:50.9877496Z echo "Workflow: ${GITHUB_WORKFLOW}" 2025-04-21T21:50:50.9877818Z echo "Job name: ${JOB_NAME}" 2025-04-21T21:50:50.9878096Z  2025-04-21T21:50:50.9878454Z # Use relative path here as this could be checked out anywhere, not necessarily 2025-04-21T21:50:50.9878904Z # in runner workspace 2025-04-21T21:50:50.9879302Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2025-04-21T21:50:50.9879751Z  --workflow "${GITHUB_WORKFLOW}" \ 2025-04-21T21:50:50.9880057Z  --job-name "${JOB_NAME}" \ 2025-04-21T21:50:50.9881757Z  --test-matrix "{"include": [{"config": "inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}" \ 2025-04-21T21:50:50.9883467Z  --selected-test-configs "" \ 2025-04-21T21:50:50.9883785Z  --pr-number "${PR_NUMBER}" \ 2025-04-21T21:50:50.9884078Z  --tag "${TAG}" \ 2025-04-21T21:50:50.9884346Z  --event-name "${EVENT_NAME}" \ 2025-04-21T21:50:50.9884650Z  --schedule "${SCHEDULE}" \ 2025-04-21T21:50:50.9884946Z  --branch "${HEAD_BRANCH}" 2025-04-21T21:50:50.9893738Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:50:50.9894088Z env: 2025-04-21T21:50:50.9894294Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:50:50.9894613Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T21:50:50.9895129Z GITHUB_TOKEN: *** 2025-04-21T21:50:50.9895548Z JOB_NAME: cuda12.6-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2025-04-21T21:50:50.9896025Z PR_NUMBER: 2025-04-21T21:50:50.9896258Z TAG: ciflow/inductor/148893 2025-04-21T21:50:50.9896552Z EVENT_NAME: push 2025-04-21T21:50:50.9896896Z SCHEDULE: 2025-04-21T21:50:50.9897140Z HEAD_BRANCH: 2025-04-21T21:50:50.9897339Z ##[endgroup] 2025-04-21T21:50:50.9925200Z Workflow: inductor 2025-04-21T21:50:50.9925644Z Job name: cuda12.6-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2025-04-21T21:50:51.1913685Z INFO:root:Found no test-config label on the PR, so all test configs are included 2025-04-21T21:50:51.3403726Z ##[group]Run echo "Filtered matrix:" 2025-04-21T21:50:51.3404059Z echo "Filtered matrix:" 2025-04-21T21:50:51.3405733Z echo "{"include": [{"config": "inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}" 2025-04-21T21:50:51.3407453Z  2025-04-21T21:50:51.3407649Z echo 2025-04-21T21:50:51.3407915Z echo "Is the current job unstable? False" 2025-04-21T21:50:51.3408238Z  2025-04-21T21:50:51.3408428Z echo 2025-04-21T21:50:51.3408680Z echo "Is keep-going label set? False" 2025-04-21T21:50:51.3408986Z  2025-04-21T21:50:51.3409176Z echo 2025-04-21T21:50:51.3409405Z echo "Renabled issues? " 2025-04-21T21:50:51.3418152Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:50:51.3418691Z env: 2025-04-21T21:50:51.3418902Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:50:51.3419223Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T21:50:51.3420985Z ##[endgroup] 2025-04-21T21:50:51.3449332Z Filtered matrix: 2025-04-21T21:50:51.3450953Z {include: [{config: inductor_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: inductor_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: inductor_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: inductor_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: inductor_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}]} 2025-04-21T21:50:51.3452488Z 2025-04-21T21:50:51.3452653Z Is the current job unstable? False 2025-04-21T21:50:51.3452848Z 2025-04-21T21:50:51.3452951Z Is keep-going label set? False 2025-04-21T21:50:51.3453132Z 2025-04-21T21:50:51.3453221Z Renabled issues? 2025-04-21T21:50:51.3501692Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2025-04-21T21:50:51.3502210Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2025-04-21T21:50:51.3511139Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T21:50:51.3511497Z env: 2025-04-21T21:50:51.3511711Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:50:51.3512029Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T21:50:51.3512370Z JOB_TIMEOUT: 240 2025-04-21T21:50:51.3512597Z ##[endgroup] 2025-04-21T21:50:51.3589709Z ##[group]Run set -x 2025-04-21T21:50:51.3590028Z set -x 2025-04-21T21:50:51.3590251Z  2025-04-21T21:50:51.3590493Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2025-04-21T21:50:51.3590875Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2025-04-21T21:50:51.3591261Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2025-04-21T21:50:51.3591616Z  TEST_COMMAND=.ci/onnx/test.sh 2025-04-21T21:50:51.3591904Z else 2025-04-21T21:50:51.3592152Z  TEST_COMMAND=.ci/pytorch/test.sh 2025-04-21T21:50:51.3592456Z fi 2025-04-21T21:50:51.3592657Z  2025-04-21T21:50:51.3592948Z # Leaving 1GB for the runner and other things 2025-04-21T21:50:51.3593582Z TOTAL_AVAILABLE_MEMORY_IN_GB=$(awk '/MemTotal/ { printf "%.3f \n", $2/1024/1024 - 1 }' /proc/meminfo) 2025-04-21T21:50:51.3594529Z # https://docs.docker.com/engine/containers/resource_constraints/#--memory-swap-details, the 3GB swap 2025-04-21T21:50:51.3595305Z # comes from https://github.com/pytorch/test-infra/pull/6058 2025-04-21T21:50:51.3595884Z TOTAL_MEMORY_WITH_SWAP=$(("${TOTAL_AVAILABLE_MEMORY_IN_GB%.*}" + 3)) 2025-04-21T21:50:51.3596329Z  2025-04-21T21:50:51.3596612Z if [[ ${BUILD_ENVIRONMENT} == *"s390x"* ]]; then 2025-04-21T21:50:51.3597017Z  SHM_OPTS= 2025-04-21T21:50:51.3597273Z  JENKINS_USER= 2025-04-21T21:50:51.3597643Z  # ensure that docker container cleanly exits in 12 hours 2025-04-21T21:50:51.3598157Z  # if for some reason cleanup action doesn't stop container 2025-04-21T21:50:51.3598584Z  # when job is cancelled 2025-04-21T21:50:51.3598909Z  DOCKER_SHELL_CMD="sleep 12h" 2025-04-21T21:50:51.3599221Z  2025-04-21T21:50:51.3599606Z  # since some steps are skipped on s390x, if they are necessary, run them here 2025-04-21T21:50:51.3600193Z  env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2025-04-21T21:50:51.3600669Z  env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2025-04-21T21:50:51.3601043Z else 2025-04-21T21:50:51.3601300Z  SHM_OPTS="--shm-size=${SHM_SIZE}" 2025-04-21T21:50:51.3601660Z  JENKINS_USER="--user jenkins" 2025-04-21T21:50:51.3601999Z  DOCKER_SHELL_CMD= 2025-04-21T21:50:51.3602263Z fi 2025-04-21T21:50:51.3602474Z  2025-04-21T21:50:51.3602833Z # detached container should get cleaned up by teardown_ec2_linux 2025-04-21T21:50:51.3603405Z # TODO: Stop building test binaries as part of the build phase 2025-04-21T21:50:51.3604566Z # Used for GPU_FLAG, SHM_OPTS, JENKINS_USER and DOCKER_SHELL_CMD since that doesn't play nice 2025-04-21T21:50:51.3605078Z # shellcheck disable=SC2086,SC2090 2025-04-21T21:50:51.3605409Z container_name=$(docker run \ 2025-04-21T21:50:51.3605706Z  ${GPU_FLAG:-} \ 2025-04-21T21:50:51.3606008Z  ${SCCACHE_SERVER_PORT_DOCKER_FLAG:-} \ 2025-04-21T21:50:51.3606349Z  -e BUILD_ENVIRONMENT \ 2025-04-21T21:50:51.3606633Z  -e PR_NUMBER \ 2025-04-21T21:50:51.3606904Z  -e GITHUB_ACTIONS \ 2025-04-21T21:50:51.3607191Z  -e GITHUB_REPOSITORY \ 2025-04-21T21:50:51.3607489Z  -e GITHUB_WORKFLOW \ 2025-04-21T21:50:51.3607768Z  -e GITHUB_JOB \ 2025-04-21T21:50:51.3608035Z  -e GITHUB_RUN_ID \ 2025-04-21T21:50:51.3608312Z  -e GITHUB_RUN_NUMBER \ 2025-04-21T21:50:51.3608598Z  -e GITHUB_RUN_ATTEMPT \ 2025-04-21T21:50:51.3608893Z  -e JOB_ID \ 2025-04-21T21:50:51.3609155Z  -e JOB_NAME \ 2025-04-21T21:50:51.3609416Z  -e BASE_SHA \ 2025-04-21T21:50:51.3609674Z  -e BRANCH \ 2025-04-21T21:50:51.3609912Z  -e SHA1 \ 2025-04-21T21:50:51.3610165Z  -e AWS_DEFAULT_REGION \ 2025-04-21T21:50:51.3610573Z  -e IN_WHEEL_TEST \ 2025-04-21T21:50:51.3610844Z  -e SHARD_NUMBER \ 2025-04-21T21:50:51.3611118Z  -e TEST_CONFIG \ 2025-04-21T21:50:51.3611387Z  -e NUM_TEST_SHARDS \ 2025-04-21T21:50:51.3611674Z  -e REENABLED_ISSUES \ 2025-04-21T21:50:51.3611966Z  -e CONTINUE_THROUGH_ERROR \ 2025-04-21T21:50:51.3612278Z  -e VERBOSE_TEST_LOGS \ 2025-04-21T21:50:51.3612568Z  -e TEST_SHOWLOCALS \ 2025-04-21T21:50:51.3612846Z  -e NO_TEST_TIMEOUT \ 2025-04-21T21:50:51.3613118Z  -e NO_TD \ 2025-04-21T21:50:51.3613371Z  -e TD_DISTRIBUTED \ 2025-04-21T21:50:51.3623522Z  -e PR_LABELS \ 2025-04-21T21:50:51.3623862Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2025-04-21T21:50:51.3624186Z  -e SCCACHE_BUCKET \ 2025-04-21T21:50:51.3624474Z  -e SCCACHE_REGION \ 2025-04-21T21:50:51.3624744Z  -e XLA_CUDA \ 2025-04-21T21:50:51.3625023Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2025-04-21T21:50:51.3625380Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2025-04-21T21:50:51.3625794Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2025-04-21T21:50:51.3626145Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2025-04-21T21:50:51.3626475Z  -e HUGGING_FACE_HUB_TOKEN \ 2025-04-21T21:50:51.3626791Z  -e SCRIBE_GRAPHQL_ACCESS_TOKEN \ 2025-04-21T21:50:51.3627107Z  -e DASHBOARD_TAG \ 2025-04-21T21:50:51.3627371Z  -e IS_A100_RUNNER \ 2025-04-21T21:50:51.3627649Z  -e ARTIFACTS_FILE_SUFFIX \ 2025-04-21T21:50:51.3627996Z  --memory="${TOTAL_AVAILABLE_MEMORY_IN_GB%.*}g" \ 2025-04-21T21:50:51.3628390Z  --memory-swap="${TOTAL_MEMORY_WITH_SWAP}g" \ 2025-04-21T21:50:51.3628785Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2025-04-21T21:50:51.3629162Z  --security-opt seccomp=unconfined \ 2025-04-21T21:50:51.3629488Z  --cap-add=SYS_PTRACE \ 2025-04-21T21:50:51.3629762Z  --ipc=host \ 2025-04-21T21:50:51.3630013Z  ${SHM_OPTS} \ 2025-04-21T21:50:51.3630255Z  --tty \ 2025-04-21T21:50:51.3630481Z  --detach \ 2025-04-21T21:50:51.3630738Z  --name="${container_name}" \ 2025-04-21T21:50:51.3631037Z  ${JENKINS_USER} \ 2025-04-21T21:50:51.3631360Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2025-04-21T21:50:51.3631747Z  -w /var/lib/jenkins/workspace \ 2025-04-21T21:50:51.3632056Z  "${DOCKER_IMAGE}" \ 2025-04-21T21:50:51.3632320Z  ${DOCKER_SHELL_CMD} 2025-04-21T21:50:51.3632571Z ) 2025-04-21T21:50:51.3632970Z # Propagate download.pytorch.org IP to container 2025-04-21T21:50:51.3633594Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2025-04-21T21:50:51.3634245Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2025-04-21T21:50:51.3634631Z  2025-04-21T21:50:51.3634886Z if [[ ${BUILD_ENVIRONMENT} == *"s390x"* ]]; then 2025-04-21T21:50:51.3635421Z  docker exec -t "${container_name}" sh -c "python3 -m pip install -r .ci/docker/requirements-ci.txt" 2025-04-21T21:50:51.3635902Z fi 2025-04-21T21:50:51.3636104Z  2025-04-21T21:50:51.3636561Z docker exec -t "${container_name}" sh -c "python3 -m pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2025-04-21T21:50:51.3644780Z shell: /usr/bin/bash -e {0} 2025-04-21T21:50:51.3645037Z env: 2025-04-21T21:50:51.3645250Z GIT_DEFAULT_BRANCH: main 2025-04-21T21:50:51.3645568Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T21:50:51.3646017Z BUILD_ENVIRONMENT: linux-focal-cuda12.6-py3.10-gcc9-sm86 2025-04-21T21:50:51.3646376Z PR_NUMBER: 2025-04-21T21:50:51.3646621Z GITHUB_REPOSITORY: pytorch/pytorch 2025-04-21T21:50:51.3646917Z GITHUB_WORKFLOW: inductor 2025-04-21T21:50:51.3647171Z GITHUB_JOB: test 2025-04-21T21:50:51.3647524Z GITHUB_RUN_ID: 14580756816 2025-04-21T21:50:51.3647786Z GITHUB_RUN_NUMBER: 128203 2025-04-21T21:50:51.3648048Z GITHUB_RUN_ATTEMPT: 1 2025-04-21T21:50:51.3648293Z JOB_ID: 40900324325 2025-04-21T21:50:51.3648718Z JOB_NAME: cuda12.6-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2025-04-21T21:50:51.3649187Z BRANCH: 2025-04-21T21:50:51.3649436Z SHA1: b68c0effe97351b0c24b2f7aef671559be6aa977 2025-04-21T21:50:51.3649790Z BASE_SHA: b68c0effe97351b0c24b2f7aef671559be6aa977 2025-04-21T21:50:51.3650122Z TEST_CONFIG: inductor_timm 2025-04-21T21:50:51.3650378Z SHARD_NUMBER: 2 2025-04-21T21:50:51.3650602Z NUM_TEST_SHARDS: 2 2025-04-21T21:50:51.3650831Z REENABLED_ISSUES: 2025-04-21T21:50:51.3651070Z CONTINUE_THROUGH_ERROR: False 2025-04-21T21:50:51.3651339Z VERBOSE_TEST_LOGS: False 2025-04-21T21:50:51.3651586Z TEST_SHOWLOCALS: False 2025-04-21T21:50:51.3651836Z NO_TEST_TIMEOUT: False 2025-04-21T21:50:51.3652070Z NO_TD: False 2025-04-21T21:50:51.3652288Z TD_DISTRIBUTED: False 2025-04-21T21:50:51.3652590Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2025-04-21T21:50:51.3652934Z SCCACHE_REGION: us-east-1 2025-04-21T21:50:51.3653174Z SHM_SIZE: 2g 2025-04-21T21:50:51.3653936Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.6-cudnn9-py3-gcc9-inductor-benchmarks:f583c1656c56372ad0e1c57f3463a017fbf03138 2025-04-21T21:50:51.3654742Z XLA_CUDA: 2025-04-21T21:50:51.3655085Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2025-04-21T21:50:51.3655505Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2025-04-21T21:50:51.3655818Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2025-04-21T21:50:51.3656102Z DASHBOARD_TAG: 2025-04-21T21:50:51.3656479Z HUGGING_FACE_HUB_TOKEN: *** 2025-04-21T21:50:51.3656868Z SCRIBE_GRAPHQL_ACCESS_TOKEN: *** 2025-04-21T21:50:51.3657146Z IS_A100_RUNNER: 0 2025-04-21T21:50:51.3657550Z ARTIFACTS_FILE_SUFFIX: test-inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_40900324325 2025-04-21T21:50:51.3658015Z ##[endgroup] 2025-04-21T21:50:51.3684906Z + [[ inductor_timm == \m\u\l\t\i\g\p\u ]] 2025-04-21T21:50:51.3685282Z + [[ linux-focal-cuda12.6-py3.10-gcc9-sm86 == *onnx* ]] 2025-04-21T21:50:51.3685630Z + TEST_COMMAND=.ci/pytorch/test.sh 2025-04-21T21:50:51.3688172Z ++ awk '/MemTotal/ { printf "%.3f \n", $2/1024/1024 - 1 }' /proc/meminfo 2025-04-21T21:50:51.3710770Z + TOTAL_AVAILABLE_MEMORY_IN_GB='61.094 ' 2025-04-21T21:50:51.3711106Z + TOTAL_MEMORY_WITH_SWAP=64 2025-04-21T21:50:51.3711428Z + [[ linux-focal-cuda12.6-py3.10-gcc9-sm86 == *\s\3\9\0\x* ]] 2025-04-21T21:50:51.3711793Z + SHM_OPTS=--shm-size=2g 2025-04-21T21:50:51.3712225Z + JENKINS_USER='--user jenkins' 2025-04-21T21:50:51.3712479Z + DOCKER_SHELL_CMD= 2025-04-21T21:50:51.3722614Z +++ nproc --ignore=2 2025-04-21T21:50:51.3739785Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=14 -e SCCACHE_BUCKET -e SCCACHE_REGION -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e SCRIBE_GRAPHQL_ACCESS_TOKEN -e DASHBOARD_TAG -e IS_A100_RUNNER -e ARTIFACTS_FILE_SUFFIX --memory=61g --memory-swap=64g --env-file=/tmp/github_env_14580756816 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.6-cudnn9-py3-gcc9-inductor-benchmarks:f583c1656c56372ad0e1c57f3463a017fbf03138 2025-04-21T21:51:03.3088626Z + container_name=f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T21:51:03.3090774Z + grep download.pytorch.org /etc/hosts 2025-04-21T21:51:03.3092820Z + docker exec -i f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 sudo bash -c '/bin/cat >> /etc/hosts' 2025-04-21T21:51:03.4753721Z + echo DOCKER_CONTAINER_ID=f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T21:51:03.4754333Z + [[ linux-focal-cuda12.6-py3.10-gcc9-sm86 == *\s\3\9\0\x* ]] 2025-04-21T21:51:03.4757332Z ++ echo dist/torch-2.8.0a0+gitb68c0ef-cp310-cp310-linux_x86_64.whl 2025-04-21T21:51:03.4760330Z + docker exec -t f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 sh -c 'python3 -m pip install dist/torch-2.8.0a0+gitb68c0ef-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2025-04-21T21:51:03.8866690Z Processing ./dist/torch-2.8.0a0+gitb68c0ef-cp310-cp310-linux_x86_64.whl (from torch==2.8.0a0+gitb68c0ef) 2025-04-21T21:51:04.2231727Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.8.0a0+gitb68c0ef->torch==2.8.0a0+gitb68c0ef) (3.16.1) 2025-04-21T21:51:04.2234287Z Requirement already satisfied: typing-extensions>=4.10.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.8.0a0+gitb68c0ef->torch==2.8.0a0+gitb68c0ef) (4.13.2) 2025-04-21T21:51:04.2567917Z Collecting sympy>=1.13.3 (from torch==2.8.0a0+gitb68c0ef->torch==2.8.0a0+gitb68c0ef) 2025-04-21T21:51:04.2580509Z Using cached sympy-1.13.3-py3-none-any.whl.metadata (12 kB) 2025-04-21T21:51:04.2598675Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.8.0a0+gitb68c0ef->torch==2.8.0a0+gitb68c0ef) (2.8.8) 2025-04-21T21:51:04.2602016Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.8.0a0+gitb68c0ef->torch==2.8.0a0+gitb68c0ef) (3.1.6) 2025-04-21T21:51:04.2605706Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.8.0a0+gitb68c0ef->torch==2.8.0a0+gitb68c0ef) (2024.10.0) 2025-04-21T21:51:04.2620538Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.8.0a0+gitb68c0ef->torch==2.8.0a0+gitb68c0ef) (3.3.0) 2025-04-21T21:51:04.2639226Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.8.0a0+gitb68c0ef->torch==2.8.0a0+gitb68c0ef) (1.22.4) 2025-04-21T21:51:04.2647290Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy>=1.13.3->torch==2.8.0a0+gitb68c0ef->torch==2.8.0a0+gitb68c0ef) (1.3.0) 2025-04-21T21:51:04.3033413Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.8.0a0+gitb68c0ef->torch==2.8.0a0+gitb68c0ef) (3.0.2) 2025-04-21T21:51:04.3117515Z Using cached sympy-1.13.3-py3-none-any.whl (6.2 MB) 2025-04-21T21:51:04.9560820Z Installing collected packages: sympy, torch 2025-04-21T21:51:04.9561198Z Attempting uninstall: sympy 2025-04-21T21:51:04.9573045Z Found existing installation: sympy 1.13.1 2025-04-21T21:51:05.1630236Z Uninstalling sympy-1.13.1: 2025-04-21T21:51:06.4729554Z Successfully uninstalled sympy-1.13.1 2025-04-21T21:51:21.0103688Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2025-04-21T21:51:21.0104680Z timm 1.0.14 requires torchvision, which is not installed. 2025-04-21T21:51:21.0105226Z Successfully installed sympy-1.13.3 torch-2.8.0a0+gitb68c0ef 2025-04-21T21:51:21.1088949Z + export TERM=vt100 2025-04-21T21:51:21.1089204Z + TERM=vt100 2025-04-21T21:51:21.1091582Z ++ dirname .ci/pytorch/test.sh 2025-04-21T21:51:21.1104154Z + source .ci/pytorch/common.sh 2025-04-21T21:51:21.1108463Z +++ dirname .ci/pytorch/common.sh 2025-04-21T21:51:21.1119023Z ++ source .ci/pytorch/common_utils.sh 2025-04-21T21:51:21.1120555Z +++ declare -f -t trap_add 2025-04-21T21:51:21.1126816Z ++ set -ex -o pipefail 2025-04-21T21:51:21.1127279Z ++ [[ linux-focal-cuda12.6-py3.10-gcc9-sm86 == *rocm* ]] 2025-04-21T21:51:21.1127701Z ++ BUILD_TEST_LIBTORCH=0 2025-04-21T21:51:21.1128208Z + [[ linux-focal-cuda12.6-py3.10-gcc9-sm86 != *rocm* ]] 2025-04-21T21:51:21.1128662Z + [[ linux-focal-cuda12.6-py3.10-gcc9-sm86 != *s390x* ]] 2025-04-21T21:51:21.1129008Z + [[ -d /var/lib/jenkins/workspace ]] 2025-04-21T21:51:21.1132033Z ++ stat -c %u /var/lib/jenkins/workspace 2025-04-21T21:51:21.1152602Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2025-04-21T21:51:21.1153006Z + trap_add cleanup_workspace EXIT 2025-04-21T21:51:21.1153297Z + trap_add_cmd=cleanup_workspace 2025-04-21T21:51:21.1153554Z + shift 2025-04-21T21:51:21.1153759Z + for trap_add_name in "$@" 2025-04-21T21:51:21.1160580Z +++ trap -p EXIT 2025-04-21T21:51:21.1163756Z ++ eval 'extract_trap_cmd ' 2025-04-21T21:51:21.1164105Z +++ extract_trap_cmd 2025-04-21T21:51:21.1164338Z +++ printf '%s\n' '' 2025-04-21T21:51:21.1164575Z ++ printf '%s\n' cleanup_workspace 2025-04-21T21:51:21.1167074Z + trap -- ' 2025-04-21T21:51:21.1167338Z cleanup_workspace' EXIT 2025-04-21T21:51:21.1167684Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2025-04-21T21:51:21.8738726Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2025-04-21T21:51:21.8762116Z + echo 'Environment variables:' 2025-04-21T21:51:21.8762403Z Environment variables: 2025-04-21T21:51:21.8762676Z + env 2025-04-21T21:51:21.8773344Z NV_LIBCUBLAS_VERSION=12.6.4.1-1 2025-04-21T21:51:21.8773732Z NVIDIA_VISIBLE_DEVICES=all 2025-04-21T21:51:21.8774096Z NV_NVML_DEV_VERSION=12.6.77-1 2025-04-21T21:51:21.8774529Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2025-04-21T21:51:21.8775190Z CONTINUE_THROUGH_ERROR=False 2025-04-21T21:51:21.8775551Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.23.4-1+cuda12.6 2025-04-21T21:51:21.8775968Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.23.4-1 2025-04-21T21:51:21.8776476Z BUILD_ENVIRONMENT=linux-focal-cuda12.6-py3.10-gcc9-sm86 2025-04-21T21:51:21.8776860Z HOSTNAME=f3774c702cf4 2025-04-21T21:51:21.8777616Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_c92d17c4-5f5b-479e-a8d4-6c632516b6b5 2025-04-21T21:51:21.8778446Z GITHUB_ACTION=__self 2025-04-21T21:51:21.8778695Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2025-04-21T21:51:21.8783706Z NVIDIA_REQUIRE_CUDA=cuda>=12.6 brand=unknown,driver>=470,driver<471 brand=grid,driver>=470,driver<471 brand=tesla,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=vapps,driver>=470,driver<471 brand=vpc,driver>=470,driver<471 brand=vcs,driver>=470,driver<471 brand=vws,driver>=470,driver<471 brand=cloudgaming,driver>=470,driver<471 brand=unknown,driver>=535,driver<536 brand=grid,driver>=535,driver<536 brand=tesla,driver>=535,driver<536 brand=nvidia,driver>=535,driver<536 brand=quadro,driver>=535,driver<536 brand=quadrortx,driver>=535,driver<536 brand=nvidiartx,driver>=535,driver<536 brand=vapps,driver>=535,driver<536 brand=vpc,driver>=535,driver<536 brand=vcs,driver>=535,driver<536 brand=vws,driver>=535,driver<536 brand=cloudgaming,driver>=535,driver<536 brand=unknown,driver>=550,driver<551 brand=grid,driver>=550,driver<551 brand=tesla,driver>=550,driver<551 brand=nvidia,driver>=550,driver<551 brand=quadro,driver>=550,driver<551 brand=quadrortx,driver>=550,driver<551 brand=nvidiartx,driver>=550,driver<551 brand=vapps,driver>=550,driver<551 brand=vpc,driver>=550,driver<551 brand=vcs,driver>=550,driver<551 brand=vws,driver>=550,driver<551 brand=cloudgaming,driver>=550,driver<551 2025-04-21T21:51:21.8789111Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-6=12.6.4.1-1 2025-04-21T21:51:21.8789456Z NV_NVTX_VERSION=12.6.77-1 2025-04-21T21:51:21.8789891Z GITHUB_RUN_NUMBER=128203 2025-04-21T21:51:21.8790141Z TEST_CONFIG=inductor_timm 2025-04-21T21:51:21.8790403Z GITHUB_REPOSITORY_OWNER_ID=21003710 2025-04-21T21:51:21.8790710Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2025-04-21T21:51:21.8790991Z IS_A100_RUNNER=0 2025-04-21T21:51:21.8791221Z NV_CUDA_CUDART_DEV_VERSION=12.6.77-1 2025-04-21T21:51:21.8791509Z NV_LIBCUSPARSE_VERSION=12.5.4.2-1 2025-04-21T21:51:21.8791988Z SCRIBE_GRAPHQL_ACCESS_TOKEN=*** 2025-04-21T21:51:21.8792260Z NV_LIBNPP_VERSION=12.3.1.54-1 2025-04-21T21:51:21.8792549Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2025-04-21T21:51:21.8792897Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2025-04-21T21:51:21.8793226Z GITHUB_REF_TYPE=tag 2025-04-21T21:51:21.8793460Z TORCH_CUDA_ARCH_LIST=Maxwell 2025-04-21T21:51:21.8793712Z NCCL_VERSION=2.23.4-1 2025-04-21T21:51:21.8793981Z BASE_SHA=b68c0effe97351b0c24b2f7aef671559be6aa977 2025-04-21T21:51:21.8794291Z XLA_CUDA= 2025-04-21T21:51:21.8794506Z NCCL_LIB_DIR=/usr/local/cuda/lib64/ 2025-04-21T21:51:21.8794898Z HUGGING_FACE_HUB_TOKEN=*** 2025-04-21T21:51:21.8795257Z *** 2025-04-21T21:51:21.8795463Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2025-04-21T21:51:21.8795746Z GITHUB_REPOSITORY_ID=65600975 2025-04-21T21:51:21.8796001Z GITHUB_ACTIONS=true 2025-04-21T21:51:21.8796242Z NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T21:51:21.8796554Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-6=12.6.80-1 2025-04-21T21:51:21.8796888Z NV_LIBNPP_PACKAGE=libnpp-12-6=12.3.1.54-1 2025-04-21T21:51:21.8797222Z SHA1=b68c0effe97351b0c24b2f7aef671559be6aa977 2025-04-21T21:51:21.8797553Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2025-04-21T21:51:21.8797896Z GITHUB_SHA=b68c0effe97351b0c24b2f7aef671559be6aa977 2025-04-21T21:51:21.8798460Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor.yml@refs/tags/ciflow/inductor/148893 2025-04-21T21:51:21.8798968Z UCC_HOME=/usr 2025-04-21T21:51:21.8799196Z NV_LIBCUBLAS_DEV_VERSION=12.6.4.1-1 2025-04-21T21:51:21.8799472Z VERBOSE_TEST_LOGS=False 2025-04-21T21:51:21.8799724Z NVIDIA_PRODUCT_NAME=CUDA 2025-04-21T21:51:21.8800021Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-6 2025-04-21T21:51:21.8800362Z GITHUB_REF=refs/tags/ciflow/inductor/148893 2025-04-21T21:51:21.8800674Z NV_CUDA_CUDART_VERSION=12.6.77-1 2025-04-21T21:51:21.8800941Z SHARD_NUMBER=2 2025-04-21T21:51:21.8801158Z GITHUB_REF_PROTECTED=false 2025-04-21T21:51:21.8801414Z HOME=/var/lib/jenkins 2025-04-21T21:51:21.8801683Z GITHUB_API_URL=https://api.github.com 2025-04-21T21:51:21.8801986Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2025-04-21T21:51:21.8802316Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2025-04-21T21:51:21.8802641Z CUDA_VERSION=12.6.3 2025-04-21T21:51:21.8803068Z NV_LIBCUBLAS_PACKAGE=libcublas-12-6=12.6.4.1-1 2025-04-21T21:51:21.8803365Z USE_SYSTEM_NCCL=1 2025-04-21T21:51:21.8803583Z NUM_TEST_SHARDS=2 2025-04-21T21:51:21.8803796Z UCX_HOME=/usr 2025-04-21T21:51:21.8804151Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-6=12.6.3-1 2025-04-21T21:51:21.8804847Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_c92d17c4-5f5b-479e-a8d4-6c632516b6b5 2025-04-21T21:51:21.8805662Z JOB_NAME=cuda12.6-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2025-04-21T21:51:21.8806412Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_c92d17c4-5f5b-479e-a8d4-6c632516b6b5 2025-04-21T21:51:21.8807146Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2025-04-21T21:51:21.8807599Z GITHUB_EVENT_NAME=push 2025-04-21T21:51:21.8807834Z DASHBOARD_TAG= 2025-04-21T21:51:21.8808054Z GITHUB_RUN_ID=14580756816 2025-04-21T21:51:21.8808343Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-6=12.3.1.54-1 2025-04-21T21:51:21.8808697Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-6 2025-04-21T21:51:21.8809335Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_c92d17c4-5f5b-479e-a8d4-6c632516b6b5 2025-04-21T21:51:21.8809952Z GITHUB_ACTOR=pytorch-bot[bot] 2025-04-21T21:51:21.8810320Z NV_LIBNPP_DEV_VERSION=12.3.1.54-1 2025-04-21T21:51:21.8810587Z PR_NUMBER= 2025-04-21T21:51:21.8810787Z GITHUB_RUN_ATTEMPT=1 2025-04-21T21:51:21.8811032Z ANACONDA_PYTHON_VERSION=3.10 2025-04-21T21:51:21.8811348Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2025-04-21T21:51:21.8811672Z TERM=vt100 2025-04-21T21:51:21.8811885Z NV_LIBCUSPARSE_DEV_VERSION=12.5.4.2-1 2025-04-21T21:51:21.8812172Z INSTALLED_VISION=yes 2025-04-21T21:51:21.8812396Z BRANCH= 2025-04-21T21:51:21.8812595Z SCCACHE_REGION=us-east-1 2025-04-21T21:51:21.8812851Z OPENSSL_ROOT_DIR=/opt/openssl 2025-04-21T21:51:21.8813138Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2025-04-21T21:51:21.8813428Z CUDA_PATH=/usr/local/cuda 2025-04-21T21:51:21.8813918Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2025-04-21T21:51:21.8814451Z GITHUB_SERVER_URL=https://github.com 2025-04-21T21:51:21.8814783Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2025-04-21T21:51:21.8815114Z REENABLED_ISSUES= 2025-04-21T21:51:21.8815358Z SHLVL=1 2025-04-21T21:51:21.8815565Z MAX_JOBS=14 2025-04-21T21:51:21.8815778Z NV_CUDA_LIB_VERSION=12.6.3-1 2025-04-21T21:51:21.8816030Z NVARCH=x86_64 2025-04-21T21:51:21.8816240Z GITHUB_ACTOR_ID=54816060 2025-04-21T21:51:21.8816567Z GITHUB_WORKFLOW_SHA=b68c0effe97351b0c24b2f7aef671559be6aa977 2025-04-21T21:51:21.8816947Z GITHUB_REF_NAME=ciflow/inductor/148893 2025-04-21T21:51:21.8817361Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2025-04-21T21:51:21.8817753Z GITHUB_JOB=test 2025-04-21T21:51:21.8818001Z NV_LIBNCCL_PACKAGE=libnccl2=2.23.4-1+cuda12.6 2025-04-21T21:51:21.8818385Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2025-04-21T21:51:21.8818737Z NO_TEST_TIMEOUT=False 2025-04-21T21:51:21.8818974Z TD_DISTRIBUTED=False 2025-04-21T21:51:21.8819229Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.6.3-1 2025-04-21T21:51:21.8819529Z GITHUB_REPOSITORY=pytorch/pytorch 2025-04-21T21:51:21.8819813Z NV_NVPROF_VERSION=12.6.80-1 2025-04-21T21:51:21.8820073Z GITHUB_RETENTION_DAYS=90 2025-04-21T21:51:21.8820321Z OPENSSL_DIR=/opt/openssl 2025-04-21T21:51:21.8820573Z GITHUB_ACTION_REPOSITORY= 2025-04-21T21:51:21.8821296Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2025-04-21T21:51:21.8822025Z GITHUB_BASE_REF= 2025-04-21T21:51:21.8822422Z ARTIFACTS_FILE_SUFFIX=test-inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_40900324325 2025-04-21T21:51:21.8822900Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2025-04-21T21:51:21.8823160Z CI=true 2025-04-21T21:51:21.8823365Z NV_LIBNCCL_PACKAGE_VERSION=2.23.4-1 2025-04-21T21:51:21.8824763Z GITHUB_REPOSITORY_OWNER=pytorch 2025-04-21T21:51:21.8825032Z JOB_ID=40900324325 2025-04-21T21:51:21.8825256Z INSTALLED_PROTOBUF=yes 2025-04-21T21:51:21.8825588Z GITHUB_HEAD_REF= 2025-04-21T21:51:21.8825815Z GITHUB_ACTION_REF= 2025-04-21T21:51:21.8826085Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2025-04-21T21:51:21.8826419Z TEST_SHOWLOCALS=False 2025-04-21T21:51:21.8826668Z GITHUB_WORKFLOW=inductor 2025-04-21T21:51:21.8826925Z DEBIAN_FRONTEND=noninteractive 2025-04-21T21:51:21.8827503Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_c92d17c4-5f5b-479e-a8d4-6c632516b6b5 2025-04-21T21:51:21.8828086Z NO_TD=False 2025-04-21T21:51:21.8828310Z SKIP_SCCACHE_INITIALIZATION=1 2025-04-21T21:51:21.8828600Z NCCL_INCLUDE_DIR=/usr/local/cuda/include/ 2025-04-21T21:51:21.8828889Z _=/usr/bin/env 2025-04-21T21:51:21.8829192Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2025-04-21T21:51:21.8959663Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2025-04-21T21:51:21.8960423Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2025-04-21T21:51:21.8960985Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2025-04-21T21:51:21.8961659Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2025-04-21T21:51:21.8962359Z + BUILD_DIR=build 2025-04-21T21:51:21.8962619Z + BUILD_RENAMED_DIR=build_renamed 2025-04-21T21:51:21.8962912Z + BUILD_BIN_DIR=build/bin 2025-04-21T21:51:21.8963148Z + SHARD_NUMBER=2 2025-04-21T21:51:21.8963376Z + NUM_TEST_SHARDS=2 2025-04-21T21:51:21.8963621Z + export TORCH_SERIALIZATION_DEBUG=1 2025-04-21T21:51:21.8963916Z + TORCH_SERIALIZATION_DEBUG=1 2025-04-21T21:51:21.8964172Z + export VALGRIND=ON 2025-04-21T21:51:21.8964399Z + VALGRIND=ON 2025-04-21T21:51:21.8964689Z + [[ linux-focal-cuda12.6-py3.10-gcc9-sm86 == *clang9* ]] 2025-04-21T21:51:21.8965099Z + [[ linux-focal-cuda12.6-py3.10-gcc9-sm86 == *xpu* ]] 2025-04-21T21:51:21.8965538Z + [[ linux-focal-cuda12.6-py3.10-gcc9-sm86 == *s390x* ]] 2025-04-21T21:51:21.8965939Z + [[ 0 == \1 ]] 2025-04-21T21:51:21.8966193Z + [[ False == \1 ]] 2025-04-21T21:51:21.8966477Z + [[ linux-focal-cuda12.6-py3.10-gcc9-sm86 != *bazel* ]] 2025-04-21T21:51:21.8967682Z ++ realpath build/custom_test_artifacts 2025-04-21T21:51:21.8979387Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2025-04-21T21:51:21.8992184Z + [[ -n '' ]] 2025-04-21T21:51:21.8992689Z + echo 'Environment variables' 2025-04-21T21:51:21.8993039Z Environment variables 2025-04-21T21:51:21.8993317Z + env 2025-04-21T21:51:21.8993570Z NV_LIBCUBLAS_VERSION=12.6.4.1-1 2025-04-21T21:51:21.8993852Z NVIDIA_VISIBLE_DEVICES=all 2025-04-21T21:51:21.8994125Z NV_NVML_DEV_VERSION=12.6.77-1 2025-04-21T21:51:21.8994556Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2025-04-21T21:51:21.8995071Z CONTINUE_THROUGH_ERROR=False 2025-04-21T21:51:21.8995394Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.23.4-1+cuda12.6 2025-04-21T21:51:21.8995765Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.23.4-1 2025-04-21T21:51:21.8996167Z BUILD_ENVIRONMENT=linux-focal-cuda12.6-py3.10-gcc9-sm86 2025-04-21T21:51:21.8996726Z HOSTNAME=f3774c702cf4 2025-04-21T21:51:21.8997273Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_c92d17c4-5f5b-479e-a8d4-6c632516b6b5 2025-04-21T21:51:21.8997852Z GITHUB_ACTION=__self 2025-04-21T21:51:21.8998112Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2025-04-21T21:51:21.9003343Z NVIDIA_REQUIRE_CUDA=cuda>=12.6 brand=unknown,driver>=470,driver<471 brand=grid,driver>=470,driver<471 brand=tesla,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=vapps,driver>=470,driver<471 brand=vpc,driver>=470,driver<471 brand=vcs,driver>=470,driver<471 brand=vws,driver>=470,driver<471 brand=cloudgaming,driver>=470,driver<471 brand=unknown,driver>=535,driver<536 brand=grid,driver>=535,driver<536 brand=tesla,driver>=535,driver<536 brand=nvidia,driver>=535,driver<536 brand=quadro,driver>=535,driver<536 brand=quadrortx,driver>=535,driver<536 brand=nvidiartx,driver>=535,driver<536 brand=vapps,driver>=535,driver<536 brand=vpc,driver>=535,driver<536 brand=vcs,driver>=535,driver<536 brand=vws,driver>=535,driver<536 brand=cloudgaming,driver>=535,driver<536 brand=unknown,driver>=550,driver<551 brand=grid,driver>=550,driver<551 brand=tesla,driver>=550,driver<551 brand=nvidia,driver>=550,driver<551 brand=quadro,driver>=550,driver<551 brand=quadrortx,driver>=550,driver<551 brand=nvidiartx,driver>=550,driver<551 brand=vapps,driver>=550,driver<551 brand=vpc,driver>=550,driver<551 brand=vcs,driver>=550,driver<551 brand=vws,driver>=550,driver<551 brand=cloudgaming,driver>=550,driver<551 2025-04-21T21:51:21.9008688Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-6=12.6.4.1-1 2025-04-21T21:51:21.9009040Z NV_NVTX_VERSION=12.6.77-1 2025-04-21T21:51:21.9009298Z GITHUB_RUN_NUMBER=128203 2025-04-21T21:51:21.9009780Z TEST_CONFIG=inductor_timm 2025-04-21T21:51:21.9010064Z GITHUB_REPOSITORY_OWNER_ID=21003710 2025-04-21T21:51:21.9010447Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2025-04-21T21:51:21.9010755Z IS_A100_RUNNER=0 2025-04-21T21:51:21.9010993Z NV_CUDA_CUDART_DEV_VERSION=12.6.77-1 2025-04-21T21:51:21.9011290Z NV_LIBCUSPARSE_VERSION=12.5.4.2-1 2025-04-21T21:51:21.9011900Z SCRIBE_GRAPHQL_ACCESS_TOKEN=*** 2025-04-21T21:51:21.9012184Z NV_LIBNPP_VERSION=12.3.1.54-1 2025-04-21T21:51:21.9012476Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2025-04-21T21:51:21.9012828Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2025-04-21T21:51:21.9013156Z GITHUB_REF_TYPE=tag 2025-04-21T21:51:21.9013393Z TORCH_CUDA_ARCH_LIST=Maxwell 2025-04-21T21:51:21.9013664Z NCCL_VERSION=2.23.4-1 2025-04-21T21:51:21.9013935Z BASE_SHA=b68c0effe97351b0c24b2f7aef671559be6aa977 2025-04-21T21:51:21.9014247Z XLA_CUDA= 2025-04-21T21:51:21.9014465Z NCCL_LIB_DIR=/usr/local/cuda/lib64/ 2025-04-21T21:51:21.9014860Z HUGGING_FACE_HUB_TOKEN=*** 2025-04-21T21:51:21.9015152Z *** 2025-04-21T21:51:21.9015359Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2025-04-21T21:51:21.9015644Z GITHUB_REPOSITORY_ID=65600975 2025-04-21T21:51:21.9015898Z GITHUB_ACTIONS=true 2025-04-21T21:51:21.9016138Z NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T21:51:21.9016454Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-6=12.6.80-1 2025-04-21T21:51:21.9016800Z NV_LIBNPP_PACKAGE=libnpp-12-6=12.3.1.54-1 2025-04-21T21:51:21.9017140Z SHA1=b68c0effe97351b0c24b2f7aef671559be6aa977 2025-04-21T21:51:21.9017543Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2025-04-21T21:51:21.9017918Z GITHUB_SHA=b68c0effe97351b0c24b2f7aef671559be6aa977 2025-04-21T21:51:21.9018480Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor.yml@refs/tags/ciflow/inductor/148893 2025-04-21T21:51:21.9018993Z UCC_HOME=/usr 2025-04-21T21:51:21.9019220Z TORCH_SERIALIZATION_DEBUG=1 2025-04-21T21:51:21.9019491Z NV_LIBCUBLAS_DEV_VERSION=12.6.4.1-1 2025-04-21T21:51:21.9019776Z VERBOSE_TEST_LOGS=False 2025-04-21T21:51:21.9020026Z NVIDIA_PRODUCT_NAME=CUDA 2025-04-21T21:51:21.9020321Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-6 2025-04-21T21:51:21.9020679Z GITHUB_REF=refs/tags/ciflow/inductor/148893 2025-04-21T21:51:21.9020997Z NV_CUDA_CUDART_VERSION=12.6.77-1 2025-04-21T21:51:21.9021261Z SHARD_NUMBER=2 2025-04-21T21:51:21.9021488Z GITHUB_REF_PROTECTED=false 2025-04-21T21:51:21.9021752Z HOME=/var/lib/jenkins 2025-04-21T21:51:21.9022018Z GITHUB_API_URL=https://api.github.com 2025-04-21T21:51:21.9022339Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2025-04-21T21:51:21.9022674Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2025-04-21T21:51:21.9023002Z CUDA_VERSION=12.6.3 2025-04-21T21:51:21.9023260Z NV_LIBCUBLAS_PACKAGE=libcublas-12-6=12.6.4.1-1 2025-04-21T21:51:21.9023572Z USE_SYSTEM_NCCL=1 2025-04-21T21:51:21.9023796Z NUM_TEST_SHARDS=2 2025-04-21T21:51:21.9024005Z UCX_HOME=/usr 2025-04-21T21:51:21.9024333Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-6=12.6.3-1 2025-04-21T21:51:21.9025036Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_c92d17c4-5f5b-479e-a8d4-6c632516b6b5 2025-04-21T21:51:21.9026037Z JOB_NAME=cuda12.6-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2025-04-21T21:51:21.9026787Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_c92d17c4-5f5b-479e-a8d4-6c632516b6b5 2025-04-21T21:51:21.9027528Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2025-04-21T21:51:21.9028011Z GITHUB_EVENT_NAME=push 2025-04-21T21:51:21.9028251Z DASHBOARD_TAG= 2025-04-21T21:51:21.9028464Z GITHUB_RUN_ID=14580756816 2025-04-21T21:51:21.9028753Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-6=12.3.1.54-1 2025-04-21T21:51:21.9029095Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-6 2025-04-21T21:51:21.9029733Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_c92d17c4-5f5b-479e-a8d4-6c632516b6b5 2025-04-21T21:51:21.9030363Z GITHUB_ACTOR=pytorch-bot[bot] 2025-04-21T21:51:21.9030640Z NV_LIBNPP_DEV_VERSION=12.3.1.54-1 2025-04-21T21:51:21.9030954Z PR_NUMBER= 2025-04-21T21:51:21.9031221Z GITHUB_RUN_ATTEMPT=1 2025-04-21T21:51:21.9031448Z VALGRIND=ON 2025-04-21T21:51:21.9031658Z ANACONDA_PYTHON_VERSION=3.10 2025-04-21T21:51:21.9031973Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2025-04-21T21:51:21.9032303Z TERM=vt100 2025-04-21T21:51:21.9032630Z NV_LIBCUSPARSE_DEV_VERSION=12.5.4.2-1 2025-04-21T21:51:21.9032912Z INSTALLED_VISION=yes 2025-04-21T21:51:21.9033135Z BRANCH= 2025-04-21T21:51:21.9033338Z SCCACHE_REGION=us-east-1 2025-04-21T21:51:21.9033588Z OPENSSL_ROOT_DIR=/opt/openssl 2025-04-21T21:51:21.9033872Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2025-04-21T21:51:21.9034169Z CUDA_PATH=/usr/local/cuda 2025-04-21T21:51:21.9034648Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2025-04-21T21:51:21.9035220Z GITHUB_SERVER_URL=https://github.com 2025-04-21T21:51:21.9035584Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2025-04-21T21:51:21.9035902Z REENABLED_ISSUES= 2025-04-21T21:51:21.9036119Z SHLVL=1 2025-04-21T21:51:21.9036318Z MAX_JOBS=14 2025-04-21T21:51:21.9036528Z NV_CUDA_LIB_VERSION=12.6.3-1 2025-04-21T21:51:21.9036775Z NVARCH=x86_64 2025-04-21T21:51:21.9036988Z GITHUB_ACTOR_ID=54816060 2025-04-21T21:51:21.9037309Z GITHUB_WORKFLOW_SHA=b68c0effe97351b0c24b2f7aef671559be6aa977 2025-04-21T21:51:21.9037693Z GITHUB_REF_NAME=ciflow/inductor/148893 2025-04-21T21:51:21.9038108Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2025-04-21T21:51:21.9038505Z GITHUB_JOB=test 2025-04-21T21:51:21.9038750Z NV_LIBNCCL_PACKAGE=libnccl2=2.23.4-1+cuda12.6 2025-04-21T21:51:21.9039137Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2025-04-21T21:51:21.9039492Z NO_TEST_TIMEOUT=False 2025-04-21T21:51:21.9039722Z TD_DISTRIBUTED=False 2025-04-21T21:51:21.9039975Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.6.3-1 2025-04-21T21:51:21.9040285Z GITHUB_REPOSITORY=pytorch/pytorch 2025-04-21T21:51:21.9040565Z NV_NVPROF_VERSION=12.6.80-1 2025-04-21T21:51:21.9040836Z GITHUB_RETENTION_DAYS=90 2025-04-21T21:51:21.9041097Z OPENSSL_DIR=/opt/openssl 2025-04-21T21:51:21.9041344Z GITHUB_ACTION_REPOSITORY= 2025-04-21T21:51:21.9042051Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2025-04-21T21:51:21.9042784Z GITHUB_BASE_REF= 2025-04-21T21:51:21.9043190Z ARTIFACTS_FILE_SUFFIX=test-inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_40900324325 2025-04-21T21:51:21.9043659Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2025-04-21T21:51:21.9043925Z CI=true 2025-04-21T21:51:21.9044139Z NV_LIBNCCL_PACKAGE_VERSION=2.23.4-1 2025-04-21T21:51:21.9044430Z GITHUB_REPOSITORY_OWNER=pytorch 2025-04-21T21:51:21.9044688Z JOB_ID=40900324325 2025-04-21T21:51:21.9044914Z INSTALLED_PROTOBUF=yes 2025-04-21T21:51:21.9045145Z GITHUB_HEAD_REF= 2025-04-21T21:51:21.9045367Z GITHUB_ACTION_REF= 2025-04-21T21:51:21.9045650Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2025-04-21T21:51:21.9045981Z TEST_SHOWLOCALS=False 2025-04-21T21:51:21.9046338Z GITHUB_WORKFLOW=inductor 2025-04-21T21:51:21.9046601Z DEBIAN_FRONTEND=noninteractive 2025-04-21T21:51:21.9047178Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_c92d17c4-5f5b-479e-a8d4-6c632516b6b5 2025-04-21T21:51:21.9047757Z NO_TD=False 2025-04-21T21:51:21.9047986Z SKIP_SCCACHE_INITIALIZATION=1 2025-04-21T21:51:21.9048287Z NCCL_INCLUDE_DIR=/usr/local/cuda/include/ 2025-04-21T21:51:21.9048574Z _=/usr/bin/env 2025-04-21T21:51:21.9048814Z + echo 'Testing pytorch' 2025-04-21T21:51:21.9049059Z Testing pytorch 2025-04-21T21:51:21.9049279Z + export LANG=C.UTF-8 2025-04-21T21:51:21.9049509Z + LANG=C.UTF-8 2025-04-21T21:51:21.9049719Z + PR_NUMBER= 2025-04-21T21:51:21.9049939Z + [[ inductor_timm == \d\e\f\a\u\l\t ]] 2025-04-21T21:51:21.9050261Z + [[ inductor_timm == \d\i\s\t\r\i\b\u\t\e\d ]] 2025-04-21T21:51:21.9050574Z + [[ inductor_timm == \s\l\o\w ]] 2025-04-21T21:51:21.9050947Z + [[ linux-focal-cuda12.6-py3.10-gcc9-sm86 == *slow-gradcheck* ]] 2025-04-21T21:51:21.9051394Z + [[ linux-focal-cuda12.6-py3.10-gcc9-sm86 == *cuda* ]] 2025-04-21T21:51:21.9051765Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2025-04-21T21:51:21.9052087Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2025-04-21T21:51:21.9052376Z + [[ inductor_timm == *crossref* ]] 2025-04-21T21:51:21.9052800Z + [[ linux-focal-cuda12.6-py3.10-gcc9-sm86 == *rocm* ]] 2025-04-21T21:51:21.9053192Z + [[ linux-focal-cuda12.6-py3.10-gcc9-sm86 == *xpu* ]] 2025-04-21T21:51:21.9053589Z + [[ linux-focal-cuda12.6-py3.10-gcc9-sm86 != *-bazel-* ]] 2025-04-21T21:51:21.9053957Z + pip_install --user ninja==1.10.2 2025-04-21T21:51:21.9054333Z + pip_install_pkg='python3 -m pip install --progress-bar off' 2025-04-21T21:51:21.9054794Z + python3 -m pip install --progress-bar off --user ninja==1.10.2 2025-04-21T21:51:22.3737506Z Collecting ninja==1.10.2 2025-04-21T21:51:22.3918076Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2025-04-21T21:51:22.4031941Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2025-04-21T21:51:23.0014492Z Installing collected packages: ninja 2025-04-21T21:51:23.0093443Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2025-04-21T21:51:23.0094475Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2025-04-21T21:51:23.0162210Z Successfully installed ninja-1.10.2 2025-04-21T21:51:23.0951046Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2025-04-21T21:51:23.0952514Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2025-04-21T21:51:23.0953413Z + [[ linux-focal-cuda12.6-py3.10-gcc9-sm86 == *aarch64* ]] 2025-04-21T21:51:23.0953800Z + install_tlparse 2025-04-21T21:51:23.0954058Z + pip_install --user tlparse==0.3.30 2025-04-21T21:51:23.0954434Z + pip_install_pkg='python3 -m pip install --progress-bar off' 2025-04-21T21:51:23.0954891Z + python3 -m pip install --progress-bar off --user tlparse==0.3.30 2025-04-21T21:51:23.5109766Z Collecting tlparse==0.3.30 2025-04-21T21:51:23.5278796Z Downloading tlparse-0.3.30-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.9 kB) 2025-04-21T21:51:23.5412635Z Downloading tlparse-0.3.30-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.3 MB) 2025-04-21T21:51:24.1609314Z Installing collected packages: tlparse 2025-04-21T21:51:24.1969877Z Successfully installed tlparse-0.3.30 2025-04-21T21:51:24.2779718Z ++ python -m site --user-base 2025-04-21T21:51:24.2977977Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2025-04-21T21:51:24.2979397Z + [[ linux-focal-cuda12.6-py3.10-gcc9-sm86 == *asan* ]] 2025-04-21T21:51:24.2979807Z + [[ linux-focal-cuda12.6-py3.10-gcc9-sm86 == *-debug* ]] 2025-04-21T21:51:24.2980229Z + [[ linux-focal-cuda12.6-py3.10-gcc9-sm86 != *-bazel-* ]] 2025-04-21T21:51:24.2980807Z + echo 'We are not in debug mode: linux-focal-cuda12.6-py3.10-gcc9-sm86. Expect the assertion to pass' 2025-04-21T21:51:24.2981501Z We are not in debug mode: linux-focal-cuda12.6-py3.10-gcc9-sm86. Expect the assertion to pass 2025-04-21T21:51:24.2983699Z + cd test 2025-04-21T21:51:24.2984519Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2025-04-21T21:51:26.0128845Z + [[ inductor_timm == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2025-04-21T21:51:26.0129235Z + [[ inductor_timm == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2025-04-21T21:51:26.0135672Z + DYNAMO_BENCHMARK_FLAGS=() 2025-04-21T21:51:26.0136067Z + [[ inductor_timm == *pr_time_benchmarks* ]] 2025-04-21T21:51:26.0136446Z + [[ inductor_timm == *dynamo_eager* ]] 2025-04-21T21:51:26.0136749Z + [[ inductor_timm == *aot_eager* ]] 2025-04-21T21:51:26.0137097Z + [[ inductor_timm == *aot_inductor* ]] 2025-04-21T21:51:26.0137437Z + [[ inductor_timm == *max_autotune_inductor* ]] 2025-04-21T21:51:26.0137782Z + [[ inductor_timm == *inductor* ]] 2025-04-21T21:51:26.0138390Z + [[ inductor_timm != *perf* ]] 2025-04-21T21:51:26.0138714Z + DYNAMO_BENCHMARK_FLAGS+=(--inductor) 2025-04-21T21:51:26.0139009Z + [[ inductor_timm == *dynamic* ]] 2025-04-21T21:51:26.0139289Z + [[ inductor_timm == *cpu* ]] 2025-04-21T21:51:26.0139570Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2025-04-21T21:51:26.0175326Z + [[ linux-focal-cuda12.6-py3.10-gcc9-sm86 == *libtorch* ]] 2025-04-21T21:51:26.0175764Z + [[ linux-focal-cuda12.6-py3.10-gcc9-sm86 == *-bazel-* ]] 2025-04-21T21:51:26.0179013Z + cd test 2025-04-21T21:51:26.0179450Z + python -c 'import torch; print(torch.__config__.show())' 2025-04-21T21:51:27.5292061Z PyTorch built with: 2025-04-21T21:51:27.5292338Z - GCC 9.4 2025-04-21T21:51:27.5292562Z - C++ Version: 201703 2025-04-21T21:51:27.5293178Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2025-04-21T21:51:27.5294087Z - Intel(R) MKL-DNN v3.7.1 (Git Hash 8d263e693366ef8db40acc569cc7d8edf644556d) 2025-04-21T21:51:27.5294640Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2025-04-21T21:51:27.5294960Z - LAPACK is enabled (usually provided by MKL) 2025-04-21T21:51:27.5295260Z - NNPACK is enabled 2025-04-21T21:51:27.5295687Z - CPU capability usage: AVX2 2025-04-21T21:51:27.5295954Z - CUDA Runtime 12.6 2025-04-21T21:51:27.5296277Z - NVCC architecture flags: -gencode;arch=compute_86,code=sm_86 2025-04-21T21:51:27.5296637Z - CuDNN 90.5.1 2025-04-21T21:51:27.5296852Z - Magma 2.6.1 2025-04-21T21:51:27.5301027Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, COMMIT_SHA=b68c0effe97351b0c24b2f7aef671559be6aa977, CUDA_VERSION=12.6, CUDNN_VERSION=9.5.1, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Werror -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, TORCH_VERSION=2.8.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2025-04-21T21:51:27.5305257Z 2025-04-21T21:51:27.9296718Z + cd test 2025-04-21T21:51:27.9297078Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2025-04-21T21:51:29.3305085Z ATen/Parallel: 2025-04-21T21:51:29.3305598Z at::get_num_threads() : 8 2025-04-21T21:51:29.3305986Z at::get_num_interop_threads() : 16 2025-04-21T21:51:29.3306385Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2025-04-21T21:51:29.3306783Z omp_get_max_threads() : 8 2025-04-21T21:51:29.3307349Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2025-04-21T21:51:29.3307887Z mkl_get_max_threads() : 8 2025-04-21T21:51:29.3308253Z Intel(R) MKL-DNN v3.7.1 (Git Hash 8d263e693366ef8db40acc569cc7d8edf644556d) 2025-04-21T21:51:29.3308678Z std::thread::hardware_concurrency() : 16 2025-04-21T21:51:29.3308974Z Environment variables: 2025-04-21T21:51:29.3309222Z OMP_NUM_THREADS : [not set] 2025-04-21T21:51:29.3309487Z MKL_NUM_THREADS : [not set] 2025-04-21T21:51:29.3309773Z ATen parallel backend: OpenMP 2025-04-21T21:51:29.3309991Z 2025-04-21T21:51:29.6317684Z + [[ inductor_timm == *numpy_2* ]] 2025-04-21T21:51:29.6318070Z + [[ linux-focal-cuda12.6-py3.10-gcc9-sm86 == *aarch64* ]] 2025-04-21T21:51:29.6318461Z + [[ inductor_timm == *backward* ]] 2025-04-21T21:51:29.6318751Z + [[ inductor_timm == *xla* ]] 2025-04-21T21:51:29.6319023Z + [[ inductor_timm == *executorch* ]] 2025-04-21T21:51:29.6319570Z + [[ inductor_timm == \j\i\t\_\l\e\g\a\c\y ]] 2025-04-21T21:51:29.6319947Z + [[ linux-focal-cuda12.6-py3.10-gcc9-sm86 == *libtorch* ]] 2025-04-21T21:51:29.6320318Z + [[ inductor_timm == distributed ]] 2025-04-21T21:51:29.6320621Z + [[ inductor_timm == *operator_benchmark* ]] 2025-04-21T21:51:29.6320958Z + [[ inductor_timm == *inductor_distributed* ]] 2025-04-21T21:51:29.6321315Z + [[ inductor_timm == *inductor-halide* ]] 2025-04-21T21:51:29.6321772Z + [[ inductor_timm == *inductor-triton-cpu* ]] 2025-04-21T21:51:29.6322237Z + [[ inductor_timm == *inductor-micro-benchmark* ]] 2025-04-21T21:51:29.6322661Z + [[ inductor_timm == *huggingface* ]] 2025-04-21T21:51:29.6323061Z + [[ inductor_timm == *timm* ]] 2025-04-21T21:51:29.6323431Z + install_torchvision 2025-04-21T21:51:29.6323750Z + local orig_preload 2025-04-21T21:51:29.6323972Z + local commit 2025-04-21T21:51:29.6324205Z ++ get_pinned_commit vision 2025-04-21T21:51:29.6324477Z ++ cat .github/ci_commit_pins/vision.txt 2025-04-21T21:51:29.6343865Z + commit=d23a6e1664d20707c11781299611436e1f0c104f 2025-04-21T21:51:29.6344226Z + orig_preload= 2025-04-21T21:51:29.6344446Z + '[' -n '' ']' 2025-04-21T21:51:29.6344968Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2025-04-21T21:51:29.6345717Z + pip_install_pkg='python3 -m pip install --progress-bar off' 2025-04-21T21:51:29.6346477Z + python3 -m pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2025-04-21T21:51:29.9831942Z Collecting git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2025-04-21T21:51:29.9835681Z Cloning https://github.com/pytorch/vision.git (to revision d23a6e1664d20707c11781299611436e1f0c104f) to /tmp/pip-req-build-03qxyvh8 2025-04-21T21:51:29.9866107Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-03qxyvh8 2025-04-21T21:51:31.4867200Z Running command git rev-parse -q --verify 'sha^d23a6e1664d20707c11781299611436e1f0c104f' 2025-04-21T21:51:31.4898102Z Running command git fetch -q https://github.com/pytorch/vision.git d23a6e1664d20707c11781299611436e1f0c104f 2025-04-21T21:51:32.9153730Z Running command git checkout -q d23a6e1664d20707c11781299611436e1f0c104f 2025-04-21T21:51:33.2873049Z Resolved https://github.com/pytorch/vision.git to commit d23a6e1664d20707c11781299611436e1f0c104f 2025-04-21T21:51:35.9805872Z Preparing metadata (setup.py) ... [?25l- \ done 2025-04-21T21:51:35.9839569Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (1.22.4) 2025-04-21T21:51:35.9841742Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (2.8.0a0+gitb68c0ef) 2025-04-21T21:51:35.9846667Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (11.0.0) 2025-04-21T21:51:35.9913938Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.16.1) 2025-04-21T21:51:35.9915826Z Requirement already satisfied: typing-extensions>=4.10.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (4.13.2) 2025-04-21T21:51:35.9919444Z Requirement already satisfied: sympy>=1.13.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (1.13.3) 2025-04-21T21:51:35.9922292Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.8.8) 2025-04-21T21:51:35.9925075Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.1.6) 2025-04-21T21:51:35.9928598Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2024.10.0) 2025-04-21T21:51:35.9942012Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy>=1.13.3->torch->torchvision==0.19.0a0+d23a6e1) (1.3.0) 2025-04-21T21:51:36.0415215Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.19.0a0+d23a6e1) (3.0.2) 2025-04-21T21:51:36.0480705Z Building wheels for collected packages: torchvision 2025-04-21T21:52:53.6050357Z Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - done 2025-04-21T21:52:53.6083794Z [?25h Created wheel for torchvision: filename=torchvision-0.19.0a0+d23a6e1-cp310-cp310-linux_x86_64.whl size=2077339 sha256=3f490d8b327ca8e4dcbba50356cc70bb336d2bec024e1e81cb51eda034dbf973 2025-04-21T21:52:53.6084946Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/0e/56/35/02931e71eb23fd2b85591c7ec05b733ca7c8b328a2fd151f96 2025-04-21T21:52:53.6120500Z Successfully built torchvision 2025-04-21T21:52:54.1350410Z Installing collected packages: torchvision 2025-04-21T21:52:54.5480775Z Successfully installed torchvision-0.19.0a0+d23a6e1 2025-04-21T21:52:54.6922401Z + '[' -n '' ']' 2025-04-21T21:52:54.6922659Z + id=1 2025-04-21T21:52:54.6922899Z + test_dynamo_benchmark timm_models 1 2025-04-21T21:52:54.6925815Z ++ pwd 2025-04-21T21:52:54.6928698Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2025-04-21T21:52:54.6929132Z + local suite=timm_models 2025-04-21T21:52:54.6929368Z + shift 2025-04-21T21:52:54.6929554Z + local shard_id=1 2025-04-21T21:52:54.6929766Z + shift 2025-04-21T21:52:54.6929979Z + [[ inductor_timm == *perf_compare* ]] 2025-04-21T21:52:54.6930266Z + [[ inductor_timm == *perf* ]] 2025-04-21T21:52:54.6930548Z + [[ inductor_timm == *cpu* ]] 2025-04-21T21:52:54.6930818Z + [[ inductor_timm == *aot_inductor* ]] 2025-04-21T21:52:54.6931125Z + [[ inductor_timm == *max_autotune_inductor* ]] 2025-04-21T21:52:54.6931570Z + test_single_dynamo_benchmark inference timm_models 1 --inference --bfloat16 2025-04-21T21:52:54.6933765Z ++ pwd 2025-04-21T21:52:54.6936386Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2025-04-21T21:52:54.6936813Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports 2025-04-21T21:52:54.7003428Z + local name=inference 2025-04-21T21:52:54.7003802Z + shift 2025-04-21T21:52:54.7004125Z + local suite=timm_models 2025-04-21T21:52:54.7004778Z + shift 2025-04-21T21:52:54.7004971Z + local shard_id=1 2025-04-21T21:52:54.7005188Z + shift 2025-04-21T21:52:54.7005380Z + partition_flags=() 2025-04-21T21:52:54.7005622Z + local partition_flags 2025-04-21T21:52:54.7006230Z + [[ -n 2 ]] 2025-04-21T21:52:54.7006429Z + [[ -n 1 ]] 2025-04-21T21:52:54.7006821Z + partition_flags=(--total-partitions "$NUM_TEST_SHARDS" --partition-id "$shard_id") 2025-04-21T21:52:54.7007305Z + [[ inductor_timm == *perf_compare* ]] 2025-04-21T21:52:54.7007606Z + [[ inductor_timm == *perf* ]] 2025-04-21T21:52:54.7007878Z + [[ inductor_timm == *_avx2* ]] 2025-04-21T21:52:54.7008151Z + [[ inductor_timm == *_avx512* ]] 2025-04-21T21:52:54.7009266Z + python benchmarks/dynamo/timm_models.py --ci --accuracy --timing --explain --print-compilation-time --inductor --device cuda --inference --bfloat16 --total-partitions 2 --partition-id 1 --output /var/lib/jenkins/workspace/test/test-reports/inference_timm_models.csv 2025-04-21T21:53:01.8895639Z 2025-04-21T21:53:02.0431489Z loading model: 0it [00:00, ?it/s] 2025-04-21T21:53:02.0431741Z 2025-04-21T21:53:02.1305911Z model.safetensors: 0% 0.00/29.6M [00:00 will be ignored 2025-04-21T22:13:34.1867020Z Compilation time (from dynamo_timed): 70.964275129 2025-04-21T22:13:34.1896703Z pass 2025-04-21T22:13:34.1947821Z TIMING: entire_frame_compile:61.31175 gc:0.00545 _recursive_pre_grad_passes:0.02903 pad_mm_benchmark:0.00198 _recursive_joint_graph_passes:0.48945 _recursive_post_grad_passes:0.59363 async_compile.wait:3.64884 code_gen:22.88558 inductor_compile:37.90734 backend_compile:47.99909 cudagraphify.get_container:0.46615 entire_backward_compile:9.65253 CUDAGraphNode.record:4.17041 total_wall_time:70.96428 2025-04-21T22:13:34.1949981Z STATS: call_* op count: 1018 | FakeTensorMode.__torch_dispatch__:44671 | FakeTensor.__torch_dispatch__:36780 | ProxyTorchDispatchMode.__torch_dispatch__:18964 2025-04-21T22:13:34.1950797Z Dynamo produced 4 graphs covering 1018 ops with 7 graph breaks (5 unique) 2025-04-21T22:13:42.2086179Z 2025-04-21T22:13:43.0944790Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:13:43.0945525Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:13:43.0946124Z cuda train mobilenetv2_100 2025-04-21T22:14:24.0434185Z W0421 22:14:24.042000 23518 site-packages/torch/_logging/_internal.py:1130] [6/0] Profiler function will be ignored 2025-04-21T22:15:00.4652027Z Compilation time (from dynamo_timed): 71.535543789 2025-04-21T22:15:00.4684282Z pass 2025-04-21T22:15:00.4729979Z TIMING: entire_frame_compile:60.4116 gc:0.00554 _recursive_pre_grad_passes:0.02859 pad_mm_benchmark:0.00199 _recursive_joint_graph_passes:0.57727 _recursive_post_grad_passes:0.68665 async_compile.wait:2.56766 code_gen:22.42697 inductor_compile:38.21325 backend_compile:47.13721 cudagraphify.get_container:0.45713 entire_backward_compile:11.12394 CUDAGraphNode.record:2.6038 total_wall_time:71.53554 2025-04-21T22:15:00.4733490Z STATS: call_* op count: 1017 | FakeTensorMode.__torch_dispatch__:45514 | FakeTensor.__torch_dispatch__:39548 | ProxyTorchDispatchMode.__torch_dispatch__:18980 2025-04-21T22:15:00.4734330Z Dynamo produced 3 graphs covering 1017 ops with 7 graph breaks (5 unique) 2025-04-21T22:15:08.3928570Z 2025-04-21T22:15:09.4074231Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:15:09.4075226Z loading model: 0it [00:01, ?it/s] 2025-04-21T22:15:09.4075831Z cuda train mobilenetv3_large_100 2025-04-21T22:15:53.9862795Z W0421 22:15:53.985000 25039 site-packages/torch/_logging/_internal.py:1130] [6/0] Profiler function will be ignored 2025-04-21T22:16:36.0545066Z Compilation time (from dynamo_timed): 80.494619122 2025-04-21T22:16:36.0574643Z pass 2025-04-21T22:16:36.0656259Z TIMING: entire_frame_compile:67.27813 gc:0.00406 _recursive_pre_grad_passes:0.03069 pad_mm_benchmark:0.00197 _recursive_joint_graph_passes:0.53208 _recursive_post_grad_passes:1.06229 async_compile.wait:3.38357 code_gen:25.42675 inductor_compile:43.35771 backend_compile:51.68739 cudagraphify.get_container:0.4779 entire_backward_compile:13.21649 CUDAGraphNode.record:2.82607 total_wall_time:80.49462 2025-04-21T22:16:36.0658118Z STATS: call_* op count: 1125 | FakeTensorMode.__torch_dispatch__:49665 | FakeTensor.__torch_dispatch__:41717 | ProxyTorchDispatchMode.__torch_dispatch__:20964 2025-04-21T22:16:36.0658912Z Dynamo produced 3 graphs covering 1125 ops with 7 graph breaks (5 unique) 2025-04-21T22:16:44.3880551Z 2025-04-21T22:16:45.4440866Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:16:45.4441246Z loading model: 0it [00:01, ?it/s] 2025-04-21T22:16:45.4441541Z cuda train mobilevit_s 2025-04-21T22:17:55.2398539Z Compilation time (from dynamo_timed): 63.679478962999994 2025-04-21T22:17:55.2428903Z pass 2025-04-21T22:17:55.2518736Z TIMING: entire_frame_compile:34.97633 gc:0.00296 _recursive_pre_grad_passes:0.00976 pad_mm_benchmark:0.0682 _recursive_joint_graph_passes:0.78482 _recursive_post_grad_passes:0.78984 async_compile.wait:2.3064 code_gen:31.75158 inductor_compile:45.61872 backend_compile:27.84642 cudagraphify.get_container:0.49652 CachingAutotuner.benchmark_all_configs:0.06615 entire_backward_compile:28.70314 CUDAGraphNode.record:2.60427 total_wall_time:63.67948 2025-04-21T22:17:55.2520768Z STATS: call_* op count: 363 | FakeTensorMode.__torch_dispatch__:40204 | FakeTensor.__torch_dispatch__:25444 | ProxyTorchDispatchMode.__torch_dispatch__:16457 2025-04-21T22:17:55.2521558Z Dynamo produced 3 graphs covering 363 ops with 6 graph breaks (5 unique) 2025-04-21T22:18:02.4717980Z 2025-04-21T22:18:03.8588036Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:18:03.8588378Z loading model: 0it [00:01, ?it/s] 2025-04-21T22:18:03.8588678Z cuda train nfnet_l0 2025-04-21T22:18:47.7546995Z W0421 22:18:47.753000 29407 site-packages/torch/_logging/_internal.py:1130] [6/0] Profiler function will be ignored 2025-04-21T22:19:44.5122521Z Compilation time (from dynamo_timed): 92.963210226 2025-04-21T22:19:44.5146709Z pass 2025-04-21T22:19:44.5388590Z TIMING: entire_frame_compile:79.28825 gc:0.00628 _recursive_pre_grad_passes:0.03911 pad_mm_benchmark:0.00234 _recursive_joint_graph_passes:0.8629 _recursive_post_grad_passes:0.79982 async_compile.wait:3.84852 code_gen:26.09609 inductor_compile:44.3663 backend_compile:57.77165 cudagraphify.get_container:0.41519 entire_backward_compile:13.67496 CUDAGraphNode.record:3.04777 total_wall_time:92.96321 2025-04-21T22:19:44.5390445Z STATS: call_* op count: 1649 | FakeTensorMode.__torch_dispatch__:63948 | FakeTensor.__torch_dispatch__:46413 | ProxyTorchDispatchMode.__torch_dispatch__:27724 2025-04-21T22:19:44.5391277Z Dynamo produced 3 graphs covering 1649 ops with 7 graph breaks (5 unique) 2025-04-21T22:19:52.9393970Z 2025-04-21T22:19:54.1031352Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:19:54.1041210Z loading model: 0it [00:01, ?it/s] 2025-04-21T22:19:54.1041546Z cuda train pit_b_224 2025-04-21T22:20:27.5572458Z Compilation time (from dynamo_timed): 27.348274074000003 2025-04-21T22:20:27.5592126Z pass 2025-04-21T22:20:27.6053625Z TIMING: entire_frame_compile:17.59209 gc:0.00279 _recursive_pre_grad_passes:0.00793 pad_mm_benchmark:0.10295 _recursive_joint_graph_passes:0.84747 _recursive_post_grad_passes:0.55725 async_compile.wait:1.79179 code_gen:9.76044 inductor_compile:16.12815 backend_compile:13.41624 cudagraphify.get_container:0.30089 CachingAutotuner.benchmark_all_configs:0.03258 entire_backward_compile:9.75619 CUDAGraphNode.record:0.85798 total_wall_time:27.34827 2025-04-21T22:20:27.6058716Z STATS: call_* op count: 310 | FakeTensorMode.__torch_dispatch__:31317 | FakeTensor.__torch_dispatch__:14232 | ProxyTorchDispatchMode.__torch_dispatch__:12566 2025-04-21T22:20:27.6059789Z Dynamo produced 2 graphs covering 310 ops with 6 graph breaks (5 unique) 2025-04-21T22:20:33.8272150Z 2025-04-21T22:20:35.8128862Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:20:35.8129205Z loading model: 0it [00:01, ?it/s] 2025-04-21T22:20:35.8129519Z cuda train pnasnet5large 2025-04-21T22:25:12.2164559Z Compilation time (from dynamo_timed): 256.26998026 2025-04-21T22:25:12.2325906Z pass 2025-04-21T22:25:12.3757589Z TIMING: entire_frame_compile:124.25658 gc:0.00362 _recursive_pre_grad_passes:0.02802 pad_mm_benchmark:0.002 _recursive_joint_graph_passes:1.13841 _recursive_post_grad_passes:3.02191 async_compile.wait:3.61189 code_gen:153.08224 inductor_compile:189.21507 backend_compile:94.23558 cudagraphify.get_container:1.78727 CachingAutotuner.benchmark_all_configs:0.0256 entire_backward_compile:132.0134 CUDAGraphNode.record:5.67107 total_wall_time:256.26998 2025-04-21T22:25:12.3759725Z STATS: call_* op count: 1161 | FakeTensorMode.__torch_dispatch__:117119 | FakeTensor.__torch_dispatch__:87487 | ProxyTorchDispatchMode.__torch_dispatch__:49824 2025-04-21T22:25:12.3760536Z Dynamo produced 2 graphs covering 1161 ops with 5 graph breaks (4 unique) 2025-04-21T22:25:25.3358011Z 2025-04-21T22:25:26.8905567Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:25:26.8906037Z loading model: 0it [00:01, ?it/s] 2025-04-21T22:25:26.8906431Z cuda train poolformer_m36 2025-04-21T22:28:33.2570650Z Compilation time (from dynamo_timed): 177.923448543 2025-04-21T22:28:33.2601968Z pass 2025-04-21T22:28:33.3042987Z TIMING: entire_frame_compile:53.75401 gc:0.00381 _recursive_pre_grad_passes:0.01111 pad_mm_benchmark:0.00202 _recursive_joint_graph_passes:1.46608 _recursive_post_grad_passes:1.04207 async_compile.wait:2.09338 code_gen:134.57451 inductor_compile:152.67459 backend_compile:44.37447 cudagraphify.get_container:0.74726 entire_backward_compile:124.16944 CachingAutotuner.benchmark_all_configs:0.03098 CUDAGraphNode.record:2.30881 total_wall_time:177.92345 2025-04-21T22:28:33.3046184Z STATS: call_* op count: 554 | FakeTensorMode.__torch_dispatch__:54743 | FakeTensor.__torch_dispatch__:41997 | ProxyTorchDispatchMode.__torch_dispatch__:22831 2025-04-21T22:28:33.3046995Z Dynamo produced 2 graphs covering 554 ops with 6 graph breaks (5 unique) 2025-04-21T22:28:41.8623834Z 2025-04-21T22:28:43.0527270Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:28:43.0527612Z loading model: 0it [00:01, ?it/s] 2025-04-21T22:28:43.0527902Z cuda train regnety_002 2025-04-21T22:29:22.4036720Z Compilation time (from dynamo_timed): 35.76865012 2025-04-21T22:29:22.4065146Z pass 2025-04-21T22:29:22.4106416Z TIMING: entire_frame_compile:25.32711 gc:0.00353 _recursive_pre_grad_passes:0.00869 pad_mm_benchmark:0.00198 _recursive_joint_graph_passes:0.36587 _recursive_post_grad_passes:0.55654 async_compile.wait:0.92121 code_gen:11.41546 inductor_compile:19.93418 backend_compile:18.45569 cudagraphify.get_container:0.42956 entire_backward_compile:10.44154 CUDAGraphNode.record:1.09271 total_wall_time:35.76865 2025-04-21T22:29:22.4108919Z STATS: call_* op count: 271 | FakeTensorMode.__torch_dispatch__:27839 | FakeTensor.__torch_dispatch__:21345 | ProxyTorchDispatchMode.__torch_dispatch__:12213 2025-04-21T22:29:22.4109733Z Dynamo produced 2 graphs covering 271 ops with 6 graph breaks (5 unique) 2025-04-21T22:29:29.1012523Z 2025-04-21T22:29:30.2588388Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:29:30.2588732Z loading model: 0it [00:01, ?it/s] 2025-04-21T22:29:30.2589039Z cuda train repvgg_a2 2025-04-21T22:30:14.3102179Z W0421 22:30:14.309000 36716 site-packages/torch/_logging/_internal.py:1130] [6/0] Profiler function will be ignored 2025-04-21T22:30:52.6796180Z Compilation time (from dynamo_timed): 75.037401283 2025-04-21T22:30:52.6828848Z pass 2025-04-21T22:30:52.7018507Z TIMING: entire_frame_compile:63.61171 gc:0.00471 _recursive_pre_grad_passes:0.03051 pad_mm_benchmark:0.00198 _recursive_joint_graph_passes:0.51635 _recursive_post_grad_passes:0.65692 async_compile.wait:4.97559 code_gen:24.85109 inductor_compile:40.56425 backend_compile:49.55363 cudagraphify.get_container:0.43026 entire_backward_compile:11.42569 CUDAGraphNode.record:2.62704 total_wall_time:75.0374 2025-04-21T22:30:52.7020372Z STATS: call_* op count: 1094 | FakeTensorMode.__torch_dispatch__:46236 | FakeTensor.__torch_dispatch__:40624 | ProxyTorchDispatchMode.__torch_dispatch__:20322 2025-04-21T22:30:52.7021178Z Dynamo produced 3 graphs covering 1094 ops with 7 graph breaks (5 unique) 2025-04-21T22:31:00.9999464Z 2025-04-21T22:31:02.6434127Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:31:02.6434481Z loading model: 0it [00:01, ?it/s] 2025-04-21T22:31:02.6434775Z cuda train res2net101_26w_4s 2025-04-21T22:33:21.8529410Z Compilation time (from dynamo_timed): 127.639645703 2025-04-21T22:33:21.8624753Z pass 2025-04-21T22:33:21.8947274Z TIMING: entire_frame_compile:88.09465 gc:0.00167 _recursive_pre_grad_passes:0.02342 pad_mm_benchmark:0.002 _recursive_joint_graph_passes:0.88779 _recursive_post_grad_passes:1.97632 async_compile.wait:0.48177 code_gen:48.97928 inductor_compile:79.75005 backend_compile:69.08924 cudagraphify.get_container:1.32661 entire_backward_compile:39.545 CUDAGraphNode.record:3.88415 total_wall_time:127.63965 2025-04-21T22:33:21.8949129Z STATS: call_* op count: 977 | FakeTensorMode.__torch_dispatch__:86935 | FakeTensor.__torch_dispatch__:79338 | ProxyTorchDispatchMode.__torch_dispatch__:37838 2025-04-21T22:33:21.8949923Z Dynamo produced 2 graphs covering 977 ops with 6 graph breaks (5 unique) 2025-04-21T22:33:32.9400587Z 2025-04-21T22:33:34.3157430Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:33:34.3157766Z loading model: 0it [00:01, ?it/s] 2025-04-21T22:33:34.3158062Z cuda train res2net50_14w_8s 2025-04-21T22:35:38.7548020Z Compilation time (from dynamo_timed): 114.97696638400001 2025-04-21T22:35:38.7628120Z pass 2025-04-21T22:35:38.7808791Z TIMING: entire_frame_compile:77.52832 gc:0.0025 _recursive_pre_grad_passes:0.02075 pad_mm_benchmark:0.00197 _recursive_joint_graph_passes:0.79669 _recursive_post_grad_passes:2.44774 async_compile.wait:0.6026 code_gen:46.32577 inductor_compile:72.71348 backend_compile:61.2375 cudagraphify.get_container:1.16797 entire_backward_compile:37.44865 CUDAGraphNode.record:3.38437 total_wall_time:114.97697 2025-04-21T22:35:38.7810641Z STATS: call_* op count: 852 | FakeTensorMode.__torch_dispatch__:76962 | FakeTensor.__torch_dispatch__:70528 | ProxyTorchDispatchMode.__torch_dispatch__:33251 2025-04-21T22:35:38.7811428Z Dynamo produced 2 graphs covering 852 ops with 6 graph breaks (5 unique) 2025-04-21T22:35:49.3630589Z 2025-04-21T22:35:50.5950381Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:35:50.5950846Z loading model: 0it [00:01, ?it/s] 2025-04-21T22:35:50.5951275Z cuda train res2next50 2025-04-21T22:36:58.6262252Z Compilation time (from dynamo_timed): 62.018078893 2025-04-21T22:36:58.6307714Z pass 2025-04-21T22:36:58.6459575Z TIMING: entire_frame_compile:40.39459 gc:0.00258 _recursive_pre_grad_passes:0.01333 pad_mm_benchmark:0.00199 _recursive_joint_graph_passes:0.51625 _recursive_post_grad_passes:0.96345 async_compile.wait:0.34761 code_gen:23.59796 inductor_compile:37.82674 backend_compile:30.87478 cudagraphify.get_container:0.65472 entire_backward_compile:21.62349 CUDAGraphNode.record:1.73064 total_wall_time:62.01808 2025-04-21T22:36:58.6462615Z STATS: call_* op count: 484 | FakeTensorMode.__torch_dispatch__:43353 | FakeTensor.__torch_dispatch__:39414 | ProxyTorchDispatchMode.__torch_dispatch__:18883 2025-04-21T22:36:58.6463881Z Dynamo produced 2 graphs covering 484 ops with 6 graph breaks (5 unique) 2025-04-21T22:37:06.3723705Z 2025-04-21T22:37:07.1437438Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:37:07.1437938Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:37:07.1438341Z cuda train resmlp_12_224 2025-04-21T22:37:27.4844352Z Compilation time (from dynamo_timed): 17.053876209000002 2025-04-21T22:37:27.4858330Z pass 2025-04-21T22:37:27.4988511Z TIMING: entire_frame_compile:11.50918 gc:0.00272 _recursive_pre_grad_passes:0.00616 pad_mm_benchmark:0.29023 _recursive_joint_graph_passes:0.79333 _recursive_post_grad_passes:0.2933 async_compile.wait:0.10794 code_gen:4.72748 inductor_compile:8.42351 backend_compile:8.36385 cudagraphify.get_container:0.24627 entire_backward_compile:5.54469 CUDAGraphNode.record:0.61392 total_wall_time:17.05388 2025-04-21T22:37:27.4990474Z STATS: call_* op count: 179 | FakeTensorMode.__torch_dispatch__:19985 | ProxyTorchDispatchMode.__torch_dispatch__:8361 | FakeTensor.__torch_dispatch__:9241 2025-04-21T22:37:27.4999939Z Dynamo produced 2 graphs covering 179 ops with 6 graph breaks (5 unique) 2025-04-21T22:37:33.3261009Z 2025-04-21T22:37:35.0150749Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:37:35.0151149Z loading model: 0it [00:01, ?it/s] 2025-04-21T22:37:35.0151450Z cuda train resnest101e 2025-04-21T22:39:39.0666111Z Compilation time (from dynamo_timed): 111.37256217699999 2025-04-21T22:39:39.0752624Z pass 2025-04-21T22:39:39.1140002Z TIMING: entire_frame_compile:74.59253 gc:0.00368 _recursive_pre_grad_passes:0.02202 pad_mm_benchmark:0.00219 _recursive_joint_graph_passes:1.12916 _recursive_post_grad_passes:2.7415 async_compile.wait:0.45126 code_gen:37.4415 inductor_compile:65.48623 backend_compile:56.0045 cudagraphify.get_container:1.1239 entire_backward_compile:36.78003 CUDAGraphNode.record:3.11119 total_wall_time:111.37256 2025-04-21T22:39:39.1142968Z STATS: call_* op count: 1028 | FakeTensorMode.__torch_dispatch__:88619 | FakeTensor.__torch_dispatch__:68543 | ProxyTorchDispatchMode.__torch_dispatch__:39181 2025-04-21T22:39:39.1144216Z Dynamo produced 2 graphs covering 1028 ops with 6 graph breaks (5 unique) 2025-04-21T22:39:49.0051225Z 2025-04-21T22:39:50.2068640Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:39:50.2069000Z loading model: 0it [00:01, ?it/s] 2025-04-21T22:39:50.2069296Z cuda train rexnet_100 2025-04-21T22:40:54.7052287Z W0421 22:40:54.704000 44811 site-packages/torch/_logging/_internal.py:1130] [6/0] Profiler function will be ignored 2025-04-21T22:41:49.5956759Z Compilation time (from dynamo_timed): 111.06887661 2025-04-21T22:41:49.5995152Z pass 2025-04-21T22:41:49.6078816Z TIMING: entire_frame_compile:91.0455 gc:0.00491 _recursive_pre_grad_passes:0.03848 pad_mm_benchmark:0.00201 _recursive_joint_graph_passes:0.6463 _recursive_post_grad_passes:1.87062 async_compile.wait:4.49111 code_gen:39.53063 inductor_compile:63.39406 backend_compile:70.76868 cudagraphify.get_container:0.67084 entire_backward_compile:20.02338 CUDAGraphNode.record:3.9739 total_wall_time:111.06888 2025-04-21T22:41:49.6080745Z STATS: call_* op count: 1490 | FakeTensorMode.__torch_dispatch__:67813 | FakeTensor.__torch_dispatch__:54342 | ProxyTorchDispatchMode.__torch_dispatch__:27870 2025-04-21T22:41:49.6081587Z Dynamo produced 3 graphs covering 1490 ops with 7 graph breaks (5 unique) 2025-04-21T22:41:59.2016865Z 2025-04-21T22:42:00.3923832Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:42:00.3924700Z loading model: 0it [00:01, ?it/s] 2025-04-21T22:42:00.3925013Z cuda train sebotnet33ts_256 2025-04-21T22:42:58.7230405Z Compilation time (from dynamo_timed): 52.059607151 2025-04-21T22:42:58.7257904Z pass 2025-04-21T22:42:58.7388400Z TIMING: entire_frame_compile:33.25167 gc:0.00366 _recursive_pre_grad_passes:0.00992 pad_mm_benchmark:0.74261 _recursive_joint_graph_passes:1.29315 _recursive_post_grad_passes:0.6942 async_compile.wait:1.686 code_gen:23.77826 inductor_compile:35.29174 backend_compile:26.76048 cudagraphify.get_container:0.47839 CachingAutotuner.benchmark_all_configs:0.09424 entire_backward_compile:18.80794 CUDAGraphNode.record:2.38722 total_wall_time:52.05961 2025-04-21T22:42:58.7390472Z STATS: call_* op count: 372 | FakeTensorMode.__torch_dispatch__:33177 | FakeTensor.__torch_dispatch__:23102 | ProxyTorchDispatchMode.__torch_dispatch__:13451 2025-04-21T22:42:58.7391285Z Dynamo produced 3 graphs covering 372 ops with 6 graph breaks (5 unique) 2025-04-21T22:43:05.8764770Z 2025-04-21T22:43:07.1807141Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:43:07.1807498Z loading model: 0it [00:01, ?it/s] 2025-04-21T22:43:07.1807800Z cuda train selecsls42b 2025-04-21T22:43:39.6068732Z Compilation time (from dynamo_timed): 28.410059273999998 2025-04-21T22:43:39.6091314Z pass 2025-04-21T22:43:39.6357425Z TIMING: entire_frame_compile:20.55511 gc:0.00379 _recursive_pre_grad_passes:0.00779 pad_mm_benchmark:0.00201 _recursive_joint_graph_passes:0.31777 _recursive_post_grad_passes:0.41979 async_compile.wait:0.33799 code_gen:9.26953 inductor_compile:15.96743 backend_compile:15.27274 cudagraphify.get_container:0.37711 entire_backward_compile:7.85494 CUDAGraphNode.record:0.86564 total_wall_time:28.41006 2025-04-21T22:43:39.6359303Z STATS: call_* op count: 178 | FakeTensorMode.__torch_dispatch__:20686 | FakeTensor.__torch_dispatch__:16978 | ProxyTorchDispatchMode.__torch_dispatch__:8921 2025-04-21T22:43:39.6360086Z Dynamo produced 2 graphs covering 178 ops with 6 graph breaks (5 unique) 2025-04-21T22:43:46.0808817Z 2025-04-21T22:43:47.0164233Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:43:47.0164563Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:43:47.0164858Z cuda train spnasnet_100 2025-04-21T22:44:33.8665430Z W0421 22:44:33.865000 52334 site-packages/torch/_logging/_internal.py:1130] [6/0] Profiler function will be ignored 2025-04-21T22:45:19.2251134Z Compilation time (from dynamo_timed): 85.32809847200001 2025-04-21T22:45:19.2286542Z pass 2025-04-21T22:45:19.2330408Z TIMING: entire_frame_compile:73.78617 gc:0.00466 _recursive_pre_grad_passes:0.03317 pad_mm_benchmark:0.00199 _recursive_joint_graph_passes:0.57033 _recursive_post_grad_passes:0.7195 async_compile.wait:3.29772 code_gen:26.11735 inductor_compile:45.1453 backend_compile:57.57455 cudagraphify.get_container:0.55078 entire_backward_compile:11.54193 CUDAGraphNode.record:3.20594 total_wall_time:85.3281 2025-04-21T22:45:19.2334534Z STATS: call_* op count: 1245 | FakeTensorMode.__torch_dispatch__:54399 | FakeTensor.__torch_dispatch__:45156 | ProxyTorchDispatchMode.__torch_dispatch__:23265 2025-04-21T22:45:19.2335349Z Dynamo produced 3 graphs covering 1245 ops with 7 graph breaks (5 unique) 2025-04-21T22:45:27.8806596Z 2025-04-21T22:45:29.8566093Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:45:29.8566735Z loading model: 0it [00:01, ?it/s] 2025-04-21T22:45:29.8567338Z cuda train swin_base_patch4_window7_224 2025-04-21T22:46:56.9122439Z skipping cudagraphs due to deterministic index put. Found from : 2025-04-21T22:46:56.9128336Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 424, in torch_dynamo_resume_in_forward_and_backward_pass_at_422 2025-04-21T22:46:56.9129380Z pred = mod(*cloned_inputs) 2025-04-21T22:46:56.9130182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 838, in forward 2025-04-21T22:46:56.9131007Z x = self.forward_features(x) 2025-04-21T22:46:56.9132450Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 830, in forward_features 2025-04-21T22:46:56.9133372Z x = self.layers(x) 2025-04-21T22:46:56.9134151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 559, in forward 2025-04-21T22:46:56.9134998Z x = self.blocks(x) 2025-04-21T22:46:56.9135771Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 406, in forward 2025-04-21T22:46:56.9136702Z x = x + self.drop_path1(self._attn(self.norm1(x))) 2025-04-21T22:46:56.9137617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 390, in _attn 2025-04-21T22:46:56.9138736Z attn_windows = self.attn(x_windows, mask=attn_mask) # nW*B, window_size*window_size, C 2025-04-21T22:46:56.9139854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 194, in forward 2025-04-21T22:46:56.9140759Z attn = attn + self._get_rel_pos_bias() 2025-04-21T22:46:56.9141710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/swin_transformer.py", line 165, in _get_rel_pos_bias 2025-04-21T22:46:56.9142782Z relative_position_bias = self.relative_position_bias_table[ 2025-04-21T22:46:56.9143198Z 2025-04-21T22:46:57.3450894Z W0421 22:46:57.343000 53854 site-packages/torch/_logging/_internal.py:1130] [6/0] Profiler function will be ignored 2025-04-21T22:48:08.2953859Z Compilation time (from dynamo_timed): 147.83091476500002 2025-04-21T22:48:08.2998747Z pass 2025-04-21T22:48:08.3445317Z TIMING: entire_frame_compile:119.18724 gc:0.00493 _recursive_pre_grad_passes:0.05736 pad_mm_benchmark:1.67257 _recursive_joint_graph_passes:4.23315 _recursive_post_grad_passes:1.94047 async_compile.wait:7.46516 code_gen:53.05332 inductor_compile:82.63988 backend_compile:96.19446 cudagraphify.get_container:0.67428 CachingAutotuner.benchmark_all_configs:0.22925 entire_backward_compile:28.64368 CUDAGraphNode.record:3.03999 total_wall_time:147.83091 2025-04-21T22:48:08.3449395Z STATS: call_* op count: 3106 | FakeTensorMode.__torch_dispatch__:127269 | FakeTensor.__torch_dispatch__:69692 | ProxyTorchDispatchMode.__torch_dispatch__:52945 2025-04-21T22:48:08.3451003Z Dynamo produced 3 graphs covering 3106 ops with 7 graph breaks (5 unique) 2025-04-21T22:48:18.8645638Z 2025-04-21T22:48:22.1198835Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:48:22.1199652Z loading model: 0it [00:03, ?it/s] 2025-04-21T22:48:22.1200399Z cuda train swsl_resnext101_32x16d 2025-04-21T22:49:37.8333391Z Compilation time (from dynamo_timed): 60.661157539 2025-04-21T22:49:37.8388225Z pass 2025-04-21T22:49:37.9762675Z TIMING: entire_frame_compile:44.47122 gc:0.00268 _recursive_pre_grad_passes:0.01384 pad_mm_benchmark:0.00204 _recursive_joint_graph_passes:0.8973 _recursive_post_grad_passes:1.01936 async_compile.wait:0.22491 code_gen:17.37208 inductor_compile:32.49823 backend_compile:33.08052 cudagraphify.get_container:0.74232 entire_backward_compile:16.18993 CUDAGraphNode.record:1.82833 total_wall_time:60.66116 2025-04-21T22:49:37.9764541Z STATS: call_* op count: 453 | FakeTensorMode.__torch_dispatch__:48552 | FakeTensor.__torch_dispatch__:41497 | ProxyTorchDispatchMode.__torch_dispatch__:22005 2025-04-21T22:49:37.9765327Z Dynamo produced 2 graphs covering 453 ops with 6 graph breaks (5 unique) 2025-04-21T22:49:45.7805393Z 2025-04-21T22:49:46.8175588Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:49:46.8175940Z loading model: 0it [00:01, ?it/s] 2025-04-21T22:49:46.8176247Z cuda train tf_efficientnet_b0 2025-04-21T22:50:41.3043545Z Compilation time (from dynamo_timed): 49.919852678 2025-04-21T22:50:41.3078131Z pass 2025-04-21T22:50:41.3167508Z TIMING: entire_frame_compile:33.35888 gc:0.00281 _recursive_pre_grad_passes:0.00955 pad_mm_benchmark:0.00199 _recursive_joint_graph_passes:0.41955 _recursive_post_grad_passes:0.77695 async_compile.wait:0.64566 code_gen:19.3784 inductor_compile:31.30449 backend_compile:25.6715 cudagraphify.get_container:0.55532 entire_backward_compile:16.56098 CUDAGraphNode.record:1.46304 total_wall_time:49.91985 2025-04-21T22:50:41.3169384Z STATS: call_* op count: 297 | FakeTensorMode.__torch_dispatch__:35135 | FakeTensor.__torch_dispatch__:28043 | ProxyTorchDispatchMode.__torch_dispatch__:14653 2025-04-21T22:50:41.3170168Z Dynamo produced 2 graphs covering 297 ops with 6 graph breaks (5 unique) 2025-04-21T22:50:48.7122465Z 2025-04-21T22:50:49.8930249Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:50:49.8930586Z loading model: 0it [00:01, ?it/s] 2025-04-21T22:50:49.8930891Z cuda train tf_mixnet_l 2025-04-21T22:52:07.6902819Z Compilation time (from dynamo_timed): 71.89469687500001 2025-04-21T22:52:07.6948763Z pass 2025-04-21T22:52:07.7044166Z TIMING: entire_frame_compile:49.20776 gc:0.00283 _recursive_pre_grad_passes:0.01375 pad_mm_benchmark:0.00198 _recursive_joint_graph_passes:0.51134 _recursive_post_grad_passes:0.97608 async_compile.wait:0.92966 code_gen:29.23318 inductor_compile:45.06992 backend_compile:37.0754 cudagraphify.get_container:0.72618 entire_backward_compile:22.68693 CUDAGraphNode.record:1.99242 total_wall_time:71.8947 2025-04-21T22:52:07.7045999Z STATS: call_* op count: 598 | FakeTensorMode.__torch_dispatch__:49814 | FakeTensor.__torch_dispatch__:35857 | ProxyTorchDispatchMode.__torch_dispatch__:19827 2025-04-21T22:52:07.7046783Z Dynamo produced 2 graphs covering 598 ops with 6 graph breaks (5 unique) 2025-04-21T22:52:15.8723446Z 2025-04-21T22:52:16.9485435Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:52:16.9485875Z loading model: 0it [00:01, ?it/s] 2025-04-21T22:52:16.9486265Z cuda train tinynet_a 2025-04-21T22:53:17.1350420Z Compilation time (from dynamo_timed): 55.464342063000004 2025-04-21T22:53:17.1390201Z pass 2025-04-21T22:53:17.1470037Z TIMING: entire_frame_compile:37.83009 gc:0.0037 _recursive_pre_grad_passes:0.01038 pad_mm_benchmark:0.00201 _recursive_joint_graph_passes:0.46994 _recursive_post_grad_passes:1.31648 async_compile.wait:0.64509 code_gen:19.41092 inductor_compile:33.33745 backend_compile:28.33812 cudagraphify.get_container:0.52267 entire_backward_compile:17.63426 CUDAGraphNode.record:1.61612 total_wall_time:55.46434 2025-04-21T22:53:17.1471892Z STATS: call_* op count: 346 | FakeTensorMode.__torch_dispatch__:41025 | FakeTensor.__torch_dispatch__:33164 | ProxyTorchDispatchMode.__torch_dispatch__:17287 2025-04-21T22:53:17.1472691Z Dynamo produced 2 graphs covering 346 ops with 6 graph breaks (5 unique) 2025-04-21T22:53:24.9339481Z 2025-04-21T22:53:25.9766785Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:53:25.9768235Z loading model: 0it [00:01, ?it/s] 2025-04-21T22:53:25.9768551Z cuda train tnt_s_patch16_224 2025-04-21T22:54:32.9344919Z skipping cudagraphs due to deterministic index put. Found from : 2025-04-21T22:54:32.9346336Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 424, in torch_dynamo_resume_in_forward_and_backward_pass_at_422 2025-04-21T22:54:32.9347336Z pred = mod(*cloned_inputs) 2025-04-21T22:54:32.9348041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/tnt.py", line 335, in forward 2025-04-21T22:54:32.9348768Z x = self.forward_features(x) 2025-04-21T22:54:32.9349519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/tnt.py", line 311, in forward_features 2025-04-21T22:54:32.9350380Z pixel_embed = self.pixel_embed(x, self.pixel_pos) 2025-04-21T22:54:32.9351168Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/tnt.py", line 183, in forward 2025-04-21T22:54:32.9351900Z x = self.unfold(x) 2025-04-21T22:54:32.9352116Z 2025-04-21T22:54:33.2696610Z W0421 22:54:33.268000 64131 site-packages/torch/_logging/_internal.py:1130] [6/0] Profiler function will be ignored 2025-04-21T22:55:49.7217502Z Compilation time (from dynamo_timed): 135.567602249 2025-04-21T22:55:49.7259608Z pass 2025-04-21T22:55:49.7509745Z TIMING: entire_frame_compile:115.93542 gc:0.0065 _recursive_pre_grad_passes:0.06225 pad_mm_benchmark:1.08288 _recursive_joint_graph_passes:3.56893 _recursive_post_grad_passes:1.45688 async_compile.wait:7.02839 code_gen:44.07244 inductor_compile:72.60745 backend_compile:93.59733 cudagraphify.get_container:0.57431 CachingAutotuner.benchmark_all_configs:0.04079 entire_backward_compile:19.63218 CUDAGraphNode.record:2.94702 total_wall_time:135.5676 2025-04-21T22:55:49.7511805Z STATS: call_* op count: 2553 | FakeTensorMode.__torch_dispatch__:121406 | FakeTensor.__torch_dispatch__:69232 | ProxyTorchDispatchMode.__torch_dispatch__:52009 2025-04-21T22:55:49.7512637Z Dynamo produced 3 graphs covering 2553 ops with 7 graph breaks (5 unique) 2025-04-21T22:56:00.5871563Z 2025-04-21T22:56:02.0119614Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:56:02.0120066Z loading model: 0it [00:01, ?it/s] 2025-04-21T22:56:02.0120486Z cuda train twins_pcpvt_base 2025-04-21T22:57:21.3401539Z W0421 22:57:21.339000 65263 site-packages/torch/_logging/_internal.py:1130] [6/0] Profiler function will be ignored 2025-04-21T22:59:24.5691455Z Compilation time (from dynamo_timed): 184.533177197 2025-04-21T22:59:24.5750754Z pass 2025-04-21T22:59:24.6057400Z TIMING: entire_frame_compile:157.22352 gc:0.00477 _recursive_pre_grad_passes:0.07762 pad_mm_benchmark:0.26761 _recursive_joint_graph_passes:2.57514 _recursive_post_grad_passes:1.84215 async_compile.wait:6.09568 code_gen:59.13613 inductor_compile:99.78196 backend_compile:124.78875 cudagraphify.get_container:0.67013 entire_backward_compile:27.30966 CUDAGraphNode.record:7.38289 total_wall_time:184.53318 2025-04-21T22:59:24.6059356Z STATS: call_* op count: 3474 | FakeTensorMode.__torch_dispatch__:150645 | FakeTensor.__torch_dispatch__:94156 | ProxyTorchDispatchMode.__torch_dispatch__:63213 2025-04-21T22:59:24.6060175Z Dynamo produced 3 graphs covering 3474 ops with 7 graph breaks (5 unique) 2025-04-21T22:59:37.3104807Z 2025-04-21T22:59:38.6146811Z loading model: 0it [00:00, ?it/s] 2025-04-21T22:59:38.6147453Z loading model: 0it [00:01, ?it/s] 2025-04-21T22:59:38.6148068Z cuda train visformer_small 2025-04-21T23:00:10.7461120Z W0421 23:00:10.745000 66908 site-packages/torch/_logging/_internal.py:1130] [6/0] Profiler function will be ignored 2025-04-21T23:00:40.9614872Z Compilation time (from dynamo_timed): 56.32688573 2025-04-21T23:00:40.9636820Z pass 2025-04-21T23:00:40.9818674Z TIMING: entire_frame_compile:49.31016 gc:0.00531 _recursive_pre_grad_passes:0.02453 pad_mm_benchmark:0.74861 _recursive_joint_graph_passes:2.0882 _recursive_post_grad_passes:0.52442 async_compile.wait:2.64558 code_gen:15.88659 inductor_compile:27.41967 backend_compile:37.48345 cudagraphify.get_container:0.36766 entire_backward_compile:7.01673 CUDAGraphNode.record:1.99445 total_wall_time:56.32689 2025-04-21T23:00:40.9821107Z STATS: call_* op count: 947 | FakeTensorMode.__torch_dispatch__:40677 | FakeTensor.__torch_dispatch__:29986 | ProxyTorchDispatchMode.__torch_dispatch__:16839 2025-04-21T23:00:40.9821907Z Dynamo produced 3 graphs covering 947 ops with 7 graph breaks (5 unique) 2025-04-21T23:00:48.2349619Z 2025-04-21T23:00:50.2617220Z loading model: 0it [00:00, ?it/s] 2025-04-21T23:00:50.2617578Z loading model: 0it [00:02, ?it/s] 2025-04-21T23:00:50.2617880Z cuda train vit_base_patch16_224 2025-04-21T23:01:17.1892207Z W0421 23:01:17.188000 68010 site-packages/torch/_logging/_internal.py:1130] [6/0] Profiler function will be ignored 2025-04-21T23:01:49.4118410Z Compilation time (from dynamo_timed): 50.571676790000005 2025-04-21T23:01:49.4135049Z pass 2025-04-21T23:01:49.4683335Z TIMING: entire_frame_compile:43.59838 gc:0.00534 _recursive_pre_grad_passes:0.02667 pad_mm_benchmark:0.09298 _recursive_joint_graph_passes:0.88142 _recursive_post_grad_passes:0.56583 async_compile.wait:2.39894 code_gen:14.40152 inductor_compile:25.5171 backend_compile:34.2741 cudagraphify.get_container:0.30592 entire_backward_compile:6.9733 CUDAGraphNode.record:1.8523 total_wall_time:50.57168 2025-04-21T23:01:49.4685197Z STATS: call_* op count: 1046 | FakeTensorMode.__torch_dispatch__:44552 | FakeTensor.__torch_dispatch__:27627 | ProxyTorchDispatchMode.__torch_dispatch__:19026 2025-04-21T23:01:49.4685994Z Dynamo produced 3 graphs covering 1046 ops with 7 graph breaks (5 unique) 2025-04-21T23:01:56.5288596Z 2025-04-21T23:01:57.8375477Z loading model: 0it [00:00, ?it/s] 2025-04-21T23:01:57.8375874Z loading model: 0it [00:01, ?it/s] 2025-04-21T23:01:57.8376190Z cuda train volo_d1_224 2025-04-21T23:02:30.1119105Z skipping cudagraphs due to deterministic index put. Found from : 2025-04-21T23:02:30.1119924Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 424, in torch_dynamo_resume_in_forward_and_backward_pass_at_422 2025-04-21T23:02:30.1120626Z pred = mod(*cloned_inputs) 2025-04-21T23:02:30.1121127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/volo.py", line 822, in forward 2025-04-21T23:02:30.1121645Z x = self.forward_features(x) 2025-04-21T23:02:30.1122163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/volo.py", line 795, in forward_features 2025-04-21T23:02:30.1122708Z x = self.forward_tokens(x) 2025-04-21T23:02:30.1123216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/volo.py", line 642, in forward_tokens 2025-04-21T23:02:30.1123734Z x = block(x) 2025-04-21T23:02:30.1124165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/volo.py", line 135, in forward 2025-04-21T23:02:30.1124715Z x = x + self.drop_path1(self.attn(self.norm1(x))) 2025-04-21T23:02:30.1125241Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/volo.py", line 90, in forward 2025-04-21T23:02:30.1125934Z x = F.fold(x, output_size=(H, W), kernel_size=self.kernel_size, padding=self.padding, stride=self.stride) 2025-04-21T23:02:30.1126338Z 2025-04-21T23:02:52.9618688Z W0421 23:02:52.960000 68494 site-packages/torch/_logging/_internal.py:1130] [6/0] Profiler function will be ignored 2025-04-21T23:03:46.0120222Z Compilation time (from dynamo_timed): 101.64373746400001 2025-04-21T23:03:46.0149347Z pass 2025-04-21T23:03:46.0278293Z TIMING: entire_frame_compile:80.30003 gc:0.00586 _recursive_pre_grad_passes:0.0414 pad_mm_benchmark:0.90849 _recursive_joint_graph_passes:2.31863 _recursive_post_grad_passes:1.09825 async_compile.wait:5.14929 code_gen:35.95396 inductor_compile:56.65539 backend_compile:63.26562 CachingAutotuner.benchmark_all_configs:0.1692 entire_backward_compile:21.34371 cudagraphify.get_container:1.01303 CUDAGraphNode.record:1.01607 total_wall_time:101.64374 2025-04-21T23:03:46.0280688Z STATS: call_* op count: 1786 | FakeTensorMode.__torch_dispatch__:80273 | FakeTensor.__torch_dispatch__:48029 | ProxyTorchDispatchMode.__torch_dispatch__:34603 2025-04-21T23:03:46.0281495Z Dynamo produced 3 graphs covering 1786 ops with 7 graph breaks (5 unique) 2025-04-21T23:03:54.9237972Z 2025-04-21T23:03:58.4556927Z loading model: 0it [00:00, ?it/s] 2025-04-21T23:03:58.4557280Z loading model: 0it [00:03, ?it/s] 2025-04-21T23:03:58.4557584Z cuda train xcit_large_24_p8_224 2025-04-21T23:04:02.9148208Z WARNING:common:fp64 golden ref were not generated for xcit_large_24_p8_224. Setting accuracy check to cosine 2025-04-21T23:05:55.8306494Z W0421 23:05:55.829000 69710 site-packages/torch/_logging/_internal.py:1130] [6/0] Profiler function will be ignored 2025-04-21T23:08:30.1279646Z Compilation time (from dynamo_timed): 244.34661755000002 2025-04-21T23:08:30.1324432Z pass_due_to_skip 2025-04-21T23:08:30.1758645Z TIMING: entire_frame_compile:206.94234 gc:0.00607 _recursive_pre_grad_passes:0.09586 pad_mm_benchmark:0.29691 _recursive_joint_graph_passes:3.35955 _recursive_post_grad_passes:2.19118 async_compile.wait:8.88153 code_gen:83.02672 inductor_compile:135.68078 backend_compile:165.87017 cudagraphify.get_container:1.06561 CachingAutotuner.benchmark_all_configs:0.55945 entire_backward_compile:37.40427 CUDAGraphNode.record:9.90984 total_wall_time:244.34662 2025-04-21T23:08:30.1760716Z STATS: call_* op count: 4300 | FakeTensorMode.__torch_dispatch__:191234 | FakeTensor.__torch_dispatch__:130061 | ProxyTorchDispatchMode.__torch_dispatch__:83273 2025-04-21T23:08:30.1761531Z Dynamo produced 3 graphs covering 4300 ops with 7 graph breaks (5 unique) 2025-04-21T23:08:41.9891332Z accuracy pass_rate=96.77% 2025-04-21T23:08:41.9895103Z calls_captured gmean=806.58x mean=1124.387x 2025-04-21T23:08:41.9899163Z unique_graphs gmean=2.49x mean=2.548x 2025-04-21T23:08:41.9903167Z graph_breaks gmean=6.39x mean=6.419x 2025-04-21T23:08:41.9907286Z unique_graph_breaks gmean=4.96x mean=4.968x 2025-04-21T23:08:41.9911038Z autograd_captures gmean=0.00x mean=0.000x 2025-04-21T23:08:41.9914878Z autograd_compiles gmean=0.00x mean=0.000x 2025-04-21T23:08:41.9918491Z cudagraph_skips gmean=0.00x mean=0.097x 2025-04-21T23:08:41.9919389Z compilation_latency mean=93.464 seconds 2025-04-21T23:08:46.1406765Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy/inductor_timm_training.csv 2025-04-21T23:08:46.4072118Z mixnet_l PASS 2025-04-21T23:08:46.4075844Z mnasnet_100 PASS 2025-04-21T23:08:46.4080148Z mobilenetv2_100 PASS 2025-04-21T23:08:46.4084751Z mobilenetv3_large_100 PASS 2025-04-21T23:08:46.4089446Z mobilevit_s PASS 2025-04-21T23:08:46.4098766Z nfnet_l0 PASS 2025-04-21T23:08:46.4099107Z pit_b_224 PASS 2025-04-21T23:08:46.4103362Z pnasnet5large PASS 2025-04-21T23:08:46.4108318Z poolformer_m36 PASS 2025-04-21T23:08:46.4112721Z regnety_002 PASS 2025-04-21T23:08:46.4117140Z repvgg_a2 PASS 2025-04-21T23:08:46.4121598Z res2net101_26w_4s PASS 2025-04-21T23:08:46.4126114Z res2net50_14w_8s PASS 2025-04-21T23:08:46.4130649Z res2next50 PASS 2025-04-21T23:08:46.4135224Z resmlp_12_224 PASS 2025-04-21T23:08:46.4139691Z resnest101e PASS 2025-04-21T23:08:46.4144268Z rexnet_100 PASS 2025-04-21T23:08:46.4149064Z sebotnet33ts_256 PASS 2025-04-21T23:08:46.4153498Z selecsls42b PASS 2025-04-21T23:08:46.4158007Z spnasnet_100 PASS 2025-04-21T23:08:46.4162591Z swin_base_patch4_window7_224 PASS 2025-04-21T23:08:46.4167017Z swsl_resnext101_32x16d PASS 2025-04-21T23:08:46.4171594Z tf_efficientnet_b0 PASS 2025-04-21T23:08:46.4176539Z tf_mixnet_l PASS 2025-04-21T23:08:46.4181008Z tinynet_a PASS 2025-04-21T23:08:46.4185465Z tnt_s_patch16_224 PASS 2025-04-21T23:08:46.4190538Z twins_pcpvt_base PASS 2025-04-21T23:08:46.4194775Z visformer_small PASS 2025-04-21T23:08:46.4199199Z vit_base_patch16_224 PASS 2025-04-21T23:08:46.4203730Z volo_d1_224 PASS 2025-04-21T23:08:46.4208280Z xcit_large_24_p8_224 XFAIL 2025-04-21T23:08:46.4637352Z + python benchmarks/dynamo/check_graph_breaks.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy/inductor_timm_training.csv 2025-04-21T23:08:46.7315175Z mixnet_l PASS 2025-04-21T23:08:46.7319457Z mnasnet_100 PASS 2025-04-21T23:08:46.7324523Z mobilenetv2_100 PASS 2025-04-21T23:08:46.7329778Z mobilenetv3_large_100 PASS 2025-04-21T23:08:46.7334549Z mobilevit_s PASS 2025-04-21T23:08:46.7339536Z nfnet_l0 PASS 2025-04-21T23:08:46.7344253Z pit_b_224 PASS 2025-04-21T23:08:46.7349578Z pnasnet5large PASS 2025-04-21T23:08:46.7354310Z poolformer_m36 PASS 2025-04-21T23:08:46.7359288Z regnety_002 PASS 2025-04-21T23:08:46.7364064Z repvgg_a2 PASS 2025-04-21T23:08:46.7369017Z res2net101_26w_4s PASS 2025-04-21T23:08:46.7374078Z res2net50_14w_8s PASS 2025-04-21T23:08:46.7379888Z res2next50 PASS 2025-04-21T23:08:46.7384532Z resmlp_12_224 PASS 2025-04-21T23:08:46.7389945Z resnest101e PASS 2025-04-21T23:08:46.7394476Z rexnet_100 PASS 2025-04-21T23:08:46.7399647Z sebotnet33ts_256 PASS 2025-04-21T23:08:46.7404347Z selecsls42b PASS 2025-04-21T23:08:46.7409287Z spnasnet_100 PASS 2025-04-21T23:08:46.7414235Z swin_base_patch4_window7_224 PASS 2025-04-21T23:08:46.7419161Z swsl_resnext101_32x16d PASS 2025-04-21T23:08:46.7424087Z tf_efficientnet_b0 PASS 2025-04-21T23:08:46.7429408Z tf_mixnet_l PASS 2025-04-21T23:08:46.7434160Z tinynet_a PASS 2025-04-21T23:08:46.7439082Z tnt_s_patch16_224 PASS 2025-04-21T23:08:46.7443971Z twins_pcpvt_base PASS 2025-04-21T23:08:46.7448974Z visformer_small PASS 2025-04-21T23:08:46.7453970Z vit_base_patch16_224 PASS 2025-04-21T23:08:46.7459028Z volo_d1_224 PASS 2025-04-21T23:08:46.7463955Z xcit_large_24_p8_224 PASS 2025-04-21T23:08:46.7903006Z + cleanup_workspace 2025-04-21T23:08:46.7903971Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2025-04-21T23:08:46.7905366Z sudo may print the following warning message that can be ignored. The chown command will still run. 2025-04-21T23:08:46.7906605Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2025-04-21T23:08:46.7907462Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2025-04-21T23:08:46.7908302Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2025-04-21T23:08:46.7908895Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2025-04-21T23:08:46.7909330Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2025-04-21T23:08:47.6345216Z ##[group]Run pytorch/test-infra/.github/actions/upload-benchmark-results@main 2025-04-21T23:08:47.6345773Z with: 2025-04-21T23:08:47.6346009Z benchmark-results-dir: test/test-reports 2025-04-21T23:08:47.6346314Z dry-run: false 2025-04-21T23:08:47.6346528Z schema-version: v3 2025-04-21T23:08:47.6346955Z github-token: *** 2025-04-21T23:08:47.6347171Z env: 2025-04-21T23:08:47.6347368Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:47.6347680Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:47.6348198Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:47.6348659Z ##[endgroup] 2025-04-21T23:08:47.6374795Z ##[group]Run set -eux 2025-04-21T23:08:47.6375416Z set -eux 2025-04-21T23:08:47.6375673Z python3 -mpip install boto3==1.35.33 2025-04-21T23:08:47.6390126Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T23:08:47.6390487Z env: 2025-04-21T23:08:47.6390731Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:47.6391051Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:47.6391571Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:47.6392028Z ##[endgroup] 2025-04-21T23:08:47.6428644Z + python3 -mpip install boto3==1.35.33 2025-04-21T23:08:47.8837580Z Defaulting to user installation because normal site-packages is not writeable 2025-04-21T23:08:49.0564018Z Collecting boto3==1.35.33 2025-04-21T23:08:49.0592401Z Using cached boto3-1.35.33-py3-none-any.whl (139 kB) 2025-04-21T23:08:49.0650615Z Requirement already satisfied: s3transfer<0.11.0,>=0.10.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from boto3==1.35.33) (0.10.4) 2025-04-21T23:08:49.0654376Z Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /usr/lib/python3.9/site-packages (from boto3==1.35.33) (0.10.0) 2025-04-21T23:08:49.0662508Z Requirement already satisfied: botocore<1.36.0,>=1.35.33 in /home/ec2-user/.local/lib/python3.9/site-packages (from boto3==1.35.33) (1.35.99) 2025-04-21T23:08:49.0718093Z Requirement already satisfied: urllib3<1.27,>=1.25.4 in /usr/lib/python3.9/site-packages (from botocore<1.36.0,>=1.35.33->boto3==1.35.33) (1.25.10) 2025-04-21T23:08:49.0723482Z Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /home/ec2-user/.local/lib/python3.9/site-packages (from botocore<1.36.0,>=1.35.33->boto3==1.35.33) (2.8.2) 2025-04-21T23:08:49.0763227Z Requirement already satisfied: six>=1.5 in /usr/lib/python3.9/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.36.0,>=1.35.33->boto3==1.35.33) (1.15.0) 2025-04-21T23:08:49.2364393Z Installing collected packages: boto3 2025-04-21T23:08:49.2365038Z Attempting uninstall: boto3 2025-04-21T23:08:49.2366027Z Found existing installation: boto3 1.35.42 2025-04-21T23:08:49.2492052Z Uninstalling boto3-1.35.42: 2025-04-21T23:08:49.2507565Z Successfully uninstalled boto3-1.35.42 2025-04-21T23:08:49.3113813Z Successfully installed boto3-1.35.33 2025-04-21T23:08:49.3862528Z ##[group]Run set -eux 2025-04-21T23:08:49.3862790Z set -eux 2025-04-21T23:08:49.3863005Z  2025-04-21T23:08:49.3863229Z if [[ -z "${GITHUB_TOKEN}" ]]; then 2025-04-21T23:08:49.3863565Z  echo "Missing github-token input" 2025-04-21T23:08:49.3863870Z  exit 1 2025-04-21T23:08:49.3864076Z fi 2025-04-21T23:08:49.3872804Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T23:08:49.3873162Z env: 2025-04-21T23:08:49.3873362Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:49.3873681Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:49.3874197Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:49.3874825Z GITHUB_TOKEN: *** 2025-04-21T23:08:49.3875433Z ##[endgroup] 2025-04-21T23:08:49.3916285Z + [[ -z *** ]] 2025-04-21T23:08:49.3970442Z ##[group]Run pytorch/test-infra/.github/actions/get-workflow-job-id@main 2025-04-21T23:08:49.3971253Z with: 2025-04-21T23:08:49.3971602Z github-token: *** 2025-04-21T23:08:49.3971831Z env: 2025-04-21T23:08:49.3972045Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:49.3972369Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:49.3972904Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:49.3973375Z ##[endgroup] 2025-04-21T23:08:49.3998979Z ##[group]Run set -eux 2025-04-21T23:08:49.3999250Z set -eux 2025-04-21T23:08:49.3999492Z  2025-04-21T23:08:49.3999970Z python3 "${GITHUB_ACTION_PATH}/../../scripts/get_workflow_job_id.py" "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2025-04-21T23:08:49.4009237Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T23:08:49.4009588Z env: 2025-04-21T23:08:49.4009796Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:49.4010103Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:49.4010621Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:49.4011215Z GITHUB_TOKEN: *** 2025-04-21T23:08:49.4011442Z ##[endgroup] 2025-04-21T23:08:49.4040572Z + python3 /home/ec2-user/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/get-workflow-job-id/../../scripts/get_workflow_job_id.py 14580756816 i-09a65d907c0d198ba 2025-04-21T23:08:50.6300084Z setting job-id=40900324325 2025-04-21T23:08:50.6301361Z setting job-name=cuda12.6-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2025-04-21T23:08:50.6414193Z ##[group]Run set -eux 2025-04-21T23:08:50.6414512Z set -eux 2025-04-21T23:08:50.6414746Z  2025-04-21T23:08:50.6415110Z python3 "${GITHUB_ACTION_PATH}/../../scripts/benchmarks/gather_metadata.py" \ 2025-04-21T23:08:50.6415590Z  --schema-version "${SCHEMA_VERSION}" \ 2025-04-21T23:08:50.6415920Z  --repo "${REPO}" \ 2025-04-21T23:08:50.6416204Z  --head-branch "${HEAD_BRANCH}" \ 2025-04-21T23:08:50.6416554Z  --head-sha "${HEAD_SHA}" \ 2025-04-21T23:08:50.6416879Z  --workflow-id "${WORKFLOW_RUN_ID}" \ 2025-04-21T23:08:50.6417218Z  --run-attempt "${RUN_ATTEMPT}" \ 2025-04-21T23:08:50.6417531Z  --job-id "${JOB_ID}" \ 2025-04-21T23:08:50.6417820Z  --job-name "${JOB_NAME}" 2025-04-21T23:08:50.6426624Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T23:08:50.6426969Z env: 2025-04-21T23:08:50.6427176Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:50.6427490Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:50.6428000Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:50.6428463Z SCHEMA_VERSION: v3 2025-04-21T23:08:50.6428705Z REPO: pytorch/pytorch 2025-04-21T23:08:50.6428978Z HEAD_BRANCH: refs/tags/ciflow/inductor/148893 2025-04-21T23:08:50.6429339Z HEAD_SHA: b68c0effe97351b0c24b2f7aef671559be6aa977 2025-04-21T23:08:50.6429679Z WORKFLOW_RUN_ID: 14580756816 2025-04-21T23:08:50.6429932Z RUN_ATTEMPT: 1 2025-04-21T23:08:50.6430147Z JOB_ID: 40900324325 2025-04-21T23:08:50.6430567Z JOB_NAME: cuda12.6-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2025-04-21T23:08:50.6431028Z ##[endgroup] 2025-04-21T23:08:50.6460605Z + python3 /home/ec2-user/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/upload-benchmark-results/../../scripts/benchmarks/gather_metadata.py --schema-version v3 --repo pytorch/pytorch --head-branch refs/tags/ciflow/inductor/148893 --head-sha b68c0effe97351b0c24b2f7aef671559be6aa977 --workflow-id 14580756816 --run-attempt 1 --job-id 40900324325 --job-name 'cuda12.6-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu)' 2025-04-21T23:08:50.6789382Z ##[group]Run set -eux 2025-04-21T23:08:50.6789636Z set -eux 2025-04-21T23:08:50.6789856Z  2025-04-21T23:08:50.6790093Z # TODO (huydhn): Implement this part 2025-04-21T23:08:50.6790796Z echo "runners=[]" >> "${GITHUB_OUTPUT}" 2025-04-21T23:08:50.6798781Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T23:08:50.6799127Z env: 2025-04-21T23:08:50.6799333Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:50.6799648Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:50.6800165Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:50.6800612Z ##[endgroup] 2025-04-21T23:08:50.6831612Z + echo 'runners=[]' 2025-04-21T23:08:50.6860548Z ##[group]Run set -eux 2025-04-21T23:08:50.6860802Z set -eux 2025-04-21T23:08:50.6861019Z  2025-04-21T23:08:50.6861254Z # TODO (huydhn): Implement this part 2025-04-21T23:08:50.6861624Z echo "dependencies={}" >> "${GITHUB_OUTPUT}" 2025-04-21T23:08:50.6869628Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T23:08:50.6869971Z env: 2025-04-21T23:08:50.6870177Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:50.6870515Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:50.6871031Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:50.6871481Z ##[endgroup] 2025-04-21T23:08:50.6897571Z + echo 'dependencies={}' 2025-04-21T23:08:50.6939594Z ##[group]Run set -eux 2025-04-21T23:08:50.6939866Z set -eux 2025-04-21T23:08:50.6940075Z  2025-04-21T23:08:50.6940344Z if [[ ! -d "${BENCHMARK_RESULTS_DIR}" ]]; then 2025-04-21T23:08:50.6940794Z  echo "${BENCHMARK_RESULTS_DIR} does not exist, skipping" 2025-04-21T23:08:50.6941243Z  # We don't want the job to fail if the directory doesn't exist 2025-04-21T23:08:50.6941609Z  exit 0 2025-04-21T23:08:50.6941819Z fi 2025-04-21T23:08:50.6942015Z  2025-04-21T23:08:50.6942240Z if [[ "${DRY_RUN}" == "true" ]]; then 2025-04-21T23:08:50.6942690Z  python3 "${GITHUB_ACTION_PATH}/../../scripts/upload_benchmark_results.py" \ 2025-04-21T23:08:50.6943214Z  --benchmark-results-dir "${BENCHMARK_RESULTS_DIR}" \ 2025-04-21T23:08:50.6943604Z  --metadata "${BENCHMARK_METADATA}" \ 2025-04-21T23:08:50.6943943Z  --runners "${RUNNER_INFO}" \ 2025-04-21T23:08:50.6944271Z  --dependencies "${DEPENDENCIES}" \ 2025-04-21T23:08:50.6944576Z  --dry-run 2025-04-21T23:08:50.6944809Z else 2025-04-21T23:08:50.6945167Z  python3 "${GITHUB_ACTION_PATH}/../../scripts/upload_benchmark_results.py" \ 2025-04-21T23:08:50.6945771Z  --benchmark-results-dir "${BENCHMARK_RESULTS_DIR}" \ 2025-04-21T23:08:50.6946166Z  --metadata "${BENCHMARK_METADATA}" \ 2025-04-21T23:08:50.6946498Z  --runners "${RUNNER_INFO}" \ 2025-04-21T23:08:50.6946828Z  --dependencies "${DEPENDENCIES}" 2025-04-21T23:08:50.6947125Z fi 2025-04-21T23:08:50.6955185Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T23:08:50.6955547Z env: 2025-04-21T23:08:50.6955751Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:50.6956065Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:50.6956579Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:50.6957062Z BENCHMARK_RESULTS_DIR: test/test-reports 2025-04-21T23:08:50.6957356Z DRY_RUN: false 2025-04-21T23:08:50.6958617Z BENCHMARK_METADATA: {"timestamp": 1745276930, "schema_version": "v3", "name": "cuda12.6-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu)", "repo": "pytorch/pytorch", "head_branch": "refs/tags/ciflow/inductor/148893", "head_sha": "b68c0effe97351b0c24b2f7aef671559be6aa977", "workflow_id": 14580756816, "run_attempt": 1, "job_id": 40900324325} 2025-04-21T23:08:50.6959940Z RUNNER_INFO: [] 2025-04-21T23:08:50.6960170Z DEPENDENCIES: {} 2025-04-21T23:08:50.6960387Z ##[endgroup] 2025-04-21T23:08:50.6986340Z + [[ ! -d test/test-reports ]] 2025-04-21T23:08:50.6986960Z + [[ false == \t\r\u\e ]] 2025-04-21T23:08:50.6989138Z + python3 /home/ec2-user/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/upload-benchmark-results/../../scripts/upload_benchmark_results.py --benchmark-results-dir test/test-reports --metadata '{"timestamp": 1745276930, "schema_version": "v3", "name": "cuda12.6-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu)", "repo": "pytorch/pytorch", "head_branch": "refs/tags/ciflow/inductor/148893", "head_sha": "b68c0effe97351b0c24b2f7aef671559be6aa977", "workflow_id": 14580756816, "run_attempt": 1, "job_id": 40900324325}' --runners '[]' --dependencies '{}' 2025-04-21T23:08:50.8422367Z INFO:root:Upload test/test-reports/inference_timm_models.json to s3://ossci-benchmarks/v3/pytorch/pytorch/14580756816/40900324325/inference_timm_models.json 2025-04-21T23:08:50.8773185Z INFO:botocore.credentials:Found credentials from IAM Role: gh-ci-github-action-runners-runner-role 2025-04-21T23:08:51.0655195Z INFO:root:Upload test/test-reports/training_timm_models.json to s3://ossci-benchmarks/v3/pytorch/pytorch/14580756816/40900324325/training_timm_models.json 2025-04-21T23:08:51.2474184Z INFO:root:Upload test/test-reports/training_timm_model_graph_breaks.json to s3://ossci-benchmarks/v3/pytorch/pytorch/14580756816/40900324325/training_timm_model_graph_breaks.json 2025-04-21T23:08:51.4102358Z ##[group]Run cat test/**/*_toprint.log || true 2025-04-21T23:08:51.4102735Z cat test/**/*_toprint.log || true 2025-04-21T23:08:51.4111903Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T23:08:51.4112254Z env: 2025-04-21T23:08:51.4112466Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:51.4112772Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:51.4113288Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:51.4113744Z ##[endgroup] 2025-04-21T23:08:51.4201345Z cat: 'test/**/*_toprint.log': No such file or directory 2025-04-21T23:08:51.4238746Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2025-04-21T23:08:51.4239077Z kill "$MONITOR_SCRIPT_PID" 2025-04-21T23:08:51.4246900Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T23:08:51.4247247Z env: 2025-04-21T23:08:51.4247470Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:51.4247783Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:51.4248297Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:51.4248761Z MONITOR_SCRIPT_PID: 370981 2025-04-21T23:08:51.4249018Z ##[endgroup] 2025-04-21T23:08:51.4387089Z Prepare all required actions 2025-04-21T23:08:51.4387474Z Getting action download info 2025-04-21T23:08:51.5874747Z Download action repository 'actions/upload-artifact@v4' (SHA:ea165f8d65b6e75b540449e92b4886f43607fa02) 2025-04-21T23:08:52.0902399Z ##[group]Run ./.github/actions/upload-test-artifacts 2025-04-21T23:08:52.0902730Z with: 2025-04-21T23:08:52.0903089Z file-suffix: test-inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_40900324325 2025-04-21T23:08:52.0903529Z s3-bucket: gha-artifacts 2025-04-21T23:08:52.0903774Z env: 2025-04-21T23:08:52.0903976Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:52.0904284Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:52.0904796Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:52.0905249Z ##[endgroup] 2025-04-21T23:08:52.0946013Z ##[group]Run # Remove any previous test jsons if they exist 2025-04-21T23:08:52.0946489Z # Remove any previous test jsons if they exist 2025-04-21T23:08:52.0946835Z rm -f test-jsons-*.zip 2025-04-21T23:08:52.0947237Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test/test-reports -i '*.json' 2025-04-21T23:08:52.0956020Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T23:08:52.0956369Z env: 2025-04-21T23:08:52.0956572Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:52.0956891Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:52.0957588Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:52.0958188Z FILE_SUFFIX: test-inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_40900324325 2025-04-21T23:08:52.0958611Z ##[endgroup] 2025-04-21T23:08:52.1058826Z adding: test/test-reports/inference_timm_models.json (deflated 99%) 2025-04-21T23:08:52.1105650Z adding: test/test-reports/training_timm_models.json (deflated 99%) 2025-04-21T23:08:52.1218804Z adding: test/test-reports/training_timm_model_graph_breaks.json (deflated 98%) 2025-04-21T23:08:52.1250966Z ##[group]Run # Remove any previous test reports if they exist 2025-04-21T23:08:52.1251481Z # Remove any previous test reports if they exist 2025-04-21T23:08:52.1251864Z rm -f test-reports-*.zip 2025-04-21T23:08:52.1252315Z zip -r "test-reports-${FILE_SUFFIX}.zip" test/test-reports -i '*.xml' -i '*.csv' 2025-04-21T23:08:52.1261244Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T23:08:52.1261625Z env: 2025-04-21T23:08:52.1261840Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:52.1262156Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:52.1262679Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:52.1263300Z FILE_SUFFIX: test-inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_40900324325 2025-04-21T23:08:52.1263732Z ##[endgroup] 2025-04-21T23:08:52.1317050Z adding: test/test-reports/inference_timm_models.csv (deflated 63%) 2025-04-21T23:08:52.1317567Z adding: test/test-reports/training_timm_models.csv (deflated 61%) 2025-04-21T23:08:52.1325460Z adding: test/test-reports/training_timm_model_graph_breaks.csv (deflated 98%) 2025-04-21T23:08:52.1326279Z adding: test/test-reports/training_timm_model_graph_break_deduped.csv (deflated 74%) 2025-04-21T23:08:52.1357674Z ##[group]Run # Remove any previous usage logs if they exist 2025-04-21T23:08:52.1358124Z # Remove any previous usage logs if they exist 2025-04-21T23:08:52.1358481Z rm -f logs-*.zip 2025-04-21T23:08:52.1358817Z zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' || true 2025-04-21T23:08:52.1359293Z zip -r "logs-${FILE_SUFFIX}.zip" test/test-reports -i '*.log' || true 2025-04-21T23:08:52.1367407Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T23:08:52.1367754Z env: 2025-04-21T23:08:52.1367963Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:52.1368283Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:52.1368997Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:52.1369618Z FILE_SUFFIX: test-inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_40900324325 2025-04-21T23:08:52.1370046Z ##[endgroup] 2025-04-21T23:08:52.1462274Z adding: usage_log.txt (deflated 97%) 2025-04-21T23:08:52.1474095Z 2025-04-21T23:08:52.1474553Z zip error: Nothing to do! (logs-test-inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_40900324325.zip) 2025-04-21T23:08:52.1508101Z ##[group]Run # Remove any previous debugging artifacts if they exist 2025-04-21T23:08:52.1508595Z # Remove any previous debugging artifacts if they exist 2025-04-21T23:08:52.1508985Z rm -f debug-*.zip 2025-04-21T23:08:52.1509275Z if [ -d 'test/debug' ]; then 2025-04-21T23:08:52.1509636Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2025-04-21T23:08:52.1509993Z fi 2025-04-21T23:08:52.1518256Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T23:08:52.1518617Z env: 2025-04-21T23:08:52.1518827Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:52.1519143Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:52.1519650Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:52.1520257Z FILE_SUFFIX: test-inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_40900324325 2025-04-21T23:08:52.1520877Z ##[endgroup] 2025-04-21T23:08:52.1615886Z ##[group]Run seemethere/upload-artifact-s3@v5 2025-04-21T23:08:52.1616204Z with: 2025-04-21T23:08:52.1616404Z s3-bucket: gha-artifacts 2025-04-21T23:08:52.1616702Z s3-prefix: pytorch/pytorch/14580756816/1/artifact 2025-04-21T23:08:52.1617030Z retention-days: 14 2025-04-21T23:08:52.1617271Z if-no-files-found: warn 2025-04-21T23:08:52.1617519Z path: test-jsons-*.zip 2025-04-21T23:08:52.1617763Z name: artifact 2025-04-21T23:08:52.1617976Z region: us-east-1 2025-04-21T23:08:52.1618177Z env: 2025-04-21T23:08:52.1618374Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:52.1618692Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:52.1619199Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:52.1619655Z ##[endgroup] 2025-04-21T23:08:52.5318434Z NOTE: s3-prefix specified, ignoring name parameter 2025-04-21T23:08:52.5318874Z With the provided path, there will be 1 file uploaded 2025-04-21T23:08:52.5319325Z Uploading to s3 prefix: pytorch/pytorch/14580756816/1/artifact 2025-04-21T23:08:52.5373770Z Starting upload of test-jsons-test-inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_40900324325.zip 2025-04-21T23:08:52.7889976Z Finished upload of test-jsons-test-inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_40900324325.zip 2025-04-21T23:08:52.8179072Z ##[group]Run seemethere/upload-artifact-s3@v5 2025-04-21T23:08:52.8179398Z with: 2025-04-21T23:08:52.8179608Z s3-bucket: gha-artifacts 2025-04-21T23:08:52.8179941Z s3-prefix: pytorch/pytorch/14580756816/1/artifact 2025-04-21T23:08:52.8180325Z retention-days: 14 2025-04-21T23:08:52.8180582Z if-no-files-found: error 2025-04-21T23:08:52.8180868Z path: test-reports-*.zip 2025-04-21T23:08:52.8181136Z name: artifact 2025-04-21T23:08:52.8181361Z region: us-east-1 2025-04-21T23:08:52.8181587Z env: 2025-04-21T23:08:52.8181798Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:52.8182144Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:52.8182744Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:52.8183280Z ##[endgroup] 2025-04-21T23:08:53.1514438Z NOTE: s3-prefix specified, ignoring name parameter 2025-04-21T23:08:53.1514862Z With the provided path, there will be 1 file uploaded 2025-04-21T23:08:53.1515275Z Uploading to s3 prefix: pytorch/pytorch/14580756816/1/artifact 2025-04-21T23:08:53.1568846Z Starting upload of test-reports-test-inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_40900324325.zip 2025-04-21T23:08:53.2906623Z Finished upload of test-reports-test-inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_40900324325.zip 2025-04-21T23:08:53.3192531Z ##[group]Run seemethere/upload-artifact-s3@v5 2025-04-21T23:08:53.3192857Z with: 2025-04-21T23:08:53.3193065Z s3-bucket: gha-artifacts 2025-04-21T23:08:53.3193377Z s3-prefix: pytorch/pytorch/14580756816/1/artifact 2025-04-21T23:08:53.3193710Z retention-days: 14 2025-04-21T23:08:53.3193954Z if-no-files-found: ignore 2025-04-21T23:08:53.3194229Z path: logs-*.zip 2025-04-21T23:08:53.3194454Z name: artifact 2025-04-21T23:08:53.3194672Z region: us-east-1 2025-04-21T23:08:53.3194892Z env: 2025-04-21T23:08:53.3195105Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:53.3195420Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:53.3195945Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:53.3196409Z ##[endgroup] 2025-04-21T23:08:53.6614341Z NOTE: s3-prefix specified, ignoring name parameter 2025-04-21T23:08:53.6614777Z With the provided path, there will be 1 file uploaded 2025-04-21T23:08:53.6615201Z Uploading to s3 prefix: pytorch/pytorch/14580756816/1/artifact 2025-04-21T23:08:53.6669868Z Starting upload of logs-test-inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_40900324325.zip 2025-04-21T23:08:53.8061516Z Finished upload of logs-test-inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_40900324325.zip 2025-04-21T23:08:53.8349121Z ##[group]Run seemethere/upload-artifact-s3@v5 2025-04-21T23:08:53.8349642Z with: 2025-04-21T23:08:53.8349851Z s3-bucket: gha-artifacts 2025-04-21T23:08:53.8350163Z s3-prefix: pytorch/pytorch/14580756816/1/artifact 2025-04-21T23:08:53.8350498Z retention-days: 14 2025-04-21T23:08:53.8350742Z if-no-files-found: ignore 2025-04-21T23:08:53.8351001Z path: debug-*.zip 2025-04-21T23:08:53.8351226Z name: artifact 2025-04-21T23:08:53.8351440Z region: us-east-1 2025-04-21T23:08:53.8351658Z env: 2025-04-21T23:08:53.8351863Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:53.8352182Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:53.8352710Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:53.8353172Z ##[endgroup] 2025-04-21T23:08:54.1643198Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2025-04-21T23:08:54.1943071Z ##[group]Run # shellcheck disable=SC2156 2025-04-21T23:08:54.1943431Z # shellcheck disable=SC2156 2025-04-21T23:08:54.1943984Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2025-04-21T23:08:54.1952964Z shell: /usr/bin/bash -e {0} 2025-04-21T23:08:54.1953218Z env: 2025-04-21T23:08:54.1953420Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:54.1953726Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:54.1954241Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:54.1954701Z ##[endgroup] 2025-04-21T23:08:54.4779492Z Prepare all required actions 2025-04-21T23:08:54.4779856Z Getting action download info 2025-04-21T23:08:54.5753829Z ##[group]Run ./.github/actions/upload-utilization-stats 2025-04-21T23:08:54.5754170Z with: 2025-04-21T23:08:54.5754362Z job_id: 40900324325 2025-04-21T23:08:54.5754786Z job_name: cuda12.6-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2025-04-21T23:08:54.5755261Z workflow_name: inductor 2025-04-21T23:08:54.5755523Z workflow_run_id: 14580756816 2025-04-21T23:08:54.5755783Z workflow_attempt: 1 2025-04-21T23:08:54.5756007Z env: 2025-04-21T23:08:54.5756201Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:54.5756513Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:54.5757027Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:54.5757482Z ##[endgroup] 2025-04-21T23:08:54.5777721Z ##[group]Run echo "workflow_id: 14580756816" 2025-04-21T23:08:54.5778079Z echo "workflow_id: 14580756816" 2025-04-21T23:08:54.5778383Z echo "workflow_attempt: 1" 2025-04-21T23:08:54.5778686Z echo "workflow_Name: inductor" 2025-04-21T23:08:54.5778982Z echo "job_id: 40900324325" 2025-04-21T23:08:54.5779480Z echo "job_name: cuda12.6-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu)" 2025-04-21T23:08:54.5788182Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T23:08:54.5788540Z env: 2025-04-21T23:08:54.5788735Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:54.5789056Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:54.5789568Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:54.5790019Z ##[endgroup] 2025-04-21T23:08:54.5818022Z workflow_id: 14580756816 2025-04-21T23:08:54.5818307Z workflow_attempt: 1 2025-04-21T23:08:54.5818637Z workflow_Name: inductor 2025-04-21T23:08:54.5818913Z job_id: 40900324325 2025-04-21T23:08:54.5819336Z job_name: cuda12.6-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu) 2025-04-21T23:08:54.5858584Z ##[group]Run nick-fields/retry@v3.0.0 2025-04-21T23:08:54.5858885Z with: 2025-04-21T23:08:54.5859099Z shell: bash 2025-04-21T23:08:54.5859320Z timeout_minutes: 5 2025-04-21T23:08:54.5859551Z max_attempts: 5 2025-04-21T23:08:54.5859791Z retry_wait_seconds: 30 2025-04-21T23:08:54.5871422Z command: set -eu python3 -m pip install python-dateutil==2.8.2 boto3==1.35.42 pandas==2.1.3 2025-04-21T23:08:54.5872173Z polling_interval_seconds: 1 2025-04-21T23:08:54.5872457Z warning_on_retry: true 2025-04-21T23:08:54.5872722Z continue_on_error: false 2025-04-21T23:08:54.5872963Z env: 2025-04-21T23:08:54.5873176Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:54.5873492Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:54.5874002Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:54.5874470Z ##[endgroup] 2025-04-21T23:08:54.9179824Z Defaulting to user installation because normal site-packages is not writeable 2025-04-21T23:08:54.9363346Z Requirement already satisfied: python-dateutil==2.8.2 in /home/ec2-user/.local/lib/python3.9/site-packages (2.8.2) 2025-04-21T23:08:56.0374643Z Collecting boto3==1.35.42 2025-04-21T23:08:56.0404692Z Using cached boto3-1.35.42-py3-none-any.whl (139 kB) 2025-04-21T23:08:56.0428243Z Requirement already satisfied: pandas==2.1.3 in /home/ec2-user/.local/lib/python3.9/site-packages (2.1.3) 2025-04-21T23:08:56.0442970Z Requirement already satisfied: six>=1.5 in /usr/lib/python3.9/site-packages (from python-dateutil==2.8.2) (1.15.0) 2025-04-21T23:08:56.0488716Z Requirement already satisfied: s3transfer<0.11.0,>=0.10.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from boto3==1.35.42) (0.10.4) 2025-04-21T23:08:56.0493143Z Requirement already satisfied: botocore<1.36.0,>=1.35.42 in /home/ec2-user/.local/lib/python3.9/site-packages (from boto3==1.35.42) (1.35.99) 2025-04-21T23:08:56.0497819Z Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /usr/lib/python3.9/site-packages (from boto3==1.35.42) (0.10.0) 2025-04-21T23:08:56.1371583Z Requirement already satisfied: numpy<2,>=1.22.4 in /home/ec2-user/.local/lib/python3.9/site-packages (from pandas==2.1.3) (1.26.4) 2025-04-21T23:08:56.1373958Z Requirement already satisfied: pytz>=2020.1 in /usr/lib/python3.9/site-packages (from pandas==2.1.3) (2022.7.1) 2025-04-21T23:08:56.1380980Z Requirement already satisfied: tzdata>=2022.1 in /home/ec2-user/.local/lib/python3.9/site-packages (from pandas==2.1.3) (2025.2) 2025-04-21T23:08:56.1437979Z Requirement already satisfied: urllib3<1.27,>=1.25.4 in /usr/lib/python3.9/site-packages (from botocore<1.36.0,>=1.35.42->boto3==1.35.42) (1.25.10) 2025-04-21T23:08:56.2265142Z Installing collected packages: boto3 2025-04-21T23:08:56.2265706Z Attempting uninstall: boto3 2025-04-21T23:08:56.2268069Z Found existing installation: boto3 1.35.33 2025-04-21T23:08:56.2393217Z Uninstalling boto3-1.35.33: 2025-04-21T23:08:56.2410044Z Successfully uninstalled boto3-1.35.33 2025-04-21T23:08:56.2982476Z Successfully installed boto3-1.35.42 2025-04-21T23:08:56.6666470Z Command completed after 1 attempt(s). 2025-04-21T23:08:56.6736954Z ##[group]Run python3 -m tools.stats.upload_utilization_stats.upload_utilization_stats \ 2025-04-21T23:08:56.6737605Z python3 -m tools.stats.upload_utilization_stats.upload_utilization_stats \ 2025-04-21T23:08:56.6738089Z  --workflow-run-id "14580756816" \ 2025-04-21T23:08:56.6738423Z  --workflow-name "inductor" \ 2025-04-21T23:08:56.6738744Z  --workflow-run-attempt "1" \ 2025-04-21T23:08:56.6739042Z  --job-id "40900324325" \ 2025-04-21T23:08:56.6739538Z  --job-name "cuda12.6-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu)" 2025-04-21T23:08:56.6748460Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T23:08:56.6748809Z env: 2025-04-21T23:08:56.6749024Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:56.6749344Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:56.6749862Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:56.6750320Z ##[endgroup] 2025-04-21T23:08:58.4269731Z repo: pytorch/pytorch 2025-04-21T23:08:58.4270677Z Downloading logs-test-inductor_timm-2-2-linux.g5.4xlarge.nvidia.gpu_40900324325.zip 2025-04-21T23:08:58.4272167Z Converted Log Model: UtilizationMetadata: 2025-04-21T23:08:58.4274692Z UtilizationMetadata(level='metadata', workflow_id='14580756816', job_id='40900324325', workflow_name='inductor', job_name='cuda12.6-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu)', usage_collect_interval=1.0, data_model_version=1.0, start_at=1745272226, gpu_count=1, cpu_count=16, gpu_type='pynvml', error=None) 2025-04-21T23:08:58.4276376Z [Db Segments] detected pytest cmd: 12, generated segments: 12 2025-04-21T23:08:58.4276770Z [db model] Peek db timeseries 2025-04-21T23:08:58.4277022Z :{ 2025-04-21T23:08:58.4277225Z "created_at": 1745276938, 2025-04-21T23:08:58.4277488Z "type": "utilization", 2025-04-21T23:08:58.4277723Z "tags": [ 2025-04-21T23:08:58.4277928Z "record" 2025-04-21T23:08:58.4278137Z ], 2025-04-21T23:08:58.4278334Z "time_stamp": 1745272226, 2025-04-21T23:08:58.4278597Z "repo": "pytorch/pytorch", 2025-04-21T23:08:58.4278860Z "workflow_id": 14580756816, 2025-04-21T23:08:58.4279117Z "run_attempt": 1, 2025-04-21T23:08:58.4279353Z "job_id": 40900324325, 2025-04-21T23:08:58.4279602Z "workflow_name": "inductor", 2025-04-21T23:08:58.4280058Z "job_name": "cuda12.6-py3.10-gcc9-sm86 / test (inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu)", 2025-04-21T23:08:58.4280529Z "json_data": "{}" 2025-04-21T23:08:58.4280747Z } 2025-04-21T23:08:58.4281211Z Writing 1 documents to S3 ossci-utilization/util_metadata/v_1.0/pytorch/pytorch/14580756816/1/40900324325/metadata 2025-04-21T23:08:58.4282284Z Done! Finish writing document to S3 ossci-utilization/util_metadata/v_1.0/pytorch/pytorch/14580756816/1/40900324325/metadata 2025-04-21T23:08:58.4283147Z Writing 931 documents to S3 ossci-utilization/util_timeseries/v_1.0/pytorch/pytorch/14580756816/1/40900324325/time_series 2025-04-21T23:08:58.4284018Z Done! Finish writing document to S3 ossci-utilization/util_timeseries/v_1.0/pytorch/pytorch/14580756816/1/40900324325/time_series 2025-04-21T23:08:58.5447526Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2025-04-21T23:08:58.5447944Z with: 2025-04-21T23:08:58.5448125Z env: 2025-04-21T23:08:58.5448314Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:58.5448623Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:58.5449127Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:58.5449572Z ##[endgroup] 2025-04-21T23:08:58.5468949Z ##[group]Run set -eou pipefail 2025-04-21T23:08:58.5469254Z set -eou pipefail 2025-04-21T23:08:58.5469505Z  2025-04-21T23:08:58.5469840Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2025-04-21T23:08:58.5470266Z for _ in $(seq 1440); do 2025-04-21T23:08:58.5470574Z  # Break if no ssh session exists anymore 2025-04-21T23:08:58.5470897Z  if [ "$(who)" = "" ]; then 2025-04-21T23:08:58.5471166Z  break 2025-04-21T23:08:58.5471417Z  fi 2025-04-21T23:08:58.5471628Z  echo "." 2025-04-21T23:08:58.5471854Z  sleep 5 2025-04-21T23:08:58.5472073Z done 2025-04-21T23:08:58.5481082Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T23:08:58.5481435Z env: 2025-04-21T23:08:58.5481645Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:58.5481967Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:58.5482475Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:58.5482932Z ##[endgroup] 2025-04-21T23:08:58.5509249Z Holding runner for 2 hours until all ssh sessions have logged out 2025-04-21T23:08:58.5579360Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2025-04-21T23:08:58.5580065Z # ignore expansion of "docker ps -q" since it could be empty 2025-04-21T23:08:58.5580520Z # shellcheck disable=SC2046 2025-04-21T23:08:58.5580893Z docker stop $(docker ps -q) || true 2025-04-21T23:08:58.5581389Z # Prune all of the docker images 2025-04-21T23:08:58.5581708Z docker system prune -af 2025-04-21T23:08:58.5590551Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T23:08:58.5590908Z env: 2025-04-21T23:08:58.5591117Z GIT_DEFAULT_BRANCH: main 2025-04-21T23:08:58.5591436Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2025-04-21T23:08:58.5591950Z DOCKER_CONTAINER_ID: f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:08:58.5592416Z ##[endgroup] 2025-04-21T23:08:59.2223817Z f3774c702cf4 2025-04-21T23:09:00.9120514Z Deleted Containers: 2025-04-21T23:09:00.9120920Z f3774c702cf44fb71556b92f51531f17d7013b2afe51fc7ca201f08e2a6fcf52 2025-04-21T23:09:00.9121226Z 2025-04-21T23:09:16.6750959Z Deleted Images: 2025-04-21T23:09:16.6751861Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.6-cudnn9-py3-gcc9-inductor-benchmarks:f583c1656c56372ad0e1c57f3463a017fbf03138 2025-04-21T23:09:16.6753347Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.6-cudnn9-py3-gcc9-inductor-benchmarks@sha256:745e08885687216ce5adf3196dea7f49a9467042491210addb990693d7fef90a 2025-04-21T23:09:16.6754456Z deleted: sha256:7b29eacbf93f5bdd9e6b666f27edb34c0d2f289aa58ceaf3bdfcf4e06f505e55 2025-04-21T23:09:16.6755074Z deleted: sha256:0d03769fa66092351763fc95d7ff70e856e1c6dad39dffd06cd3cc6f705c2b90 2025-04-21T23:09:16.6755685Z deleted: sha256:03ad561430a1da2dad623afcca922b9bf999b9beec551534fb227d2905d98683 2025-04-21T23:09:16.6756358Z deleted: sha256:8999b0bbe1ce0aa447deea13eab8ab1f8c7e23d827f1cfb9ea3eda2d5862ff30 2025-04-21T23:09:16.6756956Z deleted: sha256:8c9a34c905c38106c707a3de05f76313f1dc1b5480e33030262e0c036fd86e2e 2025-04-21T23:09:16.6757537Z deleted: sha256:7aa296e90575147a3bdf977176396c671082fb5dc5c240a22ac72b9e1bc0e268 2025-04-21T23:09:16.6758130Z deleted: sha256:35429085a82ca073e718dc4ecc1c57868bb6c3ca77201e12ea81aa653bdde323 2025-04-21T23:09:16.6759101Z deleted: sha256:9a73de98215b59030f9b38fd04ed9f006b41baa1a1df96e1b130fd6771aafa70 2025-04-21T23:09:16.6759706Z deleted: sha256:aa989e7f91a166b7b6f2138d15d3a9eab2852c094c0f76f2ef4b85a015d5247a 2025-04-21T23:09:16.6760310Z deleted: sha256:aab94433009bfa1b6b672cbf53be6cfcc67e12bae3fccb39392f2ba356fed8e8 2025-04-21T23:09:16.6760920Z deleted: sha256:cf4b1db59c30d2e1191b22f938168f28c237ca50a4479f4bd96300300167a10d 2025-04-21T23:09:16.6761505Z deleted: sha256:d80a61f49181681ea66aabe7e2bd1a43e3652d30fbe47699b3c75bc320b3fd86 2025-04-21T23:09:16.6762097Z deleted: sha256:7ce0d2a5b6420f3c29e66c3a4c4e37437e961ea57a801549d154c17377b9cbe7 2025-04-21T23:09:16.6762696Z deleted: sha256:b035e1e0eefb5d757d8642ca66b5a56d8acd60b27793d8bb0fee024b61424f18 2025-04-21T23:09:16.6763291Z deleted: sha256:e9f028a07e4a18a1eb31d1377ffdc18189e530d827ec085900dab9634404cbfd 2025-04-21T23:09:16.6763895Z deleted: sha256:75a2b3d5fb5eabe18f21153ffe34412d0f6adb78a89cbb579271dc29a107109f 2025-04-21T23:09:16.6764501Z deleted: sha256:980637ca90042fdf0d35c1aa5459851186b2d884c583ac0c6018565e4123275d 2025-04-21T23:09:16.6765097Z deleted: sha256:353541bdf9a9dc69f68d46df3e6af379ce0d8fb710e2ae3ed989b4fd73a7f69e 2025-04-21T23:09:16.6765697Z deleted: sha256:9739cf32fe6a869643327bf4f2ddbdfd714451c07e0463e887cfc7827acf81b8 2025-04-21T23:09:16.6766340Z deleted: sha256:ea8da7f58fe158b166472f208d4787bd84a5f24715f3a3693b2ce09db3b56870 2025-04-21T23:09:16.6766924Z deleted: sha256:64d128ae465a7433018bb4f041b913f2c3d4fe4f7a4191fb15a976492613d25a 2025-04-21T23:09:16.6767504Z deleted: sha256:407bfba786761588aea6fe18b2486cd23dddd885d72c60a41949d7dfc6a942d5 2025-04-21T23:09:16.6768130Z deleted: sha256:9889d65ed67766c496c923d00c5e6fcf5d77647d1384b278c39e389fc2a99f71 2025-04-21T23:09:16.6768718Z deleted: sha256:885a49a89eaa7cd605832bad81aaef59d663c8b6a9941c4fd8a56414a5c15361 2025-04-21T23:09:16.6769312Z deleted: sha256:a754629243d15f78ad5b02c613ca5b138bae988a653e77ab7d7a1c2b1ff8430b 2025-04-21T23:09:16.6769920Z deleted: sha256:69986b2cde5dc2eedee2c34cc9a0f2862bd0bb76de325a159b3eb286e31ab267 2025-04-21T23:09:16.6770706Z deleted: sha256:1eb1b82d13079306ebed0331351fb0299aab14b537bd38ac3dfe9f86daba4a7c 2025-04-21T23:09:16.6771299Z deleted: sha256:3a53d5f678b95144bfb2f43df7c8f62c6eacdc36c4c9278d2672658ba981a6f1 2025-04-21T23:09:16.6771883Z deleted: sha256:816f3880d16e691a376b22135e30c5c92e77a2390692a2aec75c92a61a1028d6 2025-04-21T23:09:16.6772457Z deleted: sha256:6e9897a601a0378a8a2499be37549ea2df6a9dfbec84fad00d2e0796b0d8c897 2025-04-21T23:09:16.6773048Z deleted: sha256:4253008065add4d2e0a51dd0dc024420211cfa90f52ccf2fb3cd59a97941e820 2025-04-21T23:09:16.6773640Z deleted: sha256:84fc991bd2259ddde33c56bd6d83cd6856ce393c638a21f6c61183206847f599 2025-04-21T23:09:16.6774225Z deleted: sha256:52c94651685b1b2c788a091c20c8325005b4cb03693709ff02b9e659e790f122 2025-04-21T23:09:16.6774794Z deleted: sha256:afa943b0954b848b45f79913b746502e986aba75e5a7393fb4569f3450da4315 2025-04-21T23:09:16.6775664Z deleted: sha256:b3d00015deda08e79f7f4c3b4bf277c14ddf7cc07986f983b4e160ea069ffc47 2025-04-21T23:09:16.6776274Z deleted: sha256:e3b8a4f8a44b3187a6f78df4ee606a2a8ec8060bfca4163e47c3db94ff685c9e 2025-04-21T23:09:16.6776860Z deleted: sha256:5ad2fc574f42e35faf6c336d17334edc2e67b6601a49fbe3c12000a8091da6e8 2025-04-21T23:09:16.6777446Z deleted: sha256:397028d746ad951508252fafab3946aa1b8825cd1d8edb6c947cc54c9db7576a 2025-04-21T23:09:16.6778019Z deleted: sha256:454a57acb38698129b7675d5e6d4358906230673548a777f8a3dec9c4fab8500 2025-04-21T23:09:16.6778594Z deleted: sha256:95f1c8b10ced79e9ce5a542b118703418c66d52cc43cc8e7c9405916383bb5ac 2025-04-21T23:09:16.6779183Z deleted: sha256:06af89fbc9fb5a1ff9cc0ddaa04378aa25ac8a7d9e16b77b6aa6a48558d10cd6 2025-04-21T23:09:16.6779788Z deleted: sha256:d4d7f04a8b38bc30eb120874f5d53b51aefc2754f34b43cd5bcad3b58f14791d 2025-04-21T23:09:16.6780377Z deleted: sha256:a39fece22c881dde7689a0873a48724a18471faf44ad4de34c5987032a2f8aed 2025-04-21T23:09:16.6780952Z deleted: sha256:0b59474a1f61590a4946eb94287e7b9531ed471d49b34516ada330bf6e62ef3d 2025-04-21T23:09:16.6781812Z deleted: sha256:76af19197c611f8dd2aec8623ffa2ae9dfc96031e57c885acbf763d47cd4e10a 2025-04-21T23:09:16.6782507Z deleted: sha256:cec724490b7de23aa9af56de906b95d0722b33e990443300bc5fbaafd2d6ce7f 2025-04-21T23:09:16.6783220Z deleted: sha256:993012f9b10558b092cbaa6afb2f8de46c856adf9f39ccb3affb1ab88b4034f5 2025-04-21T23:09:16.6783807Z deleted: sha256:c03f3c99e80a992966c8a13067ca8d551914f2016063aabd58e10fc66a0b4aa7 2025-04-21T23:09:16.6784396Z deleted: sha256:a3826acb0870ba3e5dcb72651ebbae184d8964a70dfc99d88a6ada64df52070a 2025-04-21T23:09:16.6784995Z deleted: sha256:59eff532c4a87d4492cfb7a2dca5f091aba58f41f09dd44d78ed251566552fc0 2025-04-21T23:09:16.6785666Z deleted: sha256:7af9312575f7d280f4e404bfa4229d29b2f863308cd2847737dab0a80f912abd 2025-04-21T23:09:16.6786235Z deleted: sha256:255b3dc15181408481cc137ae98284eb8bbe550185f4c3600538c7f27cb97630 2025-04-21T23:09:16.6786802Z deleted: sha256:a46c03834bc35bcd52270359c7503317e2517a1bef96d6174b37510d49240a96 2025-04-21T23:09:16.6787378Z deleted: sha256:697ce89585f3321899a308e7d5fa44cfd01661762b83d4886fe6c7df23067deb 2025-04-21T23:09:16.6787959Z deleted: sha256:fe1a83ec01181b320ed6d1d9d96c7b05e0ec90ecae542be803fc5a55fd49563c 2025-04-21T23:09:16.6788551Z deleted: sha256:47a48329e8ee10fb1592b7f54749c23245dfdbdecb46c08c4b79dd9b141a13b9 2025-04-21T23:09:16.6789129Z deleted: sha256:b7b7528c3adac4746b228449b90274fd02e75b69ff3f626366683dea239108b4 2025-04-21T23:09:16.6789707Z deleted: sha256:b1563735209b77977fec8743d8915d3bb4acecdce565bf90f42e43b7a9d41eba 2025-04-21T23:09:16.6790288Z deleted: sha256:8f3dad83b005c9344155ed29ce5613f918718d2ceab31a2fc20adf404021a9ee 2025-04-21T23:09:16.6790881Z deleted: sha256:d19dff217a30a035796fe482ed413f6eda45d02ca46daa5cd9ce30ac4a5b86a9 2025-04-21T23:09:16.6791479Z deleted: sha256:ec94e4b914db9f6e5913eb4a203bf4c14cf9dffa794aaf0d01426e1d2d540662 2025-04-21T23:09:16.6792056Z deleted: sha256:94e49b5119d7680f10fa65841158de93cc070c7b19bb83c566b874158bec2fc3 2025-04-21T23:09:16.6792630Z deleted: sha256:6866ef40cf6f50d4d4c3823444758606f80314d683f650be1b6d69eff2d35329 2025-04-21T23:09:16.6793370Z deleted: sha256:f06c4dcc941cd4fa9c6b0f507c8be752e7be6b2191a3332128ab32fdb8c613eb 2025-04-21T23:09:16.6793976Z deleted: sha256:e7a9c0fbc4da83cebcbeab6a1ffc42508561041f97259ff2cf2e2e14a73d47bd 2025-04-21T23:09:16.6794563Z deleted: sha256:67f07ac2be3331b5421f04c08746bdb4ce5952c7094018813bdc134ddefd5ec4 2025-04-21T23:09:16.6795139Z deleted: sha256:9a1906e08abd2cbaad6411b296796a97964e0f89011633541b1fd2ac24e8d53a 2025-04-21T23:09:16.6795721Z deleted: sha256:594048c05b4663fff16eed21ea2fe1cba949ca1b0426b134bb275181ab727272 2025-04-21T23:09:16.6796305Z deleted: sha256:9ab83b5a3614325d1733a24e22eeae2e7ecd37cd0d0024660d9b43265e7c496b 2025-04-21T23:09:16.6796880Z deleted: sha256:de82c3b65910240582bad1cbc96209ab68e59c252450a7d42aad3a0673b154f4 2025-04-21T23:09:16.6797462Z deleted: sha256:46bc902662130b8d13f25b0294fac16ca6248034d5ccfaf3ee51cb56572946c5 2025-04-21T23:09:16.6798091Z deleted: sha256:43659ee56fac7f221f9388ec263ad2042847103213025f0a7a0ec7ca6a81b20f 2025-04-21T23:09:16.6798666Z deleted: sha256:5839c16cf80c38c4f69268b2c73c3d7ec37a33e18c9c5f49e6624239a182dbf8 2025-04-21T23:09:16.6799248Z deleted: sha256:e16d8e25c4393c96891af6d6a42889638fc30a7490ef9c5c5517798ea9860bda 2025-04-21T23:09:16.6799819Z deleted: sha256:868539926d1d83fe2b4fdb0231d8b57fb5b8b34144d733a4cf862d241e16c667 2025-04-21T23:09:16.6800404Z deleted: sha256:cc4224d01139aef3aeb29c568d184283602d52f47bb8e5e29de97bf6aa54d951 2025-04-21T23:09:16.6800985Z deleted: sha256:bd18b4fe898c9279c6e9ad952165522e24de55ab48d83289ca0ff6c8df97d85d 2025-04-21T23:09:16.6801571Z deleted: sha256:c7917cebb757d994f33b0f4bbad70fbb563939c2630d8f96113d437fa09e68cd 2025-04-21T23:09:16.6802159Z deleted: sha256:34fb1ecb50c48f883ea450e0364c90bdf16ce5e35e8a132012851ac3b040addf 2025-04-21T23:09:16.6802743Z deleted: sha256:92300fc3f1a943c0b6a295d87da98cb3d593b18b8ddc4f0b229e3ca909d03206 2025-04-21T23:09:16.6803338Z deleted: sha256:fffe76c64ef2dee2d80a8bb3ad13d65d596d04a45510b1956a976a69215dae92 2025-04-21T23:09:16.6803697Z 2025-04-21T23:09:16.6803908Z Total reclaimed space: 46.68GB 2025-04-21T23:09:16.6893249Z Post job cleanup. 2025-04-21T23:09:16.6943971Z Post job cleanup. 2025-04-21T23:09:16.7920654Z [command]/usr/bin/git version 2025-04-21T23:09:16.7963892Z git version 2.47.1 2025-04-21T23:09:16.8001016Z Copying '/home/ec2-user/.gitconfig' to '/home/ec2-user/actions-runner/_work/_temp/610bee24-264c-4163-9a9f-c0aeb810a2b6/.gitconfig' 2025-04-21T23:09:16.8012673Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/610bee24-264c-4163-9a9f-c0aeb810a2b6' before making global git config changes 2025-04-21T23:09:16.8013546Z Adding repository directory to the temporary git global config as a safe directory 2025-04-21T23:09:16.8018105Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2025-04-21T23:09:16.8060894Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2025-04-21T23:09:16.8105857Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2025-04-21T23:09:16.8473648Z Entering 'android/libs/fbjni' 2025-04-21T23:09:16.8541942Z Entering 'third_party/FP16' 2025-04-21T23:09:16.8609126Z Entering 'third_party/FXdiv' 2025-04-21T23:09:16.8677786Z Entering 'third_party/NNPACK' 2025-04-21T23:09:16.8744095Z Entering 'third_party/NVTX' 2025-04-21T23:09:16.8817263Z Entering 'third_party/VulkanMemoryAllocator' 2025-04-21T23:09:16.8892231Z Entering 'third_party/XNNPACK' 2025-04-21T23:09:16.8976387Z Entering 'third_party/benchmark' 2025-04-21T23:09:16.9048591Z Entering 'third_party/composable_kernel' 2025-04-21T23:09:16.9122494Z Entering 'third_party/cpp-httplib' 2025-04-21T23:09:16.9189759Z Entering 'third_party/cpuinfo' 2025-04-21T23:09:16.9257098Z Entering 'third_party/cudnn_frontend' 2025-04-21T23:09:16.9324890Z Entering 'third_party/cutlass' 2025-04-21T23:09:16.9401383Z Entering 'third_party/eigen' 2025-04-21T23:09:16.9471312Z Entering 'third_party/fbgemm' 2025-04-21T23:09:16.9539178Z Entering 'third_party/fbgemm/third_party/asmjit' 2025-04-21T23:09:16.9606425Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2025-04-21T23:09:16.9673159Z Entering 'third_party/fbgemm/third_party/cutlass' 2025-04-21T23:09:16.9747280Z Entering 'third_party/fbgemm/third_party/googletest' 2025-04-21T23:09:16.9814226Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2025-04-21T23:09:16.9884997Z Entering 'third_party/flash-attention' 2025-04-21T23:09:16.9952243Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-04-21T23:09:17.0025787Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-04-21T23:09:17.0103859Z Entering 'third_party/flatbuffers' 2025-04-21T23:09:17.0175601Z Entering 'third_party/fmt' 2025-04-21T23:09:17.0251030Z Entering 'third_party/gemmlowp/gemmlowp' 2025-04-21T23:09:17.0319028Z Entering 'third_party/gloo' 2025-04-21T23:09:17.0388662Z Entering 'third_party/googletest' 2025-04-21T23:09:17.0455482Z Entering 'third_party/ideep' 2025-04-21T23:09:17.0527341Z Entering 'third_party/ideep/mkl-dnn' 2025-04-21T23:09:17.0603080Z Entering 'third_party/ittapi' 2025-04-21T23:09:17.0670098Z Entering 'third_party/kineto' 2025-04-21T23:09:17.0740012Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-04-21T23:09:17.0805160Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-04-21T23:09:17.0874637Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-04-21T23:09:17.0942355Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-04-21T23:09:17.1010529Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-04-21T23:09:17.1075656Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-04-21T23:09:17.1150260Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-04-21T23:09:17.1217378Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-04-21T23:09:17.1292307Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-04-21T23:09:17.1360671Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-04-21T23:09:17.1433568Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-04-21T23:09:17.1502259Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-04-21T23:09:17.1571801Z Entering 'third_party/kleidiai' 2025-04-21T23:09:17.1639364Z Entering 'third_party/mimalloc' 2025-04-21T23:09:17.1707069Z Entering 'third_party/nlohmann' 2025-04-21T23:09:17.1775441Z Entering 'third_party/onnx' 2025-04-21T23:09:17.1856247Z Entering 'third_party/onnx/third_party/pybind11' 2025-04-21T23:09:17.1928800Z Entering 'third_party/opentelemetry-cpp' 2025-04-21T23:09:17.1997259Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-04-21T23:09:17.2062431Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-04-21T23:09:17.2131431Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-04-21T23:09:17.2198028Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-04-21T23:09:17.2264416Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-04-21T23:09:17.2330799Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-04-21T23:09:17.2396280Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-04-21T23:09:17.2460363Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-04-21T23:09:17.2529715Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-04-21T23:09:17.2601718Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-04-21T23:09:17.2689455Z Entering 'third_party/pocketfft' 2025-04-21T23:09:17.2756204Z Entering 'third_party/protobuf' 2025-04-21T23:09:17.2830147Z Entering 'third_party/protobuf/third_party/benchmark' 2025-04-21T23:09:17.2898247Z Entering 'third_party/protobuf/third_party/googletest' 2025-04-21T23:09:17.2971209Z Entering 'third_party/psimd' 2025-04-21T23:09:17.3041674Z Entering 'third_party/pthreadpool' 2025-04-21T23:09:17.3112619Z Entering 'third_party/pybind11' 2025-04-21T23:09:17.3180729Z Entering 'third_party/python-peachpy' 2025-04-21T23:09:17.3247817Z Entering 'third_party/sleef' 2025-04-21T23:09:17.3315598Z Entering 'third_party/tensorpipe' 2025-04-21T23:09:17.3381992Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-04-21T23:09:17.3448409Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-04-21T23:09:17.3515194Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-04-21T23:09:17.3582202Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-04-21T23:09:17.3648271Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-04-21T23:09:17.3749140Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2025-04-21T23:09:17.3774335Z http.https://github.com/.extraheader 2025-04-21T23:09:17.3788381Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2025-04-21T23:09:17.3825692Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2025-04-21T23:09:17.4185363Z Entering 'android/libs/fbjni' 2025-04-21T23:09:17.4229507Z http.https://github.com/.extraheader 2025-04-21T23:09:17.4273714Z Entering 'third_party/FP16' 2025-04-21T23:09:17.4317851Z http.https://github.com/.extraheader 2025-04-21T23:09:17.4361264Z Entering 'third_party/FXdiv' 2025-04-21T23:09:17.4405934Z http.https://github.com/.extraheader 2025-04-21T23:09:17.4453133Z Entering 'third_party/NNPACK' 2025-04-21T23:09:17.4497265Z http.https://github.com/.extraheader 2025-04-21T23:09:17.4540692Z Entering 'third_party/NVTX' 2025-04-21T23:09:17.4584169Z http.https://github.com/.extraheader 2025-04-21T23:09:17.4627807Z Entering 'third_party/VulkanMemoryAllocator' 2025-04-21T23:09:17.4671243Z http.https://github.com/.extraheader 2025-04-21T23:09:17.4715010Z Entering 'third_party/XNNPACK' 2025-04-21T23:09:17.4758132Z http.https://github.com/.extraheader 2025-04-21T23:09:17.4815730Z Entering 'third_party/benchmark' 2025-04-21T23:09:17.4860739Z http.https://github.com/.extraheader 2025-04-21T23:09:17.4905843Z Entering 'third_party/composable_kernel' 2025-04-21T23:09:17.4950117Z http.https://github.com/.extraheader 2025-04-21T23:09:17.5001024Z Entering 'third_party/cpp-httplib' 2025-04-21T23:09:17.5044735Z http.https://github.com/.extraheader 2025-04-21T23:09:17.5088015Z Entering 'third_party/cpuinfo' 2025-04-21T23:09:17.5131794Z http.https://github.com/.extraheader 2025-04-21T23:09:17.5177209Z Entering 'third_party/cudnn_frontend' 2025-04-21T23:09:17.5221408Z http.https://github.com/.extraheader 2025-04-21T23:09:17.5265275Z Entering 'third_party/cutlass' 2025-04-21T23:09:17.5310305Z http.https://github.com/.extraheader 2025-04-21T23:09:17.5362980Z Entering 'third_party/eigen' 2025-04-21T23:09:17.5408277Z http.https://github.com/.extraheader 2025-04-21T23:09:17.5453989Z Entering 'third_party/fbgemm' 2025-04-21T23:09:17.5498463Z http.https://github.com/.extraheader 2025-04-21T23:09:17.5542698Z Entering 'third_party/fbgemm/third_party/asmjit' 2025-04-21T23:09:17.5586023Z http.https://github.com/.extraheader 2025-04-21T23:09:17.5630605Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2025-04-21T23:09:17.5674263Z http.https://github.com/.extraheader 2025-04-21T23:09:17.5717270Z Entering 'third_party/fbgemm/third_party/cutlass' 2025-04-21T23:09:17.5761500Z http.https://github.com/.extraheader 2025-04-21T23:09:17.5811349Z Entering 'third_party/fbgemm/third_party/googletest' 2025-04-21T23:09:17.5854298Z http.https://github.com/.extraheader 2025-04-21T23:09:17.5899199Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2025-04-21T23:09:17.5946868Z http.https://github.com/.extraheader 2025-04-21T23:09:17.5991954Z Entering 'third_party/flash-attention' 2025-04-21T23:09:17.6036604Z http.https://github.com/.extraheader 2025-04-21T23:09:17.6082938Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-04-21T23:09:17.6127551Z http.https://github.com/.extraheader 2025-04-21T23:09:17.6176862Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-04-21T23:09:17.6227843Z http.https://github.com/.extraheader 2025-04-21T23:09:17.6282955Z Entering 'third_party/flatbuffers' 2025-04-21T23:09:17.6326654Z http.https://github.com/.extraheader 2025-04-21T23:09:17.6372724Z Entering 'third_party/fmt' 2025-04-21T23:09:17.6420934Z http.https://github.com/.extraheader 2025-04-21T23:09:17.6466842Z Entering 'third_party/gemmlowp/gemmlowp' 2025-04-21T23:09:17.6512078Z http.https://github.com/.extraheader 2025-04-21T23:09:17.6556604Z Entering 'third_party/gloo' 2025-04-21T23:09:17.6601532Z http.https://github.com/.extraheader 2025-04-21T23:09:17.6645946Z Entering 'third_party/googletest' 2025-04-21T23:09:17.6691163Z http.https://github.com/.extraheader 2025-04-21T23:09:17.6735600Z Entering 'third_party/ideep' 2025-04-21T23:09:17.6780486Z http.https://github.com/.extraheader 2025-04-21T23:09:17.6823172Z Entering 'third_party/ideep/mkl-dnn' 2025-04-21T23:09:17.6867089Z http.https://github.com/.extraheader 2025-04-21T23:09:17.6920622Z Entering 'third_party/ittapi' 2025-04-21T23:09:17.6964704Z http.https://github.com/.extraheader 2025-04-21T23:09:17.7008738Z Entering 'third_party/kineto' 2025-04-21T23:09:17.7052550Z http.https://github.com/.extraheader 2025-04-21T23:09:17.7095502Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-04-21T23:09:17.7138783Z http.https://github.com/.extraheader 2025-04-21T23:09:17.7183599Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-04-21T23:09:17.7227189Z http.https://github.com/.extraheader 2025-04-21T23:09:17.7272819Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-04-21T23:09:17.7316125Z http.https://github.com/.extraheader 2025-04-21T23:09:17.7360475Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-04-21T23:09:17.7405469Z http.https://github.com/.extraheader 2025-04-21T23:09:17.7449590Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-04-21T23:09:17.7493841Z http.https://github.com/.extraheader 2025-04-21T23:09:17.7537942Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-04-21T23:09:17.7583270Z http.https://github.com/.extraheader 2025-04-21T23:09:17.7632808Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-04-21T23:09:17.7677132Z http.https://github.com/.extraheader 2025-04-21T23:09:17.7721237Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-04-21T23:09:17.7764642Z http.https://github.com/.extraheader 2025-04-21T23:09:17.7811109Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-04-21T23:09:17.7852583Z http.https://github.com/.extraheader 2025-04-21T23:09:17.7898293Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-04-21T23:09:17.7941056Z http.https://github.com/.extraheader 2025-04-21T23:09:17.7989142Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-04-21T23:09:17.8032366Z http.https://github.com/.extraheader 2025-04-21T23:09:17.8075057Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-04-21T23:09:17.8118232Z http.https://github.com/.extraheader 2025-04-21T23:09:17.8164409Z Entering 'third_party/kleidiai' 2025-04-21T23:09:17.8208822Z http.https://github.com/.extraheader 2025-04-21T23:09:17.8252644Z Entering 'third_party/mimalloc' 2025-04-21T23:09:17.8297233Z http.https://github.com/.extraheader 2025-04-21T23:09:17.8341333Z Entering 'third_party/nlohmann' 2025-04-21T23:09:17.8386917Z http.https://github.com/.extraheader 2025-04-21T23:09:17.8431191Z Entering 'third_party/onnx' 2025-04-21T23:09:17.8474831Z http.https://github.com/.extraheader 2025-04-21T23:09:17.8533578Z Entering 'third_party/onnx/third_party/pybind11' 2025-04-21T23:09:17.8576830Z http.https://github.com/.extraheader 2025-04-21T23:09:17.8627842Z Entering 'third_party/opentelemetry-cpp' 2025-04-21T23:09:17.8670852Z http.https://github.com/.extraheader 2025-04-21T23:09:17.8714856Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-04-21T23:09:17.8755820Z http.https://github.com/.extraheader 2025-04-21T23:09:17.8799227Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-04-21T23:09:17.8840436Z http.https://github.com/.extraheader 2025-04-21T23:09:17.8881406Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-04-21T23:09:17.8923374Z http.https://github.com/.extraheader 2025-04-21T23:09:17.8965406Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-04-21T23:09:17.9011492Z http.https://github.com/.extraheader 2025-04-21T23:09:17.9055962Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-04-21T23:09:17.9100025Z http.https://github.com/.extraheader 2025-04-21T23:09:17.9143587Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-04-21T23:09:17.9186976Z http.https://github.com/.extraheader 2025-04-21T23:09:17.9230401Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-04-21T23:09:17.9273121Z http.https://github.com/.extraheader 2025-04-21T23:09:17.9315696Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-04-21T23:09:17.9358644Z http.https://github.com/.extraheader 2025-04-21T23:09:17.9407261Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-04-21T23:09:17.9449738Z http.https://github.com/.extraheader 2025-04-21T23:09:17.9496160Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-04-21T23:09:17.9543183Z http.https://github.com/.extraheader 2025-04-21T23:09:17.9608430Z Entering 'third_party/pocketfft' 2025-04-21T23:09:17.9651904Z http.https://github.com/.extraheader 2025-04-21T23:09:17.9695390Z Entering 'third_party/protobuf' 2025-04-21T23:09:17.9742011Z http.https://github.com/.extraheader 2025-04-21T23:09:17.9788462Z Entering 'third_party/protobuf/third_party/benchmark' 2025-04-21T23:09:17.9831857Z http.https://github.com/.extraheader 2025-04-21T23:09:17.9875266Z Entering 'third_party/protobuf/third_party/googletest' 2025-04-21T23:09:17.9918609Z http.https://github.com/.extraheader 2025-04-21T23:09:17.9965266Z Entering 'third_party/psimd' 2025-04-21T23:09:18.0010228Z http.https://github.com/.extraheader 2025-04-21T23:09:18.0054443Z Entering 'third_party/pthreadpool' 2025-04-21T23:09:18.0099641Z http.https://github.com/.extraheader 2025-04-21T23:09:18.0143696Z Entering 'third_party/pybind11' 2025-04-21T23:09:18.0188598Z http.https://github.com/.extraheader 2025-04-21T23:09:18.0232288Z Entering 'third_party/python-peachpy' 2025-04-21T23:09:18.0276901Z http.https://github.com/.extraheader 2025-04-21T23:09:18.0320172Z Entering 'third_party/sleef' 2025-04-21T23:09:18.0364317Z http.https://github.com/.extraheader 2025-04-21T23:09:18.0410569Z Entering 'third_party/tensorpipe' 2025-04-21T23:09:18.0453953Z http.https://github.com/.extraheader 2025-04-21T23:09:18.0496983Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-04-21T23:09:18.0539522Z http.https://github.com/.extraheader 2025-04-21T23:09:18.0583259Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-04-21T23:09:18.0626763Z http.https://github.com/.extraheader 2025-04-21T23:09:18.0668750Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-04-21T23:09:18.0711668Z http.https://github.com/.extraheader 2025-04-21T23:09:18.0754040Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-04-21T23:09:18.0800093Z http.https://github.com/.extraheader 2025-04-21T23:09:18.0840716Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-04-21T23:09:18.0884206Z http.https://github.com/.extraheader 2025-04-21T23:09:18.1047729Z A job completed hook has been configured by the self-hosted runner administrator 2025-04-21T23:09:18.1080263Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2025-04-21T23:09:18.1088246Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-04-21T23:09:18.1088612Z ##[endgroup] 2025-04-21T23:09:18.1185425Z [!ALERT!] Swap in detected! [!ALERT!] 2025-04-21T23:09:28.8888294Z [!ALERT!] Swap out detected [!ALERT!] 2025-04-21T23:09:46.1263103Z Cleaning up orphan processes