2026-02-21T14:01:07.9343531Z Current runner version: '2.331.0'
2026-02-21T14:01:07.9350392Z Runner name: 'i-0b3b13013e324b5e0'
2026-02-21T14:01:07.9351275Z Runner group name: 'default'
2026-02-21T14:01:07.9352360Z Machine name: 'ip-10-0-46-218'
2026-02-21T14:01:07.9355195Z ##[group]GITHUB_TOKEN Permissions
2026-02-21T14:01:07.9357924Z Contents: read
2026-02-21T14:01:07.9358612Z Metadata: read
2026-02-21T14:01:07.9359123Z ##[endgroup]
2026-02-21T14:01:07.9361338Z Secret source: Actions
2026-02-21T14:01:07.9362100Z Prepare workflow directory
2026-02-21T14:01:07.9702399Z Prepare all required actions
2026-02-21T14:01:07.9745062Z Getting action download info
2026-02-21T14:01:08.1769424Z Download action repository 'actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683' (SHA:11bd71901bbe5b1630ceea73d27597364c9af683)
2026-02-21T14:01:08.4750271Z Download action repository 'actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e' (SHA:65a9edc5881444af0b9093a5e628f2fe47ea3b2e)
2026-02-21T14:01:09.0220136Z Uses: pytorch/test-infra/.github/workflows/upload_benchmark_results.yml@refs/heads/main (bb8f04ff3961233c844fde6533c7c6c5f0857909)
2026-02-21T14:01:09.0226065Z ##[group] Inputs
2026-02-21T14:01:09.0227446Z   benchmark-artifact: benchmark-results-mi325x-int4_gemm,flash_attention
2026-02-21T14:01:09.0232799Z   benchmark-metadata: {"timestamp": 1771682437, "schema_version": "v3", "name": "run-mi325x (int4_gemm,flash_attention) / benchmark-rocm6.4-int4_gemm,flash_attention-py3.12-mi325x", "repo": "pytorch/helion", "head_branch": "refs/heads/main", "head_sha": "874a7d0cadab18218a84ad3579d329dc95c51820", "workflow_id": 22253280836, "run_attempt": 1, "job_id": 64380329937}
2026-02-21T14:01:09.0239246Z   runners-info: [{"cpu_info": "x86_64", "cpu_count": 256, "avail_mem_in_gb": 3019, "extra_info": {"hostname": "9ec7733a29ba"}, "name": "rocm", "type": "AMD Instinct MI325X", "gpu_count": 8, "avail_gpu_mem_in_gb": 2047}]
2026-02-21T14:01:09.0242041Z   dependencies: {}
2026-02-21T14:01:09.0242750Z   schema-version: v3
2026-02-21T14:01:09.0243890Z   dry-run: false
2026-02-21T14:01:09.0244707Z   if-no-files-found: ignore
2026-02-21T14:01:09.0245496Z   run_h100: true
2026-02-21T14:01:09.0246149Z   run_b200: true
2026-02-21T14:01:09.0246834Z   run_mi325x: true
2026-02-21T14:01:09.0248916Z   kernels: softmax,jsd,welford,kl_div,int4_gemm,layer_norm,layer_norm-bwd,rms_norm,rms_norm-bwd,cross_entropy,flash_attention,gemm,grouped_gemm
2026-02-21T14:01:09.0251099Z ##[endgroup]
2026-02-21T14:01:09.0253129Z Complete job name: run-mi325x (int4_gemm,flash_attention) / upload-benchmark-results / Uploading benchmark-results-mi325x-int4_gemm,flash_attention
2026-02-21T14:01:09.0942263Z A job started hook has been configured by the self-hosted runner administrator
2026-02-21T14:01:09.1066881Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh'
2026-02-21T14:01:09.1077728Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2026-02-21T14:01:09.1079047Z ##[endgroup]
2026-02-21T14:01:10.0978675Z Runner Type: linux.2xlarge
2026-02-21T14:01:10.0979699Z Instance Type: c5.2xlarge
2026-02-21T14:01:10.0980517Z AMI Name: unknown
2026-02-21T14:01:10.1006008Z AMI ID: ami-068c0051b15cdb816
2026-02-21T14:01:15.7997101Z ##[group]Run actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
2026-02-21T14:01:15.7998077Z with:
2026-02-21T14:01:15.7998590Z   repository: pytorch/test-infra
2026-02-21T14:01:15.7999152Z   ref: main
2026-02-21T14:01:15.7999607Z   path: test-infra
2026-02-21T14:01:15.8000445Z   token: ***
2026-02-21T14:01:15.8000944Z   ssh-strict: true
2026-02-21T14:01:15.8001433Z   ssh-user: git
2026-02-21T14:01:15.8001950Z   persist-credentials: true
2026-02-21T14:01:15.8002531Z   clean: true
2026-02-21T14:01:15.8003056Z   sparse-checkout-cone-mode: true
2026-02-21T14:01:15.8003686Z   fetch-depth: 1
2026-02-21T14:01:15.8004170Z   fetch-tags: false
2026-02-21T14:01:15.8004690Z   show-progress: true
2026-02-21T14:01:15.8005209Z   lfs: false
2026-02-21T14:01:15.8005708Z   submodules: false
2026-02-21T14:01:15.8006230Z   set-safe-directory: true
2026-02-21T14:01:15.8007104Z ##[endgroup]
2026-02-21T14:01:15.9184249Z Syncing repository: pytorch/test-infra
2026-02-21T14:01:15.9186636Z ##[group]Getting Git version info
2026-02-21T14:01:15.9187644Z Working directory is '/home/ec2-user/actions-runner/_work/helion/helion/test-infra'
2026-02-21T14:01:15.9189053Z [command]/usr/bin/git version
2026-02-21T14:01:15.9189633Z git version 2.50.1
2026-02-21T14:01:15.9191548Z ##[endgroup]
2026-02-21T14:01:15.9197510Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/2ce496b0-195c-4f2d-bf35-07bff660b5a5' before making global git config changes
2026-02-21T14:01:15.9199398Z Adding repository directory to the temporary git global config as a safe directory
2026-02-21T14:01:15.9201006Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/helion/helion/test-infra
2026-02-21T14:01:15.9217006Z ##[group]Initializing the repository
2026-02-21T14:01:15.9221370Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/helion/helion/test-infra
2026-02-21T14:01:15.9250284Z hint: Using 'master' as the name for the initial branch. This default branch name
2026-02-21T14:01:15.9251545Z hint: is subject to change. To configure the initial branch name to use in all
2026-02-21T14:01:15.9252700Z hint: of your new repositories, which will suppress this warning, call:
2026-02-21T14:01:15.9253562Z hint:
2026-02-21T14:01:15.9254168Z hint: 	git config --global init.defaultBranch <name>
2026-02-21T14:01:15.9254859Z hint:
2026-02-21T14:01:15.9255552Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
2026-02-21T14:01:15.9256704Z hint: 'development'. The just-created branch can be renamed via this command:
2026-02-21T14:01:15.9257596Z hint:
2026-02-21T14:01:15.9258074Z hint: 	git branch -m <name>
2026-02-21T14:01:15.9258589Z hint:
2026-02-21T14:01:15.9259346Z hint: Disable this message with "git config set advice.defaultBranchName false"
2026-02-21T14:01:15.9260773Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/helion/helion/test-infra/.git/
2026-02-21T14:01:15.9263059Z [command]/usr/bin/git remote add origin https://github.com/pytorch/test-infra
2026-02-21T14:01:15.9284024Z ##[endgroup]
2026-02-21T14:01:15.9284930Z ##[group]Disabling automatic garbage collection
2026-02-21T14:01:15.9287874Z [command]/usr/bin/git config --local gc.auto 0
2026-02-21T14:01:15.9310342Z ##[endgroup]
2026-02-21T14:01:15.9311142Z ##[group]Setting up auth
2026-02-21T14:01:15.9315486Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2026-02-21T14:01:15.9338778Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
2026-02-21T14:01:15.9606282Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2026-02-21T14:01:15.9629989Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
2026-02-21T14:01:15.9883087Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic ***
2026-02-21T14:01:15.9926547Z ##[endgroup]
2026-02-21T14:01:15.9927182Z ##[group]Fetching the repository
2026-02-21T14:01:15.9937914Z [command]/usr/bin/git -c protocol.version=2 fetch --no-tags --prune --no-recurse-submodules --depth=1 origin +refs/heads/main*:refs/remotes/origin/main* +refs/tags/main*:refs/tags/main*
2026-02-21T14:01:16.8511321Z From https://github.com/pytorch/test-infra
2026-02-21T14:01:16.8511961Z  * [new branch]      main       -> origin/main
2026-02-21T14:01:16.8532934Z ##[endgroup]
2026-02-21T14:01:16.8533638Z ##[group]Determining the checkout info
2026-02-21T14:01:16.8540551Z [command]/usr/bin/git branch --list --remote origin/main
2026-02-21T14:01:16.8570375Z   origin/main
2026-02-21T14:01:16.8578695Z ##[endgroup]
2026-02-21T14:01:16.8579455Z [command]/usr/bin/git sparse-checkout disable
2026-02-21T14:01:16.8611426Z [command]/usr/bin/git config --local --unset-all extensions.worktreeConfig
2026-02-21T14:01:16.8633288Z ##[group]Checking out the ref
2026-02-21T14:01:16.8637163Z [command]/usr/bin/git checkout --progress --force -B main refs/remotes/origin/main
2026-02-21T14:01:17.0020403Z Switched to a new branch 'main'
2026-02-21T14:01:17.0020993Z branch 'main' set up to track 'origin/main'.
2026-02-21T14:01:17.0061256Z ##[endgroup]
2026-02-21T14:01:17.0065125Z [command]/usr/bin/git log -1 --format=%H
2026-02-21T14:01:17.0085241Z bb8f04ff3961233c844fde6533c7c6c5f0857909
2026-02-21T14:01:17.0295967Z ##[group]Run actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e
2026-02-21T14:01:17.0296804Z with:
2026-02-21T14:01:17.0297312Z   name: benchmark-results-mi325x-int4_gemm,flash_attention
2026-02-21T14:01:17.0298186Z   path: /home/ec2-user/actions-runner/_work/_temp/benchmark-artifact
2026-02-21T14:01:17.0298943Z   merge-multiple: false
2026-02-21T14:01:17.0299417Z   repository: pytorch/helion
2026-02-21T14:01:17.0299887Z   run-id: 22253280836
2026-02-21T14:01:17.0300286Z ##[endgroup]
2026-02-21T14:01:17.2546761Z Downloading single artifact
2026-02-21T14:01:17.3655368Z Preparing to download the following artifacts:
2026-02-21T14:01:17.3656407Z - benchmark-results-mi325x-int4_gemm,flash_attention (ID: 5601585868, Size: 586)
2026-02-21T14:01:17.4328889Z Redirecting to blob download url: https://productionresultssa0.blob.core.windows.net/actions-results/cff08df3-00e7-4767-8b8d-229a961df982/workflow-job-run-c78291dd-97f4-59c9-9f4f-6d6486286a88/artifacts/2b99cdba311ad0e8112967ba062d0028047025d991efaf2ff9f18e5375904b48.zip
2026-02-21T14:01:17.4598157Z (node:40709) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
2026-02-21T14:01:17.4599544Z (Use `node --trace-deprecation ...` to show where the warning was created)
2026-02-21T14:01:17.4600422Z Starting download of artifact to: /home/ec2-user/actions-runner/_work/_temp/benchmark-artifact
2026-02-21T14:01:17.4633181Z Artifact download completed successfully.
2026-02-21T14:01:17.4633546Z Total of 1 artifact(s) downloaded
2026-02-21T14:01:17.4638879Z Download artifact has finished successfully
2026-02-21T14:01:17.4755392Z ##[group]Run set -eux
2026-02-21T14:01:17.4755725Z set -eux
2026-02-21T14:01:17.4755954Z 
2026-02-21T14:01:17.4756282Z if [[ ! -d "${BENCHMARK_RESULTS_DIR}" ]]; then
2026-02-21T14:01:17.4756778Z   echo "${BENCHMARK_RESULTS_DIR} does not exist, skipping"
2026-02-21T14:01:17.4757266Z   # We don't want the job to fail if the directory doesn't exist
2026-02-21T14:01:17.4757690Z   exit 0
2026-02-21T14:01:17.4757920Z fi
2026-02-21T14:01:17.4758146Z 
2026-02-21T14:01:17.4758384Z python3 -mpip install boto3==1.35.33
2026-02-21T14:01:17.4758686Z 
2026-02-21T14:01:17.4758931Z if [[ "${DRY_RUN}" == "true" ]]; then
2026-02-21T14:01:17.4759338Z   python3 ".github/scripts/upload_benchmark_results.py" \
2026-02-21T14:01:17.4759838Z     --benchmark-results-dir "${BENCHMARK_RESULTS_DIR}" \
2026-02-21T14:01:17.4760318Z     --metadata "${BENCHMARK_METADATA}" \
2026-02-21T14:01:17.4760662Z     --runners "${RUNNERS_INFO}" \
2026-02-21T14:01:17.4761004Z     --dependencies "${DEPENDENCIES}" \
2026-02-21T14:01:17.4761317Z     --dry-run
2026-02-21T14:01:17.4761565Z else
2026-02-21T14:01:17.4761882Z   python3 ".github/scripts/upload_benchmark_results.py" \
2026-02-21T14:01:17.4762494Z     --benchmark-results-dir "${BENCHMARK_RESULTS_DIR}" \
2026-02-21T14:01:17.4762926Z     --metadata "${BENCHMARK_METADATA}" \
2026-02-21T14:01:17.4763269Z     --runners "${RUNNERS_INFO}" \
2026-02-21T14:01:17.4763610Z     --dependencies "${DEPENDENCIES}"
2026-02-21T14:01:17.4764067Z fi
2026-02-21T14:01:17.4774153Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2026-02-21T14:01:17.4774526Z env:
2026-02-21T14:01:17.4774923Z   BENCHMARK_RESULTS_DIR: /home/ec2-user/actions-runner/_work/_temp/benchmark-artifact
2026-02-21T14:01:17.4775395Z   DRY_RUN: false
2026-02-21T14:01:17.4775639Z   IF_NO_FILES_FOUND: ignore
2026-02-21T14:01:17.4777148Z   BENCHMARK_METADATA: {"timestamp": 1771682437, "schema_version": "v3", "name": "run-mi325x (int4_gemm,flash_attention) / benchmark-rocm6.4-int4_gemm,flash_attention-py3.12-mi325x", "repo": "pytorch/helion", "head_branch": "refs/heads/main", "head_sha": "874a7d0cadab18218a84ad3579d329dc95c51820", "workflow_id": 22253280836, "run_attempt": 1, "job_id": 64380329937}
2026-02-21T14:01:17.4779298Z   RUNNERS_INFO: [{"cpu_info": "x86_64", "cpu_count": 256, "avail_mem_in_gb": 3019, "extra_info": {"hostname": "9ec7733a29ba"}, "name": "rocm", "type": "AMD Instinct MI325X", "gpu_count": 8, "avail_gpu_mem_in_gb": 2047}]
2026-02-21T14:01:17.4780203Z   DEPENDENCIES: {}
2026-02-21T14:01:17.4780431Z ##[endgroup]
2026-02-21T14:01:17.4812811Z + [[ ! -d /home/ec2-user/actions-runner/_work/_temp/benchmark-artifact ]]
2026-02-21T14:01:17.4813304Z + python3 -mpip install boto3==1.35.33
2026-02-21T14:01:17.7682846Z Defaulting to user installation because normal site-packages is not writeable
2026-02-21T14:01:18.8701745Z Collecting boto3==1.35.33
2026-02-21T14:01:18.8871740Z   Downloading boto3-1.35.33-py3-none-any.whl (139 kB)
2026-02-21T14:01:20.2240316Z Collecting botocore<1.36.0,>=1.35.33
2026-02-21T14:01:20.2281716Z   Downloading botocore-1.35.99-py3-none-any.whl (13.3 MB)
2026-02-21T14:01:20.3755309Z Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /usr/lib/python3.9/site-packages (from boto3==1.35.33) (0.10.0)
2026-02-21T14:01:20.4218420Z Collecting s3transfer<0.11.0,>=0.10.0
2026-02-21T14:01:20.4269467Z   Downloading s3transfer-0.10.4-py3-none-any.whl (83 kB)
2026-02-21T14:01:20.4407071Z Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /usr/lib/python3.9/site-packages (from botocore<1.36.0,>=1.35.33->boto3==1.35.33) (2.8.1)
2026-02-21T14:01:20.4430557Z Requirement already satisfied: urllib3<1.27,>=1.25.4 in /usr/lib/python3.9/site-packages (from botocore<1.36.0,>=1.35.33->boto3==1.35.33) (1.25.10)
2026-02-21T14:01:20.7106005Z Requirement already satisfied: six>=1.5 in /usr/lib/python3.9/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.36.0,>=1.35.33->boto3==1.35.33) (1.15.0)
2026-02-21T14:01:20.7926936Z Installing collected packages: botocore, s3transfer, boto3
2026-02-21T14:01:21.3435129Z Successfully installed boto3-1.35.33 botocore-1.35.99 s3transfer-0.10.4
2026-02-21T14:01:21.5512143Z + [[ false == \t\r\u\e ]]
2026-02-21T14:01:21.5515186Z + python3 .github/scripts/upload_benchmark_results.py --benchmark-results-dir /home/ec2-user/actions-runner/_work/_temp/benchmark-artifact --metadata '{"timestamp": 1771682437, "schema_version": "v3", "name": "run-mi325x (int4_gemm,flash_attention) / benchmark-rocm6.4-int4_gemm,flash_attention-py3.12-mi325x", "repo": "pytorch/helion", "head_branch": "refs/heads/main", "head_sha": "874a7d0cadab18218a84ad3579d329dc95c51820", "workflow_id": 22253280836, "run_attempt": 1, "job_id": 64380329937}' --runners '[{"cpu_info": "x86_64", "cpu_count": 256, "avail_mem_in_gb": 3019, "extra_info": {"hostname": "9ec7733a29ba"}, "name": "rocm", "type": "AMD Instinct MI325X", "gpu_count": 8, "avail_gpu_mem_in_gb": 2047}]' --dependencies '{}'
2026-02-21T14:01:21.6880907Z INFO:root:Upload /home/ec2-user/actions-runner/_work/_temp/benchmark-artifact/helionbench.json to s3://ossci-benchmarks/v3/pytorch/helion/22253280836/64380329937/helionbench.json
2026-02-21T14:01:21.7117409Z INFO:botocore.credentials:Found credentials from IAM Role: gh-ci-github-action-runners-runner-role
2026-02-21T14:01:21.9301779Z Post job cleanup.
2026-02-21T14:01:22.0255272Z [command]/usr/bin/git version
2026-02-21T14:01:22.0292646Z git version 2.50.1
2026-02-21T14:01:22.0336110Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/1e6844b7-428d-4932-a9a1-0a335dcd3872' before making global git config changes
2026-02-21T14:01:22.0337304Z Adding repository directory to the temporary git global config as a safe directory
2026-02-21T14:01:22.0340947Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/helion/helion/test-infra
2026-02-21T14:01:22.0372324Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2026-02-21T14:01:22.0408557Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
2026-02-21T14:01:22.0667885Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2026-02-21T14:01:22.0685411Z http.https://github.com/.extraheader
2026-02-21T14:01:22.0696200Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader
2026-02-21T14:01:22.0723451Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
2026-02-21T14:01:22.1030357Z A job completed hook has been configured by the self-hosted runner administrator
2026-02-21T14:01:22.1057566Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh'
2026-02-21T14:01:22.1063103Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2026-02-21T14:01:22.1063484Z ##[endgroup]
2026-02-21T14:01:22.1141431Z [!ALERT!] Swap in detected! [!ALERT!]
2026-02-21T14:01:33.4861757Z [!ALERT!] Swap out detected [!ALERT!]
2026-02-21T14:01:51.9229761Z Cleaning up orphan processes