2026-02-21T09:56:54.9340757Z Current runner version: '2.331.0'
2026-02-21T09:56:54.9347290Z Runner name: 'i-0e4bf6267de54fa0b'
2026-02-21T09:56:54.9348035Z Runner group name: 'default'
2026-02-21T09:56:54.9348975Z Machine name: 'ip-10-0-36-173'
2026-02-21T09:56:54.9351733Z ##[group]GITHUB_TOKEN Permissions
2026-02-21T09:56:54.9354197Z Contents: read
2026-02-21T09:56:54.9354761Z Metadata: read
2026-02-21T09:56:54.9355384Z ##[endgroup]
2026-02-21T09:56:54.9357538Z Secret source: Actions
2026-02-21T09:56:54.9358337Z Prepare workflow directory
2026-02-21T09:56:54.9705142Z Prepare all required actions
2026-02-21T09:56:54.9747834Z Getting action download info
2026-02-21T09:56:55.1576102Z Download action repository 'actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683' (SHA:11bd71901bbe5b1630ceea73d27597364c9af683)
2026-02-21T09:56:55.4286096Z Download action repository 'actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e' (SHA:65a9edc5881444af0b9093a5e628f2fe47ea3b2e)
2026-02-21T09:56:55.8602731Z Uses: pytorch/test-infra/.github/workflows/upload_benchmark_results.yml@refs/heads/main (bb8f04ff3961233c844fde6533c7c6c5f0857909)
2026-02-21T09:56:55.8608160Z ##[group] Inputs
2026-02-21T09:56:55.8608994Z   benchmark-artifact: benchmark-results-b200-rms_norm-bwd
2026-02-21T09:56:55.8611972Z   benchmark-metadata: {"timestamp": 1771667800, "schema_version": "v3", "name": "run-b200 (rms_norm-bwd) / benchmark-cu130-rms_norm-bwd-py3.12-b200", "repo": "pytorch/helion", "head_branch": "refs/heads/main", "head_sha": "874a7d0cadab18218a84ad3579d329dc95c51820", "workflow_id": 22253280836, "run_attempt": 1, "job_id": 64380329763}
2026-02-21T09:56:55.8615649Z   runners-info: [{"cpu_info": "x86_64", "cpu_count": 224, "avail_mem_in_gb": 2015, "extra_info": {"hostname": "22f9d4a00982"}, "name": "cuda", "type": "NVIDIA B200", "gpu_count": 1, "avail_gpu_mem_in_gb": 178}]
2026-02-21T09:56:55.8617293Z   dependencies: {}
2026-02-21T09:56:55.8617896Z   schema-version: v3
2026-02-21T09:56:55.8619024Z   dry-run: false
2026-02-21T09:56:55.8619623Z   if-no-files-found: ignore
2026-02-21T09:56:55.8620292Z   run_h100: true
2026-02-21T09:56:55.8620835Z   run_b200: true
2026-02-21T09:56:55.8621427Z   run_mi325x: true
2026-02-21T09:56:55.8622689Z   kernels: softmax,jsd,welford,kl_div,int4_gemm,layer_norm,layer_norm-bwd,rms_norm,rms_norm-bwd,cross_entropy,flash_attention,gemm,grouped_gemm
2026-02-21T09:56:55.8624161Z ##[endgroup]
2026-02-21T09:56:55.8625340Z Complete job name: run-b200 (rms_norm-bwd) / upload-benchmark-results / Uploading benchmark-results-b200-rms_norm-bwd
2026-02-21T09:56:55.9289559Z A job started hook has been configured by the self-hosted runner administrator
2026-02-21T09:56:55.9412989Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh'
2026-02-21T09:56:55.9424286Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2026-02-21T09:56:55.9425350Z ##[endgroup]
2026-02-21T09:56:57.1681300Z Runner Type: linux.2xlarge
2026-02-21T09:56:57.1682428Z Instance Type: c5.2xlarge
2026-02-21T09:56:57.1683344Z AMI Name: unknown
2026-02-21T09:56:57.1712562Z AMI ID: ami-068c0051b15cdb816
2026-02-21T09:57:02.8616671Z ##[group]Run actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
2026-02-21T09:57:02.8617225Z with:
2026-02-21T09:57:02.8617529Z   repository: pytorch/test-infra
2026-02-21T09:57:02.8617869Z   ref: main
2026-02-21T09:57:02.8618141Z   path: test-infra
2026-02-21T09:57:02.8618620Z   token: ***
2026-02-21T09:57:02.8618912Z   ssh-strict: true
2026-02-21T09:57:02.8619225Z   ssh-user: git
2026-02-21T09:57:02.8619506Z   persist-credentials: true
2026-02-21T09:57:02.8619830Z   clean: true
2026-02-21T09:57:02.8620133Z   sparse-checkout-cone-mode: true
2026-02-21T09:57:02.8620491Z   fetch-depth: 1
2026-02-21T09:57:02.8620759Z   fetch-tags: false
2026-02-21T09:57:02.8621052Z   show-progress: true
2026-02-21T09:57:02.8621332Z   lfs: false
2026-02-21T09:57:02.8621605Z   submodules: false
2026-02-21T09:57:02.8621888Z   set-safe-directory: true
2026-02-21T09:57:02.8622206Z ##[endgroup]
2026-02-21T09:57:03.0061383Z Syncing repository: pytorch/test-infra
2026-02-21T09:57:03.0063667Z ##[group]Getting Git version info
2026-02-21T09:57:03.0064297Z Working directory is '/home/ec2-user/actions-runner/_work/helion/helion/test-infra'
2026-02-21T09:57:03.0065144Z [command]/usr/bin/git version
2026-02-21T09:57:03.0065473Z git version 2.50.1
2026-02-21T09:57:03.0084412Z ##[endgroup]
2026-02-21T09:57:03.0096258Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/07b9364f-5efc-4cda-b149-d67e0a1c4ad6' before making global git config changes
2026-02-21T09:57:03.0097384Z Adding repository directory to the temporary git global config as a safe directory
2026-02-21T09:57:03.0101022Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/helion/helion/test-infra
2026-02-21T09:57:03.0130750Z ##[group]Initializing the repository
2026-02-21T09:57:03.0134910Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/helion/helion/test-infra
2026-02-21T09:57:03.0165118Z hint: Using 'master' as the name for the initial branch. This default branch name
2026-02-21T09:57:03.0166030Z hint: is subject to change. To configure the initial branch name to use in all
2026-02-21T09:57:03.0166793Z hint: of your new repositories, which will suppress this warning, call:
2026-02-21T09:57:03.0167273Z hint:
2026-02-21T09:57:03.0167648Z hint: 	git config --global init.defaultBranch <name>
2026-02-21T09:57:03.0168077Z hint:
2026-02-21T09:57:03.0168492Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
2026-02-21T09:57:03.0169153Z hint: 'development'. The just-created branch can be renamed via this command:
2026-02-21T09:57:03.0169744Z hint:
2026-02-21T09:57:03.0170036Z hint: 	git branch -m <name>
2026-02-21T09:57:03.0170463Z hint:
2026-02-21T09:57:03.0170940Z hint: Disable this message with "git config set advice.defaultBranchName false"
2026-02-21T09:57:03.0171774Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/helion/helion/test-infra/.git/
2026-02-21T09:57:03.0175000Z [command]/usr/bin/git remote add origin https://github.com/pytorch/test-infra
2026-02-21T09:57:03.0198916Z ##[endgroup]
2026-02-21T09:57:03.0199462Z ##[group]Disabling automatic garbage collection
2026-02-21T09:57:03.0202673Z [command]/usr/bin/git config --local gc.auto 0
2026-02-21T09:57:03.0225573Z ##[endgroup]
2026-02-21T09:57:03.0226048Z ##[group]Setting up auth
2026-02-21T09:57:03.0231081Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2026-02-21T09:57:03.0259209Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
2026-02-21T09:57:03.0541298Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2026-02-21T09:57:03.0564868Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
2026-02-21T09:57:03.1008462Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic ***
2026-02-21T09:57:03.1055662Z ##[endgroup]
2026-02-21T09:57:03.1056180Z ##[group]Fetching the repository
2026-02-21T09:57:03.1063400Z [command]/usr/bin/git -c protocol.version=2 fetch --no-tags --prune --no-recurse-submodules --depth=1 origin +refs/heads/main*:refs/remotes/origin/main* +refs/tags/main*:refs/tags/main*
2026-02-21T09:57:03.4333885Z From https://github.com/pytorch/test-infra
2026-02-21T09:57:03.4334599Z  * [new branch]      main       -> origin/main
2026-02-21T09:57:03.4373410Z ##[endgroup]
2026-02-21T09:57:03.4373839Z ##[group]Determining the checkout info
2026-02-21T09:57:03.4380162Z [command]/usr/bin/git branch --list --remote origin/main
2026-02-21T09:57:03.4427704Z   origin/main
2026-02-21T09:57:03.4432771Z ##[endgroup]
2026-02-21T09:57:03.4436874Z [command]/usr/bin/git sparse-checkout disable
2026-02-21T09:57:03.4479659Z [command]/usr/bin/git config --local --unset-all extensions.worktreeConfig
2026-02-21T09:57:03.4511242Z ##[group]Checking out the ref
2026-02-21T09:57:03.4514450Z [command]/usr/bin/git checkout --progress --force -B main refs/remotes/origin/main
2026-02-21T09:57:03.5959547Z Switched to a new branch 'main'
2026-02-21T09:57:03.5960483Z branch 'main' set up to track 'origin/main'.
2026-02-21T09:57:03.5969422Z ##[endgroup]
2026-02-21T09:57:03.6009315Z [command]/usr/bin/git log -1 --format=%H
2026-02-21T09:57:03.6030710Z bb8f04ff3961233c844fde6533c7c6c5f0857909
2026-02-21T09:57:03.6193632Z ##[group]Run actions/download-artifact@65a9edc5881444af0b9093a5e628f2fe47ea3b2e
2026-02-21T09:57:03.6194139Z with:
2026-02-21T09:57:03.6194392Z   name: benchmark-results-b200-rms_norm-bwd
2026-02-21T09:57:03.6194836Z   path: /home/ec2-user/actions-runner/_work/_temp/benchmark-artifact
2026-02-21T09:57:03.6195276Z   merge-multiple: false
2026-02-21T09:57:03.6195539Z   repository: pytorch/helion
2026-02-21T09:57:03.6195813Z   run-id: 22253280836
2026-02-21T09:57:03.6196058Z ##[endgroup]
2026-02-21T09:57:03.8457208Z Downloading single artifact
2026-02-21T09:57:03.9714947Z Preparing to download the following artifacts:
2026-02-21T09:57:03.9715534Z - benchmark-results-b200-rms_norm-bwd (ID: 5600748258, Size: 591)
2026-02-21T09:57:04.0218651Z Redirecting to blob download url: https://productionresultssa0.blob.core.windows.net/actions-results/cff08df3-00e7-4767-8b8d-229a961df982/workflow-job-run-ed9ac52f-29d7-5fe0-bb70-d12cbc1ecf23/artifacts/ba6717b2ec95c75a3e8bb09dda672bed2030160705223c10660de2f2f983b0f8.zip
2026-02-21T09:57:04.0220407Z Starting download of artifact to: /home/ec2-user/actions-runner/_work/_temp/benchmark-artifact
2026-02-21T09:57:04.0477382Z (node:41458) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
2026-02-21T09:57:04.0478535Z (Use `node --trace-deprecation ...` to show where the warning was created)
2026-02-21T09:57:04.0511714Z Artifact download completed successfully.
2026-02-21T09:57:04.0512124Z Total of 1 artifact(s) downloaded
2026-02-21T09:57:04.0517680Z Download artifact has finished successfully
2026-02-21T09:57:04.0621690Z ##[group]Run set -eux
2026-02-21T09:57:04.0622043Z set -eux
2026-02-21T09:57:04.0622265Z 
2026-02-21T09:57:04.0622555Z if [[ ! -d "${BENCHMARK_RESULTS_DIR}" ]]; then
2026-02-21T09:57:04.0623037Z   echo "${BENCHMARK_RESULTS_DIR} does not exist, skipping"
2026-02-21T09:57:04.0623570Z   # We don't want the job to fail if the directory doesn't exist
2026-02-21T09:57:04.0624086Z   exit 0
2026-02-21T09:57:04.0624361Z fi
2026-02-21T09:57:04.0624602Z 
2026-02-21T09:57:04.0624845Z python3 -mpip install boto3==1.35.33
2026-02-21T09:57:04.0625202Z 
2026-02-21T09:57:04.0625435Z if [[ "${DRY_RUN}" == "true" ]]; then
2026-02-21T09:57:04.0625907Z   python3 ".github/scripts/upload_benchmark_results.py" \
2026-02-21T09:57:04.0626438Z     --benchmark-results-dir "${BENCHMARK_RESULTS_DIR}" \
2026-02-21T09:57:04.0626952Z     --metadata "${BENCHMARK_METADATA}" \
2026-02-21T09:57:04.0627318Z     --runners "${RUNNERS_INFO}" \
2026-02-21T09:57:04.0627684Z     --dependencies "${DEPENDENCIES}" \
2026-02-21T09:57:04.0628021Z     --dry-run
2026-02-21T09:57:04.0628277Z else
2026-02-21T09:57:04.0628613Z   python3 ".github/scripts/upload_benchmark_results.py" \
2026-02-21T09:57:04.0629122Z     --benchmark-results-dir "${BENCHMARK_RESULTS_DIR}" \
2026-02-21T09:57:04.0629580Z     --metadata "${BENCHMARK_METADATA}" \
2026-02-21T09:57:04.0629945Z     --runners "${RUNNERS_INFO}" \
2026-02-21T09:57:04.0630305Z     --dependencies "${DEPENDENCIES}"
2026-02-21T09:57:04.0630623Z fi
2026-02-21T09:57:04.0640048Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2026-02-21T09:57:04.0640595Z env:
2026-02-21T09:57:04.0641004Z   BENCHMARK_RESULTS_DIR: /home/ec2-user/actions-runner/_work/_temp/benchmark-artifact
2026-02-21T09:57:04.0641511Z   DRY_RUN: false
2026-02-21T09:57:04.0641764Z   IF_NO_FILES_FOUND: ignore
2026-02-21T09:57:04.0643236Z   BENCHMARK_METADATA: {"timestamp": 1771667800, "schema_version": "v3", "name": "run-b200 (rms_norm-bwd) / benchmark-cu130-rms_norm-bwd-py3.12-b200", "repo": "pytorch/helion", "head_branch": "refs/heads/main", "head_sha": "874a7d0cadab18218a84ad3579d329dc95c51820", "workflow_id": 22253280836, "run_attempt": 1, "job_id": 64380329763}
2026-02-21T09:57:04.0645399Z   RUNNERS_INFO: [{"cpu_info": "x86_64", "cpu_count": 224, "avail_mem_in_gb": 2015, "extra_info": {"hostname": "22f9d4a00982"}, "name": "cuda", "type": "NVIDIA B200", "gpu_count": 1, "avail_gpu_mem_in_gb": 178}]
2026-02-21T09:57:04.0646415Z   DEPENDENCIES: {}
2026-02-21T09:57:04.0646697Z ##[endgroup]
2026-02-21T09:57:04.0676567Z + [[ ! -d /home/ec2-user/actions-runner/_work/_temp/benchmark-artifact ]]
2026-02-21T09:57:04.0677237Z + python3 -mpip install boto3==1.35.33
2026-02-21T09:57:04.3523294Z Defaulting to user installation because normal site-packages is not writeable
2026-02-21T09:57:05.4492696Z Collecting boto3==1.35.33
2026-02-21T09:57:05.4653901Z   Downloading boto3-1.35.33-py3-none-any.whl (139 kB)
2026-02-21T09:57:06.7496974Z Collecting botocore<1.36.0,>=1.35.33
2026-02-21T09:57:06.7533371Z   Downloading botocore-1.35.99-py3-none-any.whl (13.3 MB)
2026-02-21T09:57:06.8973870Z Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /usr/lib/python3.9/site-packages (from boto3==1.35.33) (0.10.0)
2026-02-21T09:57:06.9372114Z Collecting s3transfer<0.11.0,>=0.10.0
2026-02-21T09:57:06.9406257Z   Downloading s3transfer-0.10.4-py3-none-any.whl (83 kB)
2026-02-21T09:57:06.9519076Z Requirement already satisfied: urllib3<1.27,>=1.25.4 in /usr/lib/python3.9/site-packages (from botocore<1.36.0,>=1.35.33->boto3==1.35.33) (1.25.10)
2026-02-21T09:57:06.9523010Z Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /usr/lib/python3.9/site-packages (from botocore<1.36.0,>=1.35.33->boto3==1.35.33) (2.8.1)
2026-02-21T09:57:07.1035363Z Requirement already satisfied: six>=1.5 in /usr/lib/python3.9/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.36.0,>=1.35.33->boto3==1.35.33) (1.15.0)
2026-02-21T09:57:07.1862532Z Installing collected packages: botocore, s3transfer, boto3
2026-02-21T09:57:07.7388294Z Successfully installed boto3-1.35.33 botocore-1.35.99 s3transfer-0.10.4
2026-02-21T09:57:07.9354422Z + [[ false == \t\r\u\e ]]
2026-02-21T09:57:07.9357709Z + python3 .github/scripts/upload_benchmark_results.py --benchmark-results-dir /home/ec2-user/actions-runner/_work/_temp/benchmark-artifact --metadata '{"timestamp": 1771667800, "schema_version": "v3", "name": "run-b200 (rms_norm-bwd) / benchmark-cu130-rms_norm-bwd-py3.12-b200", "repo": "pytorch/helion", "head_branch": "refs/heads/main", "head_sha": "874a7d0cadab18218a84ad3579d329dc95c51820", "workflow_id": 22253280836, "run_attempt": 1, "job_id": 64380329763}' --runners '[{"cpu_info": "x86_64", "cpu_count": 224, "avail_mem_in_gb": 2015, "extra_info": {"hostname": "22f9d4a00982"}, "name": "cuda", "type": "NVIDIA B200", "gpu_count": 1, "avail_gpu_mem_in_gb": 178}]' --dependencies '{}'
2026-02-21T09:57:08.0730216Z INFO:root:Upload /home/ec2-user/actions-runner/_work/_temp/benchmark-artifact/helionbench.json to s3://ossci-benchmarks/v3/pytorch/helion/22253280836/64380329763/helionbench.json
2026-02-21T09:57:08.0976637Z INFO:botocore.credentials:Found credentials from IAM Role: gh-ci-github-action-runners-runner-role
2026-02-21T09:57:08.2976714Z Post job cleanup.
2026-02-21T09:57:08.3920497Z [command]/usr/bin/git version
2026-02-21T09:57:08.3955213Z git version 2.50.1
2026-02-21T09:57:08.3994781Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/3aeb4845-736c-4b35-b3df-5d7e60f0ee6c' before making global git config changes
2026-02-21T09:57:08.3995793Z Adding repository directory to the temporary git global config as a safe directory
2026-02-21T09:57:08.3999360Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/helion/helion/test-infra
2026-02-21T09:57:08.4029071Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2026-02-21T09:57:08.4064729Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
2026-02-21T09:57:08.4337484Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2026-02-21T09:57:08.4355809Z http.https://github.com/.extraheader
2026-02-21T09:57:08.4365286Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader
2026-02-21T09:57:08.4390799Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
2026-02-21T09:57:08.4717110Z A job completed hook has been configured by the self-hosted runner administrator
2026-02-21T09:57:08.4740939Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh'
2026-02-21T09:57:08.4746232Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2026-02-21T09:57:08.4746625Z ##[endgroup]
2026-02-21T09:57:08.4903009Z [!ALERT!] Swap in detected! [!ALERT!]
2026-02-21T09:57:19.8643465Z [!ALERT!] Swap out detected [!ALERT!]
2026-02-21T09:57:38.3690451Z Cleaning up orphan processes