2025-12-04T09:22:10.7446901Z Current runner version: '2.330.0' 2025-12-04T09:22:10.7451900Z Runner name: 'i-04ab61807b125ed1d' 2025-12-04T09:22:10.7452565Z Runner group name: 'default' 2025-12-04T09:22:10.7453268Z Machine name: 'ip-10-0-71-55' 2025-12-04T09:22:10.7455300Z ##[group]GITHUB_TOKEN Permissions 2025-12-04T09:22:10.7457556Z Contents: read 2025-12-04T09:22:10.7457981Z Metadata: read 2025-12-04T09:22:10.7458423Z ##[endgroup] 2025-12-04T09:22:10.7460213Z Secret source: Actions 2025-12-04T09:22:10.7460801Z Prepare workflow directory 2025-12-04T09:22:10.7857014Z Prepare all required actions 2025-12-04T09:22:10.7889111Z Getting action download info 2025-12-04T09:22:11.1231623Z Download action repository 'pytorch/test-infra@main' (SHA:39aa74d619174326f4e2fb0e216151c2f29d9ffd) 2025-12-04T09:22:13.5382647Z Download action repository 'pytorch/pytorch@main' (SHA:7716da9fb23f27a65b41f9f016a2afadf281c18f) 2025-12-04T09:22:30.5157785Z Download action repository 'actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065' (SHA:a26af69be951a213d495a4c3e4e4022e16d87065) 2025-12-04T09:22:30.9119813Z Download action repository 'aws-actions/configure-aws-credentials@ececac1a45f3b08a01d2dd070d28d111c5fe6722' (SHA:ececac1a45f3b08a01d2dd070d28d111c5fe6722) 2025-12-04T09:22:31.1413345Z Download action repository 'aws-actions/amazon-ecr-login@062b18b96a7aff071d4dc91bc00c4c1a7945b076' (SHA:062b18b96a7aff071d4dc91bc00c4c1a7945b076) 2025-12-04T09:22:31.3389150Z Download action repository 'seemethere/download-artifact-s3@1da556a7aa0a088e3153970611f6c432d58e80e6' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2025-12-04T09:22:31.6331127Z Download action repository 'seemethere/upload-artifact-s3@baba72d0712b404f646cebe0730933554ebce96a' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2025-12-04T09:22:31.9128531Z Getting action download info 2025-12-04T09:22:32.0457284Z Download action repository 'actions/checkout@v4' (SHA:34e114876b0b11c390a56381ad16ebd13914f8d5) 2025-12-04T09:22:32.2947929Z Getting action download info 2025-12-04T09:22:32.4487546Z Download action repository 'nick-fields/retry@v3.0.0' (SHA:7152eba30c6575329ac0576536151aca5a72780e) 2025-12-04T09:22:32.6499482Z Getting action download info 2025-12-04T09:22:32.8197387Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2025-12-04T09:22:32.9953572Z Getting action download info 2025-12-04T09:22:33.1437107Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/heads/main (ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32) 2025-12-04T09:22:33.1440472Z ##[group] Inputs 2025-12-04T09:22:33.1440891Z build-environment: linux-jammy-py3.10-gcc11-build 2025-12-04T09:22:33.1447526Z test-matrix: {"include": [{"config": "cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_avx2_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_timm", "shard": 1, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_timm", "shard": 2, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}]} 2025-12-04T09:22:33.1454194Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:22:33.1454890Z sync-tag: 2025-12-04T09:22:33.1455659Z timeout-minutes: 240 2025-12-04T09:22:33.1455919Z use-gha: 2025-12-04T09:22:33.1456186Z dashboard-tag: 2025-12-04T09:22:33.1456417Z s3-bucket: gha-artifacts 2025-12-04T09:22:33.1456686Z aws-role-to-assume: 2025-12-04T09:22:33.1457170Z disable-monitor: false 2025-12-04T09:22:33.1457495Z monitor-log-interval: 5 2025-12-04T09:22:33.1457802Z monitor-data-collect-interval: 1 2025-12-04T09:22:33.1458201Z ##[endgroup] 2025-12-04T09:22:33.1458681Z Complete job name: periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_huggingface, 1, 1, linux.8xlarge.amx) 2025-12-04T09:22:33.1933020Z A job started hook has been configured by the self-hosted runner administrator 2025-12-04T09:22:33.2011120Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2025-12-04T09:22:33.2111042Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:22:33.2111559Z ##[endgroup] 2025-12-04T09:22:34.2082721Z Runner Type: linux.8xlarge.amx 2025-12-04T09:22:34.2083176Z Instance Type: m7i-flex.8xlarge 2025-12-04T09:22:34.2083382Z AMI Name: unknown 2025-12-04T09:22:34.2110607Z AMI ID: ami-08982f1c5bf93d976 2025-12-04T09:22:38.5365751Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2025-12-04T09:22:38.5366053Z with: 2025-12-04T09:22:38.5366574Z github-secret: *** 2025-12-04T09:22:38.5367012Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2025-12-04T09:22:38.5367463Z activate-with-label: false 2025-12-04T09:22:38.5367652Z label: with-ssh 2025-12-04T09:22:38.5367817Z remove-existing-keys: true 2025-12-04T09:22:38.5368210Z fail-silently: true 2025-12-04T09:22:38.5368372Z env: 2025-12-04T09:22:38.5368512Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:22:38.5368700Z ##[endgroup] 2025-12-04T09:22:38.6329916Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2025-12-04T09:22:38.6330800Z Not on pull request and ciflow reference could not be extracted, skipping adding ssh keys 2025-12-04T09:22:38.6463795Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2025-12-04T09:22:38.6464153Z with: 2025-12-04T09:22:38.6464362Z no-sudo: true 2025-12-04T09:22:38.6464584Z submodules: recursive 2025-12-04T09:22:38.6464811Z fetch-depth: 0 2025-12-04T09:22:38.6465019Z env: 2025-12-04T09:22:38.6465213Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:22:38.6465447Z ##[endgroup] 2025-12-04T09:22:38.6530898Z ##[group]Run echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-12-04T09:22:38.6531516Z echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-12-04T09:22:38.6539991Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:22:38.6540232Z env: 2025-12-04T09:22:38.6540415Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:22:38.6540633Z ##[endgroup] 2025-12-04T09:22:38.6620103Z ##[group]Run # Use all available CPUs for fetching 2025-12-04T09:22:38.6620430Z # Use all available CPUs for fetching 2025-12-04T09:22:38.6620667Z cd "${GITHUB_WORKSPACE}" 2025-12-04T09:22:38.6620893Z git config --global fetch.parallel 0 2025-12-04T09:22:38.6621154Z git config --global submodule.fetchJobs 0 2025-12-04T09:22:38.6621385Z  2025-12-04T09:22:38.6621732Z # Clean workspace. The default checkout action should also do this, but 2025-12-04T09:22:38.6622061Z # do it here as well just in case 2025-12-04T09:22:38.6622282Z if [[ -d .git ]]; then 2025-12-04T09:22:38.6622498Z  if [ -z "${NO_SUDO}" ]; then 2025-12-04T09:22:38.6622707Z  sudo git clean -ffdx 2025-12-04T09:22:38.6622900Z  else 2025-12-04T09:22:38.6623072Z  git clean -ffdx 2025-12-04T09:22:38.6623249Z  fi 2025-12-04T09:22:38.6623405Z fi 2025-12-04T09:22:38.6627791Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:22:38.6628030Z env: 2025-12-04T09:22:38.6628186Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:22:38.6628367Z NO_SUDO: true 2025-12-04T09:22:38.6628519Z ##[endgroup] 2025-12-04T09:22:38.6739165Z ##[group]Run actions/checkout@v4 2025-12-04T09:22:38.6739406Z with: 2025-12-04T09:22:38.6739611Z ref: ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:22:38.6739840Z fetch-depth: 0 2025-12-04T09:22:38.6740024Z submodules: recursive 2025-12-04T09:22:38.6740217Z show-progress: false 2025-12-04T09:22:38.6740404Z repository: pytorch/pytorch 2025-12-04T09:22:38.6740702Z token: *** 2025-12-04T09:22:38.6740870Z ssh-strict: true 2025-12-04T09:22:38.6741037Z ssh-user: git 2025-12-04T09:22:38.6741205Z persist-credentials: true 2025-12-04T09:22:38.6741397Z clean: true 2025-12-04T09:22:38.6741576Z sparse-checkout-cone-mode: true 2025-12-04T09:22:38.6741777Z fetch-tags: false 2025-12-04T09:22:38.6741943Z lfs: false 2025-12-04T09:22:38.6742112Z set-safe-directory: true 2025-12-04T09:22:38.6742299Z env: 2025-12-04T09:22:38.6742460Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:22:38.6742646Z ##[endgroup] 2025-12-04T09:22:38.7642688Z Syncing repository: pytorch/pytorch 2025-12-04T09:22:38.7643783Z ##[group]Getting Git version info 2025-12-04T09:22:38.7644108Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2025-12-04T09:22:38.7644552Z [command]/usr/bin/git version 2025-12-04T09:22:38.7644748Z git version 2.50.1 2025-12-04T09:22:38.7645417Z ##[endgroup] 2025-12-04T09:22:38.7653729Z Copying '/home/ec2-user/.gitconfig' to '/home/ec2-user/actions-runner/_work/_temp/40aa8954-6206-4176-b2ee-3187ba997cfc/.gitconfig' 2025-12-04T09:22:38.7678914Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/40aa8954-6206-4176-b2ee-3187ba997cfc' before making global git config changes 2025-12-04T09:22:38.7679868Z Adding repository directory to the temporary git global config as a safe directory 2025-12-04T09:22:38.7687609Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2025-12-04T09:22:38.7717606Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2025-12-04T09:22:38.7726981Z ##[group]Initializing the repository 2025-12-04T09:22:38.7732603Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2025-12-04T09:22:38.7767447Z hint: Using 'master' as the name for the initial branch. This default branch name 2025-12-04T09:22:38.7772716Z hint: is subject to change. To configure the initial branch name to use in all 2025-12-04T09:22:38.7775000Z hint: of your new repositories, which will suppress this warning, call: 2025-12-04T09:22:38.7775537Z hint: 2025-12-04T09:22:38.7775935Z hint: git config --global init.defaultBranch 2025-12-04T09:22:38.7776354Z hint: 2025-12-04T09:22:38.7777196Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2025-12-04T09:22:38.7777679Z hint: 'development'. The just-created branch can be renamed via this command: 2025-12-04T09:22:38.7777961Z hint: 2025-12-04T09:22:38.7778142Z hint: git branch -m 2025-12-04T09:22:38.7778337Z hint: 2025-12-04T09:22:38.7778585Z hint: Disable this message with "git config set advice.defaultBranchName false" 2025-12-04T09:22:38.7779039Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2025-12-04T09:22:38.7779895Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2025-12-04T09:22:38.7803676Z ##[endgroup] 2025-12-04T09:22:38.7804068Z ##[group]Disabling automatic garbage collection 2025-12-04T09:22:38.7810191Z [command]/usr/bin/git config --local gc.auto 0 2025-12-04T09:22:38.7834189Z ##[endgroup] 2025-12-04T09:22:38.7834495Z ##[group]Setting up auth 2025-12-04T09:22:38.7843054Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2025-12-04T09:22:38.7870450Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2025-12-04T09:22:38.8224198Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2025-12-04T09:22:38.8255880Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2025-12-04T09:22:38.8808911Z [command]/usr/bin/git config --local --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:22:38.8857313Z [command]/usr/bin/git submodule foreach --recursive git config --local --show-origin --name-only --get-regexp remote.origin.url 2025-12-04T09:22:38.9173247Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2025-12-04T09:22:38.9230501Z ##[endgroup] 2025-12-04T09:22:38.9231186Z ##[group]Fetching the repository 2025-12-04T09:22:38.9243922Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --no-recurse-submodules origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2025-12-04T09:23:32.5294903Z From https://github.com/pytorch/pytorch 2025-12-04T09:23:32.5303830Z * [new branch] 2.6.0.dev20241004+ -> origin/2.6.0.dev20241004+ 2025-12-04T09:23:32.5304351Z * [new branch] 2.9.1 -> origin/2.9.1 2025-12-04T09:23:32.5304769Z * [new branch] AaronWang04_addmmfusion_perftest -> origin/AaronWang04_addmmfusion_perftest 2025-12-04T09:23:32.5305211Z * [new branch] Flamefire-patch-1 -> origin/Flamefire-patch-1 2025-12-04T09:23:32.5306238Z * [new branch] HDCharles-2.6.0-release-notes -> origin/HDCharles-2.6.0-release-notes 2025-12-04T09:23:32.5306596Z * [new branch] HOPrintFunc -> origin/HOPrintFunc 2025-12-04T09:23:32.5306931Z * [new branch] IvanKobzarev/stack/1 -> origin/IvanKobzarev/stack/1 2025-12-04T09:23:32.5307387Z * [new branch] NicoshevSVE128 -> origin/NicoshevSVE128 2025-12-04T09:23:32.5307756Z * [new branch] PR-AOTInductorNoneBug -> origin/PR-AOTInductorNoneBug 2025-12-04T09:23:32.5308138Z * [new branch] PR-AOTInductorNoneBugFix -> origin/PR-AOTInductorNoneBugFix 2025-12-04T09:23:32.5308508Z * [new branch] PR-FixConfigsIssue -> origin/PR-FixConfigsIssue 2025-12-04T09:23:32.5308870Z * [new branch] PR-NoneBugFix-viable -> origin/PR-NoneBugFix-viable 2025-12-04T09:23:32.5309197Z * [new branch] PR-ResetToZero -> origin/PR-ResetToZero 2025-12-04T09:23:32.5309539Z * [new branch] Update-Flash-Packaging -> origin/Update-Flash-Packaging 2025-12-04T09:23:32.5309857Z * [new branch] VLA_exp -> origin/VLA_exp 2025-12-04T09:23:32.5310157Z * [new branch] activation_bench -> origin/activation_bench 2025-12-04T09:23:32.5310474Z * [new branch] addmm-heuristic -> origin/addmm-heuristic 2025-12-04T09:23:32.5310790Z * [new branch] adi/onednn_aarch64 -> origin/adi/onednn_aarch64 2025-12-04T09:23:32.5311079Z * [new branch] adi/test -> origin/adi/test 2025-12-04T09:23:32.5311368Z * [new branch] adi/test_bgemm -> origin/adi/test_bgemm 2025-12-04T09:23:32.5311661Z * [new branch] adi/test_m8g -> origin/adi/test_m8g 2025-12-04T09:23:32.5311961Z * [new branch] adi/test_onednn -> origin/adi/test_onednn 2025-12-04T09:23:32.5312287Z * [new branch] adi/test_onednn_v3.9 -> origin/adi/test_onednn_v3.9 2025-12-04T09:23:32.5312619Z * [new branch] adi/test_presve_change -> origin/adi/test_presve_change 2025-12-04T09:23:32.5313055Z * [new branch] adi/test_timm -> origin/adi/test_timm 2025-12-04T09:23:32.5313379Z * [new branch] adi/testpresve_change -> origin/adi/testpresve_change 2025-12-04T09:23:32.5313746Z * [new branch] aditew01/test/vec_bf16 -> origin/aditew01/test/vec_bf16 2025-12-04T09:23:32.5314131Z * [new branch] ah-globalfeedback-hook -> origin/ah-globalfeedback-hook 2025-12-04T09:23:32.5318087Z * [new branch] albanD-patch-1 -> origin/albanD-patch-1 2025-12-04T09:23:32.5318518Z * [new branch] also-surround-shimh -> origin/also-surround-shimh 2025-12-04T09:23:32.5318905Z * [new branch] angelayi/aot_compile -> origin/angelayi/aot_compile 2025-12-04T09:23:32.5319373Z * [new branch] angelayi/aoti_additional_files -> origin/angelayi/aoti_additional_files 2025-12-04T09:23:32.5323055Z * [new branch] angelayi/benchmark -> origin/angelayi/benchmark 2025-12-04T09:23:32.5323537Z * [new branch] angelayi/change_pytree_serialization -> origin/angelayi/change_pytree_serialization 2025-12-04T09:23:32.5323965Z * [new branch] angelayi/cpp_loader -> origin/angelayi/cpp_loader 2025-12-04T09:23:32.5324324Z * [new branch] angelayi/inductor_const -> origin/angelayi/inductor_const 2025-12-04T09:23:32.5324658Z * [new branch] angelayi/lstm -> origin/angelayi/lstm 2025-12-04T09:23:32.5324985Z * [new branch] angelayi/no_so_weight -> origin/angelayi/no_so_weight 2025-12-04T09:23:32.5326841Z * [new branch] angelayi/scan_layers -> origin/angelayi/scan_layers 2025-12-04T09:23:32.5327490Z * [new branch] angelayi/side_eff -> origin/angelayi/side_eff 2025-12-04T09:23:32.5332295Z * [new branch] angelayi/state_dict -> origin/angelayi/state_dict 2025-12-04T09:23:32.5336601Z * [new branch] angelayi/symint_input -> origin/angelayi/symint_input 2025-12-04T09:23:32.5341541Z * [new branch] angelayi/symm_mem -> origin/angelayi/symm_mem 2025-12-04T09:23:32.5346020Z * [new branch] angelayi/test_cpp -> origin/angelayi/test_cpp 2025-12-04T09:23:32.5348331Z * [new branch] angelayi/torch_size -> origin/angelayi/torch_size 2025-12-04T09:23:32.5348721Z * [new branch] annotate_assert -> origin/annotate_assert 2025-12-04T09:23:32.5349049Z * [new branch] annotate_fallback_kernel -> origin/annotate_fallback_kernel 2025-12-04T09:23:32.5349400Z * [new branch] annotation_deepcopy -> origin/annotation_deepcopy 2025-12-04T09:23:32.5349737Z * [new branch] annotation_dynamo -> origin/annotation_dynamo 2025-12-04T09:23:32.5350054Z * [new branch] aot_eager_stack_trace -> origin/aot_eager_stack_trace 2025-12-04T09:23:32.5350397Z * [new branch] aoti-cuda-alloc -> origin/aoti-cuda-alloc 2025-12-04T09:23:32.5350721Z * [new branch] aoti_const_device -> origin/aoti_const_device 2025-12-04T09:23:32.5351049Z * [new branch] aoti_fqn_name_interface -> origin/aoti_fqn_name_interface 2025-12-04T09:23:32.5351407Z * [new branch] aoti_package_weights_binary -> origin/aoti_package_weights_binary 2025-12-04T09:23:32.5351746Z * [new branch] aoti_target_windows -> origin/aoti_target_windows 2025-12-04T09:23:32.5352126Z * [new branch] arsh/feat/inductor_check_profiling -> origin/arsh/feat/inductor_check_profiling 2025-12-04T09:23:32.5352493Z * [new branch] async_tp -> origin/async_tp 2025-12-04T09:23:32.5353049Z * [new branch] atalman-inductor-perf-cu124 -> origin/atalman-inductor-perf-cu124 2025-12-04T09:23:32.5353512Z * [new branch] atalman-inductor-perf-cu124.1 -> origin/atalman-inductor-perf-cu124.1 2025-12-04T09:23:32.5353929Z * [new branch] atalman-patch-2 -> origin/atalman-patch-2 2025-12-04T09:23:32.5354278Z * [new branch] atalman-patch-3 -> origin/atalman-patch-3 2025-12-04T09:23:32.5354614Z * [new branch] atalman-patch-4 -> origin/atalman-patch-4 2025-12-04T09:23:32.5354940Z * [new branch] atalman-patch-5 -> origin/atalman-patch-5 2025-12-04T09:23:32.5355270Z * [new branch] atalman-patch-6 -> origin/atalman-patch-6 2025-12-04T09:23:32.5355630Z * [new branch] atalman-patch-7 -> origin/atalman-patch-7 2025-12-04T09:23:32.5355964Z * [new branch] atalman-patch-8 -> origin/atalman-patch-8 2025-12-04T09:23:32.5356280Z * [new branch] atalman_inductor_2.3.1 -> origin/atalman_inductor_2.3.1 2025-12-04T09:23:32.5356616Z * [new branch] atalman_inductor_2.4.0 -> origin/atalman_inductor_2.4.0 2025-12-04T09:23:32.5356953Z * [new branch] atalman_inductor_2.4.x -> origin/atalman_inductor_2.4.x 2025-12-04T09:23:32.5357315Z * [new branch] attention_benchmarking_clean -> origin/attention_benchmarking_clean 2025-12-04T09:23:32.5357681Z * [new branch] bahuang/dt_fix_scalar_add -> origin/bahuang/dt_fix_scalar_add 2025-12-04T09:23:32.5358119Z * [new branch] bahuang/fix_debug_mode -> origin/bahuang/fix_debug_mode 2025-12-04T09:23:32.5358456Z * [new branch] bahuang/fix_expand -> origin/bahuang/fix_expand 2025-12-04T09:23:32.5358767Z * [new branch] bahuang/test -> origin/bahuang/test 2025-12-04T09:23:32.5359207Z * [new branch] base/1.5 -> origin/base/1.5 2025-12-04T09:23:32.5359554Z * [new branch] batching_sdpa_efficient_attention -> origin/batching_sdpa_efficient_attention 2025-12-04T09:23:32.5359939Z * [new branch] bench_scaled_mm_ops -> origin/bench_scaled_mm_ops 2025-12-04T09:23:32.5360297Z * [new branch] benchmark-updates -> origin/benchmark-updates 2025-12-04T09:23:32.5360640Z * [new branch] benchmarking-script -> origin/benchmarking-script 2025-12-04T09:23:32.5360982Z * [new branch] bertmaher/pinbump26 -> origin/bertmaher/pinbump26 2025-12-04T09:23:32.5361307Z * [new branch] bertrand/cutlass -> origin/bertrand/cutlass 2025-12-04T09:23:32.5361631Z * [new branch] bf/bug-static-input -> origin/bf/bug-static-input 2025-12-04T09:23:32.5361949Z * [new branch] bf/cg-backend -> origin/bf/cg-backend 2025-12-04T09:23:32.5362259Z * [new branch] bf/cg-nccl-test -> origin/bf/cg-nccl-test 2025-12-04T09:23:32.5362569Z * [new branch] bf/cg-remove-check -> origin/bf/cg-remove-check 2025-12-04T09:23:32.5362891Z * [new branch] bf/clean-torchbench-hf -> origin/bf/clean-torchbench-hf 2025-12-04T09:23:32.5363228Z * [new branch] bf/combo-debug-log -> origin/bf/combo-debug-log 2025-12-04T09:23:32.5363537Z * [new branch] bf/cudagraph -> origin/bf/cudagraph 2025-12-04T09:23:32.5363921Z * [new branch] bf/cudagraph-disable-input-mutation -> origin/bf/cudagraph-disable-input-mutation 2025-12-04T09:23:32.5365061Z * [new branch] bf/cudagraph-enable-input-mutation-support-benchmark -> origin/bf/cudagraph-enable-input-mutation-support-benchmark 2025-12-04T09:23:32.5365586Z * [new branch] bf/cudagraph-partition -> origin/bf/cudagraph-partition 2025-12-04T09:23:32.5365948Z * [new branch] bf/donated-buffer-bench -> origin/bf/donated-buffer-bench 2025-12-04T09:23:32.5366292Z * [new branch] bf/dynamo-partition -> origin/bf/dynamo-partition 2025-12-04T09:23:32.5366587Z * [new branch] bf/lite -> origin/bf/lite 2025-12-04T09:23:32.5366900Z * [new branch] bf/pa-non-divisible -> origin/bf/pa-non-divisible 2025-12-04T09:23:32.5368759Z * [new branch] bf/partition-cache-free-symbols -> origin/bf/partition-cache-free-symbols 2025-12-04T09:23:32.5369182Z * [new branch] bf/partition-memory-plan -> origin/bf/partition-memory-plan 2025-12-04T09:23:32.5369541Z * [new branch] bf/partition-move-cpu -> origin/bf/partition-move-cpu 2025-12-04T09:23:32.5369915Z * [new branch] bf/partition-view-fallback -> origin/bf/partition-view-fallback 2025-12-04T09:23:32.5370293Z * [new branch] bf/remove-check-55b0c39d -> origin/bf/remove-check-55b0c39d 2025-12-04T09:23:32.5370642Z * [new branch] bf/timm-nov-26-2025 -> origin/bf/timm-nov-26-2025 2025-12-04T09:23:32.5374113Z * [new branch] bf/transformer-pin-4-57-3 -> origin/bf/transformer-pin-4-57-3 2025-12-04T09:23:32.5374510Z * [new branch] bisect_perf_hf_T5_3acc6eac492 -> origin/bisect_perf_hf_T5_3acc6eac492 2025-12-04T09:23:32.5374889Z * [new branch] bisect_perf_hf_T5_3fcf66f61fb -> origin/bisect_perf_hf_T5_3fcf66f61fb 2025-12-04T09:23:32.5375256Z * [new branch] bisect_perf_hf_T5_4009d154129 -> origin/bisect_perf_hf_T5_4009d154129 2025-12-04T09:23:32.5375615Z * [new branch] bisect_perf_hf_T5_40d0740e73d -> origin/bisect_perf_hf_T5_40d0740e73d 2025-12-04T09:23:32.5375980Z * [new branch] bisect_perf_hf_T5_5268754e -> origin/bisect_perf_hf_T5_5268754e 2025-12-04T09:23:32.5376344Z * [new branch] bisect_perf_hf_T5_7d89a8d385c -> origin/bisect_perf_hf_T5_7d89a8d385c 2025-12-04T09:23:32.5376773Z * [new branch] bisect_perf_hf_T5_b7a25c1ee7c -> origin/bisect_perf_hf_T5_b7a25c1ee7c 2025-12-04T09:23:32.5377154Z * [new branch] bisect_perf_hf_T5_c25b201583f -> origin/bisect_perf_hf_T5_c25b201583f 2025-12-04T09:23:32.5377516Z * [new branch] bisect_perf_hf_T5_c93e57efac0 -> origin/bisect_perf_hf_T5_c93e57efac0 2025-12-04T09:23:32.5378359Z * [new branch] bisect_perf_hf_T5_ca9813ea149 -> origin/bisect_perf_hf_T5_ca9813ea149 2025-12-04T09:23:32.5378750Z * [new branch] bisect_perf_hf_T5_d65f194a -> origin/bisect_perf_hf_T5_d65f194a 2025-12-04T09:23:32.5379118Z * [new branch] bisect_perf_hf_T5_da94ab0b -> origin/bisect_perf_hf_T5_da94ab0b 2025-12-04T09:23:32.5379512Z * [new branch] bisect_perf_hf_T5_da94ab0b_new -> origin/bisect_perf_hf_T5_da94ab0b_new 2025-12-04T09:23:32.5379923Z * [new branch] bisect_perf_hf_T5_db4e8a1d8a8 -> origin/bisect_perf_hf_T5_db4e8a1d8a8 2025-12-04T09:23:32.5380317Z * [new branch] bisect_perf_hf_T5_e0d97e936a2 -> origin/bisect_perf_hf_T5_e0d97e936a2 2025-12-04T09:23:32.5383166Z * [new branch] bisect_perf_hf_T5_f23621ec563 -> origin/bisect_perf_hf_T5_f23621ec563 2025-12-04T09:23:32.5383519Z * [new branch] brister/fx_device_type -> origin/brister/fx_device_type 2025-12-04T09:23:32.5383993Z * [new branch] brister/test_inductor_all_fx -> origin/brister/test_inductor_all_fx 2025-12-04T09:23:32.5388292Z * [new branch] brister/tiled_reduction_no_numel_check -> origin/brister/tiled_reduction_no_numel_check 2025-12-04T09:23:32.5388744Z * [new branch] bwd-backup -> origin/bwd-backup 2025-12-04T09:23:32.5389055Z * [new branch] c57382a49 -> origin/c57382a49 2025-12-04T09:23:32.5389360Z * [new branch] ca_0431d47eaa -> origin/ca_0431d47eaa 2025-12-04T09:23:32.5389688Z * [new branch] ca_fix_0431d47eaa -> origin/ca_fix_0431d47eaa 2025-12-04T09:23:32.5390058Z * [new branch] camyllh/test_setup_hooks_push -> origin/camyllh/test_setup_hooks_push 2025-12-04T09:23:32.5390445Z * [new branch] cccclai-patch-1 -> origin/cccclai-patch-1 2025-12-04T09:23:32.5390921Z * [new branch] cherry-pick-159969-by-pytorch_bot_bot_ -> origin/cherry-pick-159969-by-pytorch_bot_bot_ 2025-12-04T09:23:32.5391429Z * [new branch] cherry-pick-160586-by-pytorch_bot_bot_ -> origin/cherry-pick-160586-by-pytorch_bot_bot_ 2025-12-04T09:23:32.5392053Z * [new branch] cherry-pick-162208-by-pytorch_bot_bot_ -> origin/cherry-pick-162208-by-pytorch_bot_bot_ 2025-12-04T09:23:32.5392539Z * [new branch] cherry-pick-163169-by-pytorch_bot_bot_ -> origin/cherry-pick-163169-by-pytorch_bot_bot_ 2025-12-04T09:23:32.5393279Z * [new branch] cherry-pick-165086-by-pytorch_bot_bot_ -> origin/cherry-pick-165086-by-pytorch_bot_bot_ 2025-12-04T09:23:32.5393765Z * [new branch] cherry-pick-165514-by-pytorch_bot_bot_ -> origin/cherry-pick-165514-by-pytorch_bot_bot_ 2025-12-04T09:23:32.5394247Z * [new branch] cherry-pick-165601-by-pytorch_bot_bot_ -> origin/cherry-pick-165601-by-pytorch_bot_bot_ 2025-12-04T09:23:32.5394704Z * [new branch] cherry-pick-165667-by-pytorch_bot_bot_ -> origin/cherry-pick-165667-by-pytorch_bot_bot_ 2025-12-04T09:23:32.5395153Z * [new branch] cherry-pick-165815-by-pytorch_bot_bot_ -> origin/cherry-pick-165815-by-pytorch_bot_bot_ 2025-12-04T09:23:32.5395606Z * [new branch] cherry-pick-165922-by-pytorch_bot_bot_ -> origin/cherry-pick-165922-by-pytorch_bot_bot_ 2025-12-04T09:23:32.5401115Z * [new branch] cherry-pick-166148-by-pytorch_bot_bot_ -> origin/cherry-pick-166148-by-pytorch_bot_bot_ 2025-12-04T09:23:32.5402999Z * [new branch] cherry-pick-166181-by-pytorch_bot_bot_ -> origin/cherry-pick-166181-by-pytorch_bot_bot_ 2025-12-04T09:23:32.5403661Z * [new branch] cherry-pick-166404-by-pytorch_bot_bot_ -> origin/cherry-pick-166404-by-pytorch_bot_bot_ 2025-12-04T09:23:32.5404131Z * [new branch] cherry-pick-166427-by-pytorch_bot_bot_ -> origin/cherry-pick-166427-by-pytorch_bot_bot_ 2025-12-04T09:23:32.5404672Z * [new branch] cherry-pick-166480-by-pytorch_bot_bot_ -> origin/cherry-pick-166480-by-pytorch_bot_bot_ 2025-12-04T09:23:32.5405135Z * [new branch] cherry-pick-166570-by-pytorch_bot_bot_ -> origin/cherry-pick-166570-by-pytorch_bot_bot_ 2025-12-04T09:23:32.5405593Z * [new branch] cherry-pick-166993-by-pytorch_bot_bot_ -> origin/cherry-pick-166993-by-pytorch_bot_bot_ 2025-12-04T09:23:32.5406212Z * [new branch] cherry-pick-167111-by-pytorch_bot_bot_ -> origin/cherry-pick-167111-by-pytorch_bot_bot_ 2025-12-04T09:23:32.5406659Z * [new branch] cherry-pick-167478-by-pytorch_bot_bot_ -> origin/cherry-pick-167478-by-pytorch_bot_bot_ 2025-12-04T09:23:32.5407073Z * [new branch] cherry_pick_166036_166040 -> origin/cherry_pick_166036_166040 2025-12-04T09:23:32.5407418Z * [new branch] cherry_pick_166457 -> origin/cherry_pick_166457 2025-12-04T09:23:32.5407745Z * [new branch] cherrypick_166338 -> origin/cherrypick_166338 2025-12-04T09:23:32.5408054Z * [new branch] cherrypick_166458 -> origin/cherrypick_166458 2025-12-04T09:23:32.5408380Z * [new branch] cherrypick_166586 -> origin/cherrypick_166586 2025-12-04T09:23:32.5408713Z * [new branch] cherrypick_166956 -> origin/cherrypick_166956 2025-12-04T09:23:32.5409009Z * [new branch] ci_attn -> origin/ci_attn 2025-12-04T09:23:32.5409301Z * [new branch] codex-testing -> origin/codex-testing 2025-12-04T09:23:32.5409756Z * [new branch] codex/add-check_memory_overlap-helper-functions -> origin/codex/add-check_memory_overlap-helper-functions 2025-12-04T09:23:32.5410267Z * [new branch] codex/fix-issue-121219-in-pytorch -> origin/codex/fix-issue-121219-in-pytorch 2025-12-04T09:23:32.5410801Z * [new branch] codex/investigate-segfaults-in-get_tensor_storage_id -> origin/codex/investigate-segfaults-in-get_tensor_storage_id 2025-12-04T09:23:32.5415798Z * [new branch] codex/refactor-lintrunner-config-to-use-uv-run -> origin/codex/refactor-lintrunner-config-to-use-uv-run 2025-12-04T09:23:32.5418353Z * [new branch] compatiblpy39util -> origin/compatiblpy39util 2025-12-04T09:23:32.5418772Z * [new branch] cond_hop_device -> origin/cond_hop_device 2025-12-04T09:23:32.5419088Z * [new branch] context_test -> origin/context_test 2025-12-04T09:23:32.5419506Z * [new branch] copilot/code-style-cleanup-python-pip -> origin/copilot/code-style-cleanup-python-pip 2025-12-04T09:23:32.5419965Z * [new branch] cpio/fix_new_ami_tests -> origin/cpio/fix_new_ami_tests 2025-12-04T09:23:32.5420350Z * [new branch] cpp-docs-dependency-upgrade -> origin/cpp-docs-dependency-upgrade 2025-12-04T09:23:32.5420817Z * [new branch] crpa/typo-in-inductor_comm_lowering -> origin/crpa/typo-in-inductor_comm_lowering 2025-12-04T09:23:32.5425909Z * [new branch] csl/always_produce_xml -> origin/csl/always_produce_xml 2025-12-04T09:23:32.5426393Z * [new branch] csl/build_test_more_procs -> origin/csl/build_test_more_procs 2025-12-04T09:23:32.5426777Z * [new branch] csl/build_test_more_procs2 -> origin/csl/build_test_more_procs2 2025-12-04T09:23:32.5427106Z * [new branch] csl/clean_up -> origin/csl/clean_up 2025-12-04T09:23:32.5427474Z * [new branch] csl/fix_retry_segfault_exit -> origin/csl/fix_retry_segfault_exit 2025-12-04T09:23:32.5427808Z * [new branch] csl/katex -> origin/csl/katex 2025-12-04T09:23:32.5428468Z * [new branch] csl/larger_runner -> origin/csl/larger_runner 2025-12-04T09:23:32.5428784Z * [new branch] csl/lint_testing -> origin/csl/lint_testing 2025-12-04T09:23:32.5429100Z * [new branch] csl/lint_thing -> origin/csl/lint_thing 2025-12-04T09:23:32.5429497Z * [new branch] csl/lintrunner_stuff -> origin/csl/lintrunner_stuff 2025-12-04T09:23:32.5429839Z * [new branch] csl/manually_gen_json -> origin/csl/manually_gen_json 2025-12-04T09:23:32.5430159Z * [new branch] csl/mps_sharding -> origin/csl/mps_sharding 2025-12-04T09:23:32.5430487Z * [new branch] csl/multistage_docker -> origin/csl/multistage_docker 2025-12-04T09:23:32.5430816Z * [new branch] csl/print_timing -> origin/csl/print_timing 2025-12-04T09:23:32.5431143Z * [new branch] csl/remove_experiment -> origin/csl/remove_experiment 2025-12-04T09:23:32.5431495Z * [new branch] csl/remove_maybe_unused_var -> origin/csl/remove_maybe_unused_var 2025-12-04T09:23:32.5431893Z * [new branch] csl/remove_repo_specific_autolabel -> origin/csl/remove_repo_specific_autolabel 2025-12-04T09:23:32.5432305Z * [new branch] csl/remove_run_parallel -> origin/csl/remove_run_parallel 2025-12-04T09:23:32.5432822Z * [new branch] csl/remove_unused_vars -> origin/csl/remove_unused_vars 2025-12-04T09:23:32.5433176Z * [new branch] csl/revert_open -> origin/csl/revert_open 2025-12-04T09:23:32.5433516Z * [new branch] csl/skip_build -> origin/csl/skip_build 2025-12-04T09:23:32.5433883Z * [new branch] csl/smaller_avx_amx_runenrs -> origin/csl/smaller_avx_amx_runenrs 2025-12-04T09:23:32.5434248Z * [new branch] csl/td_job_level -> origin/csl/td_job_level 2025-12-04T09:23:32.5434632Z * [new branch] csl/test_cuda_build_large_runner -> origin/csl/test_cuda_build_large_runner 2025-12-04T09:23:32.5435101Z * [new branch] csl/test_owners_autograd_dispatch_nn -> origin/csl/test_owners_autograd_dispatch_nn 2025-12-04T09:23:32.5435550Z * [new branch] csl/test_owners_higher_confidence -> origin/csl/test_owners_higher_confidence 2025-12-04T09:23:32.5435928Z * [new branch] csl/upload_json_running -> origin/csl/upload_json_running 2025-12-04T09:23:32.5436372Z * [new branch] csl/win_sccache -> origin/csl/win_sccache 2025-12-04T09:23:32.5436703Z * [new branch] csl/xml_stuff -> origin/csl/xml_stuff 2025-12-04T09:23:32.5437003Z * [new branch] cublasrelax2 -> origin/cublasrelax2 2025-12-04T09:23:32.5437802Z * [new branch] cuda_mempool -> origin/cuda_mempool 2025-12-04T09:23:32.5438388Z * [new branch] custom_lowering_dict -> origin/custom_lowering_dict 2025-12-04T09:23:32.5439160Z * [new branch] d4l3k/debug_plane_frtrace -> origin/d4l3k/debug_plane_frtrace 2025-12-04T09:23:32.5442926Z * [new branch] daxia6/2.8o3 -> origin/daxia6/2.8o3 2025-12-04T09:23:32.5443300Z * [new branch] debug-guard -> origin/debug-guard 2025-12-04T09:23:32.5443679Z * [new branch] delete-quant-docs -> origin/delete-quant-docs 2025-12-04T09:23:32.5445121Z * [new branch] dependabot/pip/dot-ci/docker/ci_commit_pins/main/transformers-4.57.0 -> origin/dependabot/pip/dot-ci/docker/ci_commit_pins/main/transformers-4.57.0 2025-12-04T09:23:32.5445902Z * [new branch] dependabot/pip/dot-ci/docker/ci_commit_pins/main/transformers-4.57.1 -> origin/dependabot/pip/dot-ci/docker/ci_commit_pins/main/transformers-4.57.1 2025-12-04T09:23:32.5446773Z * [new branch] desertfire/test_cpp_wrapper -> origin/desertfire/test_cpp_wrapper 2025-12-04T09:23:32.5447540Z * [new branch] desertfire/triton-cpu-for-aarch64 -> origin/desertfire/triton-cpu-for-aarch64 2025-12-04T09:23:32.5451933Z * [new branch] dev/dhruva/flex_attn_opt -> origin/dev/dhruva/flex_attn_opt 2025-12-04T09:23:32.5452367Z * [new branch] dev/joona/MPSNDArrayAdd -> origin/dev/joona/MPSNDArrayAdd 2025-12-04T09:23:32.5452907Z * [new branch] dev/joona/Unranked -> origin/dev/joona/Unranked 2025-12-04T09:23:32.5453242Z * [new branch] dev/joona/cat -> origin/dev/joona/cat 2025-12-04T09:23:32.5453746Z * [new branch] dev/joona/embeddingbag -> origin/dev/joona/embeddingbag 2025-12-04T09:23:32.5454610Z * [new branch] dev/joona/fix_sdpa_memtest -> origin/dev/joona/fix_sdpa_memtest 2025-12-04T09:23:32.5455052Z * [new branch] dev/joona/getTensorsString -> origin/dev/joona/getTensorsString 2025-12-04T09:23:32.5458229Z * [new branch] dev/joona/mps_linear_macos14 -> origin/dev/joona/mps_linear_macos14 2025-12-04T09:23:32.5458879Z * [new branch] dev/joona/scalar_clamp -> origin/dev/joona/scalar_clamp 2025-12-04T09:23:32.5462048Z * [new branch] dev/joona/sdpa -> origin/dev/joona/sdpa 2025-12-04T09:23:32.5462542Z * [new branch] dev/joona/sdpa_api -> origin/dev/joona/sdpa_api 2025-12-04T09:23:32.5468505Z * [new branch] dev/joona/type_inf -> origin/dev/joona/type_inf 2025-12-04T09:23:32.5468952Z * [new branch] dev/joona/ulpAssertClose -> origin/dev/joona/ulpAssertClose 2025-12-04T09:23:32.5469329Z * [new branch] dev/joona/upsize3d -> origin/dev/joona/upsize3d 2025-12-04T09:23:32.5469659Z * [new branch] disp_counter -> origin/disp_counter 2025-12-04T09:23:32.5469998Z * [new branch] divyanshk-patch-1 -> origin/divyanshk-patch-1 2025-12-04T09:23:32.5470329Z * [new branch] docs -> origin/docs 2025-12-04T09:23:32.5470630Z * [new branch] documentation -> origin/documentation 2025-12-04T09:23:32.5470963Z * [new branch] eager_model_benchmarks -> origin/eager_model_benchmarks 2025-12-04T09:23:32.5471363Z * [new branch] embg/test_inductor_ci_control -> origin/embg/test_inductor_ci_control 2025-12-04T09:23:32.5471756Z * [new branch] embg/triton_l2_prefetch_128B -> origin/embg/triton_l2_prefetch_128B 2025-12-04T09:23:32.5472131Z * [new branch] embg/triton_l2_prefetch_256B -> origin/embg/triton_l2_prefetch_256B 2025-12-04T09:23:32.5472473Z * [new branch] eqy-patch-1 -> origin/eqy-patch-1 2025-12-04T09:23:32.5472917Z * [new branch] eqy-patch-2 -> origin/eqy-patch-2 2025-12-04T09:23:32.5473220Z * [new branch] eqy-patch-3 -> origin/eqy-patch-3 2025-12-04T09:23:32.5473540Z * [new branch] eqy-patch-4 -> origin/eqy-patch-4 2025-12-04T09:23:32.5473850Z * [new branch] eqy-patch-5 -> origin/eqy-patch-5 2025-12-04T09:23:32.5474154Z * [new branch] eqy-patch-6 -> origin/eqy-patch-6 2025-12-04T09:23:32.5474508Z * [new branch] exclamaforte/amd-ma -> origin/exclamaforte/amd-ma 2025-12-04T09:23:32.5474961Z * [new branch] exclamaforte/combo-kernels-perf-run -> origin/exclamaforte/combo-kernels-perf-run 2025-12-04T09:23:32.5475480Z * [new branch] exclamaforte/do_bench_refactor -> origin/exclamaforte/do_bench_refactor 2025-12-04T09:23:32.5476337Z * [new branch] exclamaforte/enable-mem-dep-fusion -> origin/exclamaforte/enable-mem-dep-fusion 2025-12-04T09:23:32.5476895Z * [new branch] exclamaforte/fix-exhaustive-autotuning -> origin/exclamaforte/fix-exhaustive-autotuning 2025-12-04T09:23:32.5477766Z * [new branch] exclamaforte/fix-trace-parsing-fx-svg -> origin/exclamaforte/fix-trace-parsing-fx-svg 2025-12-04T09:23:32.5478367Z * [new branch] exclamaforte/force-pointwise-cat-perf-run -> origin/exclamaforte/force-pointwise-cat-perf-run 2025-12-04T09:23:32.5478955Z * [new branch] exclamaforte/fusion-data -> origin/exclamaforte/fusion-data 2025-12-04T09:23:32.5481946Z * [new branch] exclamaforte/gemm-benchmark-run -> origin/exclamaforte/gemm-benchmark-run 2025-12-04T09:23:32.5482471Z * [new branch] exclamaforte/gemm-export-model -> origin/exclamaforte/gemm-export-model 2025-12-04T09:23:32.5482876Z * [new branch] exclamaforte/gemm-model -> origin/exclamaforte/gemm-model 2025-12-04T09:23:32.5483342Z * [new branch] exclamaforte/gemm-model-all-data-collection -> origin/exclamaforte/gemm-model-all-data-collection 2025-12-04T09:23:32.5483806Z * [new branch] exclamaforte/gemm-to-amd -> origin/exclamaforte/gemm-to-amd 2025-12-04T09:23:32.5484210Z * [new branch] exclamaforte/just-gemm-model -> origin/exclamaforte/just-gemm-model 2025-12-04T09:23:32.5484702Z * [new branch] exclamaforte/just-gemm-model-no-refactor -> origin/exclamaforte/just-gemm-model-no-refactor 2025-12-04T09:23:32.5485267Z * [new branch] exclamaforte/profile-diff-algo -> origin/exclamaforte/profile-diff-algo 2025-12-04T09:23:32.5485967Z * [new branch] exclamaforte/profiler-visualization -> origin/exclamaforte/profiler-visualization 2025-12-04T09:23:32.5486701Z * [new branch] exclamaforte/test_cpp_wrapper_mode -> origin/exclamaforte/test_cpp_wrapper_mode 2025-12-04T09:23:32.5487464Z * [new branch] exclamaforte/update-autotune-configs -> origin/exclamaforte/update-autotune-configs 2025-12-04T09:23:32.5488174Z * [new branch] exclamaforte/update-autotune-configs-2 -> origin/exclamaforte/update-autotune-configs-2 2025-12-04T09:23:32.5488864Z * [new branch] exec -> origin/exec 2025-12-04T09:23:32.5490011Z * [new branch] experimental-mosaic -> origin/experimental-mosaic 2025-12-04T09:23:32.5490538Z * [new branch] export-D61047529 -> origin/export-D61047529 2025-12-04T09:23:32.5491430Z * [new branch] export-D71412006 -> origin/export-D71412006 2025-12-04T09:23:32.5492149Z * [new branch] export-D73042989 -> origin/export-D73042989 2025-12-04T09:23:32.5492849Z * [new branch] export-D78957093 -> origin/export-D78957093 2025-12-04T09:23:32.5493705Z * [new branch] export-D78996107 -> origin/export-D78996107 2025-12-04T09:23:32.5494359Z * [new branch] export-D80823877 -> origin/export-D80823877 2025-12-04T09:23:32.5495126Z * [new branch] export-D80958642 -> origin/export-D80958642 2025-12-04T09:23:32.5499268Z * [new branch] export-D81054193 -> origin/export-D81054193 2025-12-04T09:23:32.5499687Z * [new branch] export-D81204584 -> origin/export-D81204584 2025-12-04T09:23:32.5500003Z * [new branch] export-D81429090 -> origin/export-D81429090 2025-12-04T09:23:32.5500318Z * [new branch] export-D82250826 -> origin/export-D82250826 2025-12-04T09:23:32.5500648Z * [new branch] export-D82253817 -> origin/export-D82253817 2025-12-04T09:23:32.5500948Z * [new branch] export-D83541846 -> origin/export-D83541846 2025-12-04T09:23:32.5501262Z * [new branch] export-D83627170 -> origin/export-D83627170 2025-12-04T09:23:32.5501616Z * [new branch] export-D83766701 -> origin/export-D83766701 2025-12-04T09:23:32.5501945Z * [new branch] export-D83768878 -> origin/export-D83768878 2025-12-04T09:23:32.5503190Z * [new branch] export-D83769447 -> origin/export-D83769447 2025-12-04T09:23:32.5503834Z * [new branch] export-D84089824 -> origin/export-D84089824 2025-12-04T09:23:32.5504375Z * [new branch] export-D84213020 -> origin/export-D84213020 2025-12-04T09:23:32.5505909Z * [new branch] export-D84373821 -> origin/export-D84373821 2025-12-04T09:23:32.5508624Z * [new branch] export-D84612194 -> origin/export-D84612194 2025-12-04T09:23:32.5509116Z * [new branch] export-D84890985 -> origin/export-D84890985 2025-12-04T09:23:32.5509513Z * [new branch] export-D85122326 -> origin/export-D85122326 2025-12-04T09:23:32.5509981Z * [new branch] export-D86256198 -> origin/export-D86256198 2025-12-04T09:23:32.5510369Z * [new branch] export-D86460608 -> origin/export-D86460608 2025-12-04T09:23:32.5510926Z * [new branch] export-D86474796 -> origin/export-D86474796 2025-12-04T09:23:32.5511831Z * [new branch] export-D86712396 -> origin/export-D86712396 2025-12-04T09:23:32.5512588Z * [new branch] export-D87022129 -> origin/export-D87022129 2025-12-04T09:23:32.5513439Z * [new branch] export-D87838959 -> origin/export-D87838959 2025-12-04T09:23:32.5514363Z * [new branch] export-D88319437 -> origin/export-D88319437 2025-12-04T09:23:32.5515234Z * [new branch] exported-model-train-idempotent -> origin/exported-model-train-idempotent 2025-12-04T09:23:32.5515991Z * [new branch] ezyang-titan-october -> origin/ezyang-titan-october 2025-12-04T09:23:32.5516754Z * [new branch] ezyang-titan-october2 -> origin/ezyang-titan-october2 2025-12-04T09:23:32.5517497Z * [new branch] ezyang-war -> origin/ezyang-war 2025-12-04T09:23:32.5518792Z * [new branch] ezyang/wip-aot-descriptors -> origin/ezyang/wip-aot-descriptors 2025-12-04T09:23:32.5519187Z * [new branch] fa_u8_brgemm -> origin/fa_u8_brgemm 2025-12-04T09:23:32.5520594Z * [new branch] fadeputr/sequence_fbgemm -> origin/fadeputr/sequence_fbgemm 2025-12-04T09:23:32.5521102Z * [new branch] fastmath_baseline -> origin/fastmath_baseline 2025-12-04T09:23:32.5522383Z * [new branch] fbcode/warm -> origin/fbcode/warm 2025-12-04T09:23:32.5523058Z * [new branch] fca -> origin/fca 2025-12-04T09:23:32.5523787Z * [new branch] fca2_ca5984c -> origin/fca2_ca5984c 2025-12-04T09:23:32.5524491Z * [new branch] fca5 -> origin/fca5 2025-12-04T09:23:32.5525683Z * [new branch] feature/justknobs-cpp -> origin/feature/justknobs-cpp 2025-12-04T09:23:32.5526262Z * [new branch] feature/numa-forkserver -> origin/feature/numa-forkserver 2025-12-04T09:23:32.5527543Z * [new branch] ffast_math_baseline -> origin/ffast_math_baseline 2025-12-04T09:23:32.5528238Z * [new branch] ffast_math_target -> origin/ffast_math_target 2025-12-04T09:23:32.5529211Z * [new branch] findhao/base_commit -> origin/findhao/base_commit 2025-12-04T09:23:32.5529876Z * [new branch] findhao/base_commit1 -> origin/findhao/base_commit1 2025-12-04T09:23:32.5530452Z * [new branch] findhao/multistream2 -> origin/findhao/multistream2 2025-12-04T09:23:32.5531101Z * [new branch] findhao/multistream5 -> origin/findhao/multistream5 2025-12-04T09:23:32.5531754Z * [new branch] findhao/multistream6 -> origin/findhao/multistream6 2025-12-04T09:23:32.5532402Z * [new branch] findhao/operatorbench3 -> origin/findhao/operatorbench3 2025-12-04T09:23:32.5533035Z * [new branch] findhao/operatorbench5 -> origin/findhao/operatorbench5 2025-12-04T09:23:32.5533895Z * [new branch] findhao/tritonparse -> origin/findhao/tritonparse 2025-12-04T09:23:32.5534635Z * [new branch] fix-ck-gemm-template-format -> origin/fix-ck-gemm-template-format 2025-12-04T09:23:32.5535596Z * [new branch] fix-config-ignore -> origin/fix-config-ignore 2025-12-04T09:23:32.5535986Z * [new branch] fix-dict-guard -> origin/fix-dict-guard 2025-12-04T09:23:32.5537205Z * [new branch] fix_addmm_issue -> origin/fix_addmm_issue 2025-12-04T09:23:32.5537568Z * [new branch] fix_amd_missing_cluster_dims -> origin/fix_amd_missing_cluster_dims 2025-12-04T09:23:32.5542026Z * [new branch] fix_bench_bwd_pass -> origin/fix_bench_bwd_pass 2025-12-04T09:23:32.5542444Z * [new branch] fix_mem_profiler_config -> origin/fix_mem_profiler_config 2025-12-04T09:23:32.5542808Z * [new branch] fix_nvrtc_discovery -> origin/fix_nvrtc_discovery 2025-12-04T09:23:32.5543132Z * [new branch] fix_op_runner -> origin/fix_op_runner 2025-12-04T09:23:32.5543432Z * [new branch] fix_ubn_159469 -> origin/fix_ubn_159469 2025-12-04T09:23:32.5543758Z * [new branch] fixes-triage -> origin/fixes-triage 2025-12-04T09:23:32.5544057Z * [new branch] fixflashinfer -> origin/fixflashinfer 2025-12-04T09:23:32.5544866Z * [new branch] flash_decoding_cpu -> origin/flash_decoding_cpu 2025-12-04T09:23:32.5545251Z * [new branch] flex-flash -> origin/flex-flash 2025-12-04T09:23:32.5545650Z * [new branch] flex_attention_functorch_grad -> origin/flex_attention_functorch_grad 2025-12-04T09:23:32.5546005Z * [new branch] flex_flash -> origin/flex_flash 2025-12-04T09:23:32.5546953Z * [new branch] fmassa/fix_memeff_sharding_rule -> origin/fmassa/fix_memeff_sharding_rule 2025-12-04T09:23:32.5547553Z * [new branch] fmassa/tests_comm_compute_scheduler -> origin/fmassa/tests_comm_compute_scheduler 2025-12-04T09:23:32.5548043Z * [new branch] forkserver_fix -> origin/forkserver_fix 2025-12-04T09:23:32.5548835Z * [new branch] fsdp2_trace_rules -> origin/fsdp2_trace_rules 2025-12-04T09:23:32.5549576Z * [new branch] fx_cpp -> origin/fx_cpp 2025-12-04T09:23:32.5550761Z * [new branch] fy/fix-win -> origin/fy/fix-win 2025-12-04T09:23:32.5551487Z * [new branch] galv-patch-1 -> origin/galv-patch-1 2025-12-04T09:23:32.5554778Z * [new branch] galv/cudagraphs-conditional-nodes-4 -> origin/galv/cudagraphs-conditional-nodes-4 2025-12-04T09:23:32.5555512Z * [new branch] georgehong/cmakelists-patch -> origin/georgehong/cmakelists-patch 2025-12-04T09:23:32.5556008Z * [new branch] gh/AlnisM/1/base -> origin/gh/AlnisM/1/base 2025-12-04T09:23:32.5560681Z * [new branch] gh/AlnisM/1/head -> origin/gh/AlnisM/1/head 2025-12-04T09:23:32.5561194Z * [new branch] gh/EikanWang/67/base -> origin/gh/EikanWang/67/base 2025-12-04T09:23:32.5561739Z * [new branch] gh/EikanWang/67/head -> origin/gh/EikanWang/67/head 2025-12-04T09:23:32.5562516Z * [new branch] gh/Gasoonjia/1/base -> origin/gh/Gasoonjia/1/base 2025-12-04T09:23:32.5562924Z * [new branch] gh/Gasoonjia/1/head -> origin/gh/Gasoonjia/1/head 2025-12-04T09:23:32.5567296Z * [new branch] gh/H-Huang/131/base -> origin/gh/H-Huang/131/base 2025-12-04T09:23:32.5567843Z * [new branch] gh/H-Huang/131/head -> origin/gh/H-Huang/131/head 2025-12-04T09:23:32.5568282Z * [new branch] gh/H-Huang/131/orig -> origin/gh/H-Huang/131/orig 2025-12-04T09:23:32.5569125Z * [new branch] gh/H-Huang/132/base -> origin/gh/H-Huang/132/base 2025-12-04T09:23:32.5572234Z * [new branch] gh/H-Huang/132/head -> origin/gh/H-Huang/132/head 2025-12-04T09:23:32.5572682Z * [new branch] gh/H-Huang/132/orig -> origin/gh/H-Huang/132/orig 2025-12-04T09:23:32.5578782Z * [new branch] gh/H-Huang/180/base -> origin/gh/H-Huang/180/base 2025-12-04T09:23:32.5583233Z * [new branch] gh/H-Huang/180/head -> origin/gh/H-Huang/180/head 2025-12-04T09:23:32.5584817Z * [new branch] gh/H-Huang/180/orig -> origin/gh/H-Huang/180/orig 2025-12-04T09:23:32.5585212Z * [new branch] gh/H-Huang/182/base -> origin/gh/H-Huang/182/base 2025-12-04T09:23:32.5585535Z * [new branch] gh/H-Huang/182/head -> origin/gh/H-Huang/182/head 2025-12-04T09:23:32.5585865Z * [new branch] gh/H-Huang/182/orig -> origin/gh/H-Huang/182/orig 2025-12-04T09:23:32.5586185Z * [new branch] gh/H-Huang/226/base -> origin/gh/H-Huang/226/base 2025-12-04T09:23:32.5586534Z * [new branch] gh/H-Huang/226/head -> origin/gh/H-Huang/226/head 2025-12-04T09:23:32.5586883Z * [new branch] gh/H-Huang/226/orig -> origin/gh/H-Huang/226/orig 2025-12-04T09:23:32.5587234Z * [new branch] gh/H-Huang/228/base -> origin/gh/H-Huang/228/base 2025-12-04T09:23:32.5587587Z * [new branch] gh/H-Huang/228/head -> origin/gh/H-Huang/228/head 2025-12-04T09:23:32.5587939Z * [new branch] gh/H-Huang/228/orig -> origin/gh/H-Huang/228/orig 2025-12-04T09:23:32.5588328Z * [new branch] gh/IvanKobzarev/150/base -> origin/gh/IvanKobzarev/150/base 2025-12-04T09:23:32.5588716Z * [new branch] gh/IvanKobzarev/150/head -> origin/gh/IvanKobzarev/150/head 2025-12-04T09:23:32.5589102Z * [new branch] gh/IvanKobzarev/150/orig -> origin/gh/IvanKobzarev/150/orig 2025-12-04T09:23:32.5589485Z * [new branch] gh/IvanKobzarev/157/base -> origin/gh/IvanKobzarev/157/base 2025-12-04T09:23:32.5589858Z * [new branch] gh/IvanKobzarev/157/head -> origin/gh/IvanKobzarev/157/head 2025-12-04T09:23:32.5590254Z * [new branch] gh/IvanKobzarev/157/orig -> origin/gh/IvanKobzarev/157/orig 2025-12-04T09:23:32.5590635Z * [new branch] gh/IvanKobzarev/159/base -> origin/gh/IvanKobzarev/159/base 2025-12-04T09:23:32.5591012Z * [new branch] gh/IvanKobzarev/159/head -> origin/gh/IvanKobzarev/159/head 2025-12-04T09:23:32.5591388Z * [new branch] gh/IvanKobzarev/159/orig -> origin/gh/IvanKobzarev/159/orig 2025-12-04T09:23:32.5591768Z * [new branch] gh/IvanKobzarev/162/base -> origin/gh/IvanKobzarev/162/base 2025-12-04T09:23:32.5592148Z * [new branch] gh/IvanKobzarev/162/head -> origin/gh/IvanKobzarev/162/head 2025-12-04T09:23:32.5592569Z * [new branch] gh/IvanKobzarev/162/orig -> origin/gh/IvanKobzarev/162/orig 2025-12-04T09:23:32.5593195Z * [new branch] gh/IvanKobzarev/163/base -> origin/gh/IvanKobzarev/163/base 2025-12-04T09:23:32.5593599Z * [new branch] gh/IvanKobzarev/163/head -> origin/gh/IvanKobzarev/163/head 2025-12-04T09:23:32.5594027Z * [new branch] gh/IvanKobzarev/163/orig -> origin/gh/IvanKobzarev/163/orig 2025-12-04T09:23:32.5594413Z * [new branch] gh/IvanKobzarev/166/base -> origin/gh/IvanKobzarev/166/base 2025-12-04T09:23:32.5594817Z * [new branch] gh/IvanKobzarev/166/head -> origin/gh/IvanKobzarev/166/head 2025-12-04T09:23:32.5595207Z * [new branch] gh/IvanKobzarev/166/orig -> origin/gh/IvanKobzarev/166/orig 2025-12-04T09:23:32.5595580Z * [new branch] gh/IvanKobzarev/167/base -> origin/gh/IvanKobzarev/167/base 2025-12-04T09:23:32.5596097Z * [new branch] gh/IvanKobzarev/167/head -> origin/gh/IvanKobzarev/167/head 2025-12-04T09:23:32.5596482Z * [new branch] gh/IvanKobzarev/167/orig -> origin/gh/IvanKobzarev/167/orig 2025-12-04T09:23:32.5596870Z * [new branch] gh/IvanKobzarev/168/base -> origin/gh/IvanKobzarev/168/base 2025-12-04T09:23:32.5597325Z * [new branch] gh/IvanKobzarev/168/head -> origin/gh/IvanKobzarev/168/head 2025-12-04T09:23:32.5597691Z * [new branch] gh/IvanKobzarev/168/orig -> origin/gh/IvanKobzarev/168/orig 2025-12-04T09:23:32.5598053Z * [new branch] gh/IvanKobzarev/169/base -> origin/gh/IvanKobzarev/169/base 2025-12-04T09:23:32.5598420Z * [new branch] gh/IvanKobzarev/169/head -> origin/gh/IvanKobzarev/169/head 2025-12-04T09:23:32.5598794Z * [new branch] gh/IvanKobzarev/169/orig -> origin/gh/IvanKobzarev/169/orig 2025-12-04T09:23:32.5600865Z * [new branch] gh/IvanKobzarev/170/base -> origin/gh/IvanKobzarev/170/base 2025-12-04T09:23:32.5601244Z * [new branch] gh/IvanKobzarev/170/head -> origin/gh/IvanKobzarev/170/head 2025-12-04T09:23:32.5601624Z * [new branch] gh/IvanKobzarev/170/orig -> origin/gh/IvanKobzarev/170/orig 2025-12-04T09:23:32.5601995Z * [new branch] gh/IvanKobzarev/171/base -> origin/gh/IvanKobzarev/171/base 2025-12-04T09:23:32.5602369Z * [new branch] gh/IvanKobzarev/171/head -> origin/gh/IvanKobzarev/171/head 2025-12-04T09:23:32.5605003Z * [new branch] gh/IvanKobzarev/171/orig -> origin/gh/IvanKobzarev/171/orig 2025-12-04T09:23:32.5605362Z * [new branch] gh/IvanKobzarev/172/base -> origin/gh/IvanKobzarev/172/base 2025-12-04T09:23:32.5605910Z * [new branch] gh/IvanKobzarev/172/head -> origin/gh/IvanKobzarev/172/head 2025-12-04T09:23:32.5606272Z * [new branch] gh/IvanKobzarev/172/orig -> origin/gh/IvanKobzarev/172/orig 2025-12-04T09:23:32.5608663Z * [new branch] gh/IvanKobzarev/173/base -> origin/gh/IvanKobzarev/173/base 2025-12-04T09:23:32.5609021Z * [new branch] gh/IvanKobzarev/173/head -> origin/gh/IvanKobzarev/173/head 2025-12-04T09:23:32.5609465Z * [new branch] gh/IvanKobzarev/173/orig -> origin/gh/IvanKobzarev/173/orig 2025-12-04T09:23:32.5614015Z * [new branch] gh/IvanKobzarev/174/base -> origin/gh/IvanKobzarev/174/base 2025-12-04T09:23:32.5619258Z * [new branch] gh/IvanKobzarev/174/head -> origin/gh/IvanKobzarev/174/head 2025-12-04T09:23:32.5624554Z * [new branch] gh/IvanKobzarev/174/orig -> origin/gh/IvanKobzarev/174/orig 2025-12-04T09:23:32.5627079Z * [new branch] gh/IvanKobzarev/175/base -> origin/gh/IvanKobzarev/175/base 2025-12-04T09:23:32.5627868Z * [new branch] gh/IvanKobzarev/175/head -> origin/gh/IvanKobzarev/175/head 2025-12-04T09:23:32.5630484Z * [new branch] gh/IvanKobzarev/175/orig -> origin/gh/IvanKobzarev/175/orig 2025-12-04T09:23:32.5630947Z * [new branch] gh/IvanKobzarev/176/base -> origin/gh/IvanKobzarev/176/base 2025-12-04T09:23:32.5631319Z * [new branch] gh/IvanKobzarev/176/head -> origin/gh/IvanKobzarev/176/head 2025-12-04T09:23:32.5631679Z * [new branch] gh/IvanKobzarev/176/orig -> origin/gh/IvanKobzarev/176/orig 2025-12-04T09:23:32.5632041Z * [new branch] gh/IvanKobzarev/177/base -> origin/gh/IvanKobzarev/177/base 2025-12-04T09:23:32.5632397Z * [new branch] gh/IvanKobzarev/177/head -> origin/gh/IvanKobzarev/177/head 2025-12-04T09:23:32.5632855Z * [new branch] gh/IvanKobzarev/177/orig -> origin/gh/IvanKobzarev/177/orig 2025-12-04T09:23:32.5633202Z * [new branch] gh/IvanKobzarev/178/base -> origin/gh/IvanKobzarev/178/base 2025-12-04T09:23:32.5633580Z * [new branch] gh/IvanKobzarev/178/head -> origin/gh/IvanKobzarev/178/head 2025-12-04T09:23:32.5634210Z * [new branch] gh/IvanKobzarev/178/orig -> origin/gh/IvanKobzarev/178/orig 2025-12-04T09:23:32.5634593Z * [new branch] gh/IvanKobzarev/179/base -> origin/gh/IvanKobzarev/179/base 2025-12-04T09:23:32.5634949Z * [new branch] gh/IvanKobzarev/179/head -> origin/gh/IvanKobzarev/179/head 2025-12-04T09:23:32.5635377Z * [new branch] gh/IvanKobzarev/179/orig -> origin/gh/IvanKobzarev/179/orig 2025-12-04T09:23:32.5635734Z * [new branch] gh/IvanKobzarev/180/base -> origin/gh/IvanKobzarev/180/base 2025-12-04T09:23:32.5636077Z * [new branch] gh/IvanKobzarev/180/head -> origin/gh/IvanKobzarev/180/head 2025-12-04T09:23:32.5636438Z * [new branch] gh/IvanKobzarev/180/orig -> origin/gh/IvanKobzarev/180/orig 2025-12-04T09:23:32.5636787Z * [new branch] gh/IvanKobzarev/181/base -> origin/gh/IvanKobzarev/181/base 2025-12-04T09:23:32.5637144Z * [new branch] gh/IvanKobzarev/181/head -> origin/gh/IvanKobzarev/181/head 2025-12-04T09:23:32.5637491Z * [new branch] gh/IvanKobzarev/181/orig -> origin/gh/IvanKobzarev/181/orig 2025-12-04T09:23:32.5637841Z * [new branch] gh/IvanKobzarev/182/base -> origin/gh/IvanKobzarev/182/base 2025-12-04T09:23:32.5638198Z * [new branch] gh/IvanKobzarev/182/head -> origin/gh/IvanKobzarev/182/head 2025-12-04T09:23:32.5638570Z * [new branch] gh/IvanKobzarev/182/orig -> origin/gh/IvanKobzarev/182/orig 2025-12-04T09:23:32.5639017Z * [new branch] gh/IvanKobzarev/183/base -> origin/gh/IvanKobzarev/183/base 2025-12-04T09:23:32.5639370Z * [new branch] gh/IvanKobzarev/183/head -> origin/gh/IvanKobzarev/183/head 2025-12-04T09:23:32.5639727Z * [new branch] gh/IvanKobzarev/183/orig -> origin/gh/IvanKobzarev/183/orig 2025-12-04T09:23:32.5640067Z * [new branch] gh/IvanKobzarev/184/base -> origin/gh/IvanKobzarev/184/base 2025-12-04T09:23:32.5640425Z * [new branch] gh/IvanKobzarev/184/head -> origin/gh/IvanKobzarev/184/head 2025-12-04T09:23:32.5640774Z * [new branch] gh/IvanKobzarev/184/orig -> origin/gh/IvanKobzarev/184/orig 2025-12-04T09:23:32.5641171Z * [new branch] gh/NikhilAPatel/1/base -> origin/gh/NikhilAPatel/1/base 2025-12-04T09:23:32.5641540Z * [new branch] gh/NikhilAPatel/1/head -> origin/gh/NikhilAPatel/1/head 2025-12-04T09:23:32.5641892Z * [new branch] gh/NikhilAPatel/2/base -> origin/gh/NikhilAPatel/2/base 2025-12-04T09:23:32.5642239Z * [new branch] gh/NikhilAPatel/2/head -> origin/gh/NikhilAPatel/2/head 2025-12-04T09:23:32.5642585Z * [new branch] gh/NikhilAPatel/4/base -> origin/gh/NikhilAPatel/4/base 2025-12-04T09:23:32.5642923Z * [new branch] gh/NikhilAPatel/4/head -> origin/gh/NikhilAPatel/4/head 2025-12-04T09:23:32.5647422Z * [new branch] gh/NikhilAPatel/5/base -> origin/gh/NikhilAPatel/5/base 2025-12-04T09:23:32.5649520Z * [new branch] gh/NikhilAPatel/5/head -> origin/gh/NikhilAPatel/5/head 2025-12-04T09:23:32.5650033Z * [new branch] gh/NikhilAPatel/5/orig -> origin/gh/NikhilAPatel/5/orig 2025-12-04T09:23:32.5656197Z * [new branch] gh/PaliC/17/base -> origin/gh/PaliC/17/base 2025-12-04T09:23:32.5656609Z * [new branch] gh/PaliC/17/head -> origin/gh/PaliC/17/head 2025-12-04T09:23:32.5656939Z * [new branch] gh/PaliC/17/orig -> origin/gh/PaliC/17/orig 2025-12-04T09:23:32.5657266Z * [new branch] gh/PaliC/18/base -> origin/gh/PaliC/18/base 2025-12-04T09:23:32.5657579Z * [new branch] gh/PaliC/18/head -> origin/gh/PaliC/18/head 2025-12-04T09:23:32.5657888Z * [new branch] gh/PaliC/18/orig -> origin/gh/PaliC/18/orig 2025-12-04T09:23:32.5658392Z * [new branch] gh/PaliC/20/base -> origin/gh/PaliC/20/base 2025-12-04T09:23:32.5658702Z * [new branch] gh/PaliC/20/head -> origin/gh/PaliC/20/head 2025-12-04T09:23:32.5659012Z * [new branch] gh/PaliC/20/orig -> origin/gh/PaliC/20/orig 2025-12-04T09:23:32.5659425Z * [new branch] gh/PaliC/21/base -> origin/gh/PaliC/21/base 2025-12-04T09:23:32.5659780Z * [new branch] gh/PaliC/21/head -> origin/gh/PaliC/21/head 2025-12-04T09:23:32.5660114Z * [new branch] gh/PaliC/21/orig -> origin/gh/PaliC/21/orig 2025-12-04T09:23:32.5660423Z * [new branch] gh/PaliC/23/base -> origin/gh/PaliC/23/base 2025-12-04T09:23:32.5660734Z * [new branch] gh/PaliC/23/head -> origin/gh/PaliC/23/head 2025-12-04T09:23:32.5661038Z * [new branch] gh/PaliC/23/orig -> origin/gh/PaliC/23/orig 2025-12-04T09:23:32.5661348Z * [new branch] gh/PaliC/24/base -> origin/gh/PaliC/24/base 2025-12-04T09:23:32.5661793Z * [new branch] gh/PaliC/24/head -> origin/gh/PaliC/24/head 2025-12-04T09:23:32.5666474Z * [new branch] gh/PaliC/24/orig -> origin/gh/PaliC/24/orig 2025-12-04T09:23:32.5667014Z * [new branch] gh/PaliC/25/head -> origin/gh/PaliC/25/head 2025-12-04T09:23:32.5667375Z * [new branch] gh/PaliC/25/next -> origin/gh/PaliC/25/next 2025-12-04T09:23:32.5667812Z * [new branch] gh/PaliC/25/orig -> origin/gh/PaliC/25/orig 2025-12-04T09:23:32.5668589Z * [new branch] gh/PaliC/26/head -> origin/gh/PaliC/26/head 2025-12-04T09:23:32.5669049Z * [new branch] gh/PaliC/26/next -> origin/gh/PaliC/26/next 2025-12-04T09:23:32.5669412Z * [new branch] gh/PaliC/26/orig -> origin/gh/PaliC/26/orig 2025-12-04T09:23:32.5669772Z * [new branch] gh/PaliC/27/next -> origin/gh/PaliC/27/next 2025-12-04T09:23:32.5670143Z * [new branch] gh/PaliC/28/head -> origin/gh/PaliC/28/head 2025-12-04T09:23:32.5670486Z * [new branch] gh/PaliC/28/next -> origin/gh/PaliC/28/next 2025-12-04T09:23:32.5670851Z * [new branch] gh/PaliC/28/orig -> origin/gh/PaliC/28/orig 2025-12-04T09:23:32.5671209Z * [new branch] gh/PaliC/29/head -> origin/gh/PaliC/29/head 2025-12-04T09:23:32.5671557Z * [new branch] gh/PaliC/29/next -> origin/gh/PaliC/29/next 2025-12-04T09:23:32.5671896Z * [new branch] gh/PaliC/29/orig -> origin/gh/PaliC/29/orig 2025-12-04T09:23:32.5672249Z * [new branch] gh/PaliC/30/head -> origin/gh/PaliC/30/head 2025-12-04T09:23:32.5672606Z * [new branch] gh/PaliC/30/next -> origin/gh/PaliC/30/next 2025-12-04T09:23:32.5673133Z * [new branch] gh/PaliC/30/orig -> origin/gh/PaliC/30/orig 2025-12-04T09:23:32.5673485Z * [new branch] gh/PaliC/31/head -> origin/gh/PaliC/31/head 2025-12-04T09:23:32.5673824Z * [new branch] gh/PaliC/31/next -> origin/gh/PaliC/31/next 2025-12-04T09:23:32.5674169Z * [new branch] gh/PaliC/31/orig -> origin/gh/PaliC/31/orig 2025-12-04T09:23:32.5675325Z * [new branch] gh/PaulZhang12/25/base -> origin/gh/PaulZhang12/25/base 2025-12-04T09:23:32.5675743Z * [new branch] gh/PaulZhang12/25/head -> origin/gh/PaulZhang12/25/head 2025-12-04T09:23:32.5676155Z * [new branch] gh/PaulZhang12/25/orig -> origin/gh/PaulZhang12/25/orig 2025-12-04T09:23:32.5677719Z * [new branch] gh/PaulZhang12/28/base -> origin/gh/PaulZhang12/28/base 2025-12-04T09:23:32.5678339Z * [new branch] gh/PaulZhang12/28/head -> origin/gh/PaulZhang12/28/head 2025-12-04T09:23:32.5679015Z * [new branch] gh/PaulZhang12/28/orig -> origin/gh/PaulZhang12/28/orig 2025-12-04T09:23:32.5680392Z * [new branch] gh/PaulZhang12/31/base -> origin/gh/PaulZhang12/31/base 2025-12-04T09:23:32.5684101Z * [new branch] gh/PaulZhang12/31/head -> origin/gh/PaulZhang12/31/head 2025-12-04T09:23:32.5684511Z * [new branch] gh/PaulZhang12/31/orig -> origin/gh/PaulZhang12/31/orig 2025-12-04T09:23:32.5684851Z * [new branch] gh/PaulZhang12/37/base -> origin/gh/PaulZhang12/37/base 2025-12-04T09:23:32.5688286Z * [new branch] gh/PaulZhang12/37/head -> origin/gh/PaulZhang12/37/head 2025-12-04T09:23:32.5688634Z * [new branch] gh/PaulZhang12/37/orig -> origin/gh/PaulZhang12/37/orig 2025-12-04T09:23:32.5688963Z * [new branch] gh/PaulZhang12/40/base -> origin/gh/PaulZhang12/40/base 2025-12-04T09:23:32.5689298Z * [new branch] gh/PaulZhang12/40/head -> origin/gh/PaulZhang12/40/head 2025-12-04T09:23:32.5689637Z * [new branch] gh/PaulZhang12/40/orig -> origin/gh/PaulZhang12/40/orig 2025-12-04T09:23:32.5689978Z * [new branch] gh/PaulZhang12/42/base -> origin/gh/PaulZhang12/42/base 2025-12-04T09:23:32.5692597Z * [new branch] gh/PaulZhang12/42/head -> origin/gh/PaulZhang12/42/head 2025-12-04T09:23:32.5692953Z * [new branch] gh/PaulZhang12/43/base -> origin/gh/PaulZhang12/43/base 2025-12-04T09:23:32.5693296Z * [new branch] gh/PaulZhang12/43/head -> origin/gh/PaulZhang12/43/head 2025-12-04T09:23:32.5693653Z * [new branch] gh/PaulZhang12/43/orig -> origin/gh/PaulZhang12/43/orig 2025-12-04T09:23:32.5693997Z * [new branch] gh/PaulZhang12/44/base -> origin/gh/PaulZhang12/44/base 2025-12-04T09:23:32.5694337Z * [new branch] gh/PaulZhang12/44/head -> origin/gh/PaulZhang12/44/head 2025-12-04T09:23:32.5694690Z * [new branch] gh/PaulZhang12/45/base -> origin/gh/PaulZhang12/45/base 2025-12-04T09:23:32.5695033Z * [new branch] gh/PaulZhang12/45/head -> origin/gh/PaulZhang12/45/head 2025-12-04T09:23:32.5698666Z * [new branch] gh/PaulZhang12/45/orig -> origin/gh/PaulZhang12/45/orig 2025-12-04T09:23:32.5699016Z * [new branch] gh/PaulZhang12/46/base -> origin/gh/PaulZhang12/46/base 2025-12-04T09:23:32.5699362Z * [new branch] gh/PaulZhang12/46/head -> origin/gh/PaulZhang12/46/head 2025-12-04T09:23:32.5699691Z * [new branch] gh/PaulZhang12/46/orig -> origin/gh/PaulZhang12/46/orig 2025-12-04T09:23:32.5700036Z * [new branch] gh/PaulZhang12/47/base -> origin/gh/PaulZhang12/47/base 2025-12-04T09:23:32.5700378Z * [new branch] gh/PaulZhang12/47/head -> origin/gh/PaulZhang12/47/head 2025-12-04T09:23:32.5702567Z * [new branch] gh/PaulZhang12/47/orig -> origin/gh/PaulZhang12/47/orig 2025-12-04T09:23:32.5702913Z * [new branch] gh/PaulZhang12/48/base -> origin/gh/PaulZhang12/48/base 2025-12-04T09:23:32.5703255Z * [new branch] gh/PaulZhang12/48/head -> origin/gh/PaulZhang12/48/head 2025-12-04T09:23:32.5703589Z * [new branch] gh/PaulZhang12/48/orig -> origin/gh/PaulZhang12/48/orig 2025-12-04T09:23:32.5703930Z * [new branch] gh/SamGinzburg/11/base -> origin/gh/SamGinzburg/11/base 2025-12-04T09:23:32.5704260Z * [new branch] gh/SamGinzburg/11/head -> origin/gh/SamGinzburg/11/head 2025-12-04T09:23:32.5707776Z * [new branch] gh/SherlockNoMad/1/base -> origin/gh/SherlockNoMad/1/base 2025-12-04T09:23:32.5708173Z * [new branch] gh/SherlockNoMad/1/head -> origin/gh/SherlockNoMad/1/head 2025-12-04T09:23:32.5708571Z * [new branch] gh/SherlockNoMad/10/base -> origin/gh/SherlockNoMad/10/base 2025-12-04T09:23:32.5708959Z * [new branch] gh/SherlockNoMad/10/head -> origin/gh/SherlockNoMad/10/head 2025-12-04T09:23:32.5709480Z * [new branch] gh/SherlockNoMad/10/orig -> origin/gh/SherlockNoMad/10/orig 2025-12-04T09:23:32.5709859Z * [new branch] gh/SherlockNoMad/11/base -> origin/gh/SherlockNoMad/11/base 2025-12-04T09:23:32.5710235Z * [new branch] gh/SherlockNoMad/11/head -> origin/gh/SherlockNoMad/11/head 2025-12-04T09:23:32.5710699Z * [new branch] gh/SherlockNoMad/11/orig -> origin/gh/SherlockNoMad/11/orig 2025-12-04T09:23:32.5711105Z * [new branch] gh/SherlockNoMad/12/base -> origin/gh/SherlockNoMad/12/base 2025-12-04T09:23:32.5711522Z * [new branch] gh/SherlockNoMad/12/head -> origin/gh/SherlockNoMad/12/head 2025-12-04T09:23:32.5711901Z * [new branch] gh/SherlockNoMad/12/orig -> origin/gh/SherlockNoMad/12/orig 2025-12-04T09:23:32.5712371Z * [new branch] gh/SherlockNoMad/15/base -> origin/gh/SherlockNoMad/15/base 2025-12-04T09:23:32.5712859Z * [new branch] gh/SherlockNoMad/15/head -> origin/gh/SherlockNoMad/15/head 2025-12-04T09:23:32.5713281Z * [new branch] gh/SherlockNoMad/15/orig -> origin/gh/SherlockNoMad/15/orig 2025-12-04T09:23:32.5714985Z * [new branch] gh/SherlockNoMad/17/base -> origin/gh/SherlockNoMad/17/base 2025-12-04T09:23:32.5715519Z * [new branch] gh/SherlockNoMad/17/head -> origin/gh/SherlockNoMad/17/head 2025-12-04T09:23:32.5716017Z * [new branch] gh/SherlockNoMad/17/orig -> origin/gh/SherlockNoMad/17/orig 2025-12-04T09:23:32.5716714Z * [new branch] gh/SherlockNoMad/18/base -> origin/gh/SherlockNoMad/18/base 2025-12-04T09:23:32.5717713Z * [new branch] gh/SherlockNoMad/18/head -> origin/gh/SherlockNoMad/18/head 2025-12-04T09:23:32.5718161Z * [new branch] gh/SherlockNoMad/18/orig -> origin/gh/SherlockNoMad/18/orig 2025-12-04T09:23:32.5720531Z * [new branch] gh/SherlockNoMad/19/base -> origin/gh/SherlockNoMad/19/base 2025-12-04T09:23:32.5721174Z * [new branch] gh/SherlockNoMad/19/head -> origin/gh/SherlockNoMad/19/head 2025-12-04T09:23:32.5721680Z * [new branch] gh/SherlockNoMad/19/orig -> origin/gh/SherlockNoMad/19/orig 2025-12-04T09:23:32.5722201Z * [new branch] gh/SherlockNoMad/2/base -> origin/gh/SherlockNoMad/2/base 2025-12-04T09:23:32.5722611Z * [new branch] gh/SherlockNoMad/2/head -> origin/gh/SherlockNoMad/2/head 2025-12-04T09:23:32.5723006Z * [new branch] gh/SherlockNoMad/20/base -> origin/gh/SherlockNoMad/20/base 2025-12-04T09:23:32.5723580Z * [new branch] gh/SherlockNoMad/20/head -> origin/gh/SherlockNoMad/20/head 2025-12-04T09:23:32.5724227Z * [new branch] gh/SherlockNoMad/20/orig -> origin/gh/SherlockNoMad/20/orig 2025-12-04T09:23:32.5728065Z * [new branch] gh/SherlockNoMad/21/base -> origin/gh/SherlockNoMad/21/base 2025-12-04T09:23:32.5728653Z * [new branch] gh/SherlockNoMad/21/head -> origin/gh/SherlockNoMad/21/head 2025-12-04T09:23:32.5729560Z * [new branch] gh/SherlockNoMad/21/orig -> origin/gh/SherlockNoMad/21/orig 2025-12-04T09:23:32.5730079Z * [new branch] gh/SherlockNoMad/3/base -> origin/gh/SherlockNoMad/3/base 2025-12-04T09:23:32.5730525Z * [new branch] gh/SherlockNoMad/3/head -> origin/gh/SherlockNoMad/3/head 2025-12-04T09:23:32.5730923Z * [new branch] gh/SherlockNoMad/4/base -> origin/gh/SherlockNoMad/4/base 2025-12-04T09:23:32.5731325Z * [new branch] gh/SherlockNoMad/4/head -> origin/gh/SherlockNoMad/4/head 2025-12-04T09:23:32.5731714Z * [new branch] gh/SherlockNoMad/5/base -> origin/gh/SherlockNoMad/5/base 2025-12-04T09:23:32.5732111Z * [new branch] gh/SherlockNoMad/5/head -> origin/gh/SherlockNoMad/5/head 2025-12-04T09:23:32.5733868Z * [new branch] gh/Sidharth123-cpu/24/base -> origin/gh/Sidharth123-cpu/24/base 2025-12-04T09:23:32.5734455Z * [new branch] gh/Sidharth123-cpu/25/base -> origin/gh/Sidharth123-cpu/25/base 2025-12-04T09:23:32.5734853Z * [new branch] gh/Sidharth123-cpu/26/base -> origin/gh/Sidharth123-cpu/26/base 2025-12-04T09:23:32.5736835Z * [new branch] gh/Sidharth123-cpu/27/base -> origin/gh/Sidharth123-cpu/27/base 2025-12-04T09:23:32.5737480Z * [new branch] gh/StrongerXi/1/base -> origin/gh/StrongerXi/1/base 2025-12-04T09:23:32.5737853Z * [new branch] gh/StrongerXi/1/head -> origin/gh/StrongerXi/1/head 2025-12-04T09:23:32.5738978Z * [new branch] gh/StrongerXi/71/base -> origin/gh/StrongerXi/71/base 2025-12-04T09:23:32.5739758Z * [new branch] gh/StrongerXi/71/head -> origin/gh/StrongerXi/71/head 2025-12-04T09:23:32.5740678Z * [new branch] gh/StrongerXi/72/base -> origin/gh/StrongerXi/72/base 2025-12-04T09:23:32.5741254Z * [new branch] gh/StrongerXi/72/head -> origin/gh/StrongerXi/72/head 2025-12-04T09:23:32.5742741Z * [new branch] gh/StrongerXi/73/base -> origin/gh/StrongerXi/73/base 2025-12-04T09:23:32.5743092Z * [new branch] gh/StrongerXi/73/head -> origin/gh/StrongerXi/73/head 2025-12-04T09:23:32.5743661Z * [new branch] gh/StrongerXi/73/orig -> origin/gh/StrongerXi/73/orig 2025-12-04T09:23:32.5748059Z * [new branch] gh/XilunWu/160/base -> origin/gh/XilunWu/160/base 2025-12-04T09:23:32.5748443Z * [new branch] gh/XilunWu/160/head -> origin/gh/XilunWu/160/head 2025-12-04T09:23:32.5748777Z * [new branch] gh/XilunWu/160/orig -> origin/gh/XilunWu/160/orig 2025-12-04T09:23:32.5749102Z * [new branch] gh/XilunWu/163/base -> origin/gh/XilunWu/163/base 2025-12-04T09:23:32.5749427Z * [new branch] gh/XilunWu/163/head -> origin/gh/XilunWu/163/head 2025-12-04T09:23:32.5749763Z * [new branch] gh/XilunWu/163/orig -> origin/gh/XilunWu/163/orig 2025-12-04T09:23:32.5750085Z * [new branch] gh/XilunWu/168/base -> origin/gh/XilunWu/168/base 2025-12-04T09:23:32.5750436Z * [new branch] gh/XilunWu/168/head -> origin/gh/XilunWu/168/head 2025-12-04T09:23:32.5751222Z * [new branch] gh/XilunWu/168/orig -> origin/gh/XilunWu/168/orig 2025-12-04T09:23:32.5752147Z * [new branch] gh/XilunWu/169/base -> origin/gh/XilunWu/169/base 2025-12-04T09:23:32.5752897Z * [new branch] gh/XilunWu/169/head -> origin/gh/XilunWu/169/head 2025-12-04T09:23:32.5753601Z * [new branch] gh/XilunWu/169/orig -> origin/gh/XilunWu/169/orig 2025-12-04T09:23:32.5755044Z * [new branch] gh/XilunWu/170/base -> origin/gh/XilunWu/170/base 2025-12-04T09:23:32.5755381Z * [new branch] gh/XilunWu/170/head -> origin/gh/XilunWu/170/head 2025-12-04T09:23:32.5755937Z * [new branch] gh/XilunWu/170/orig -> origin/gh/XilunWu/170/orig 2025-12-04T09:23:32.5757119Z * [new branch] gh/XilunWu/171/base -> origin/gh/XilunWu/171/base 2025-12-04T09:23:32.5757665Z * [new branch] gh/XilunWu/171/head -> origin/gh/XilunWu/171/head 2025-12-04T09:23:32.5758356Z * [new branch] gh/XilunWu/171/orig -> origin/gh/XilunWu/171/orig 2025-12-04T09:23:32.5759161Z * [new branch] gh/XilunWu/173/base -> origin/gh/XilunWu/173/base 2025-12-04T09:23:32.5759890Z * [new branch] gh/XilunWu/173/head -> origin/gh/XilunWu/173/head 2025-12-04T09:23:32.5760630Z * [new branch] gh/XilunWu/173/orig -> origin/gh/XilunWu/173/orig 2025-12-04T09:23:32.5761767Z * [new branch] gh/XilunWu/175/base -> origin/gh/XilunWu/175/base 2025-12-04T09:23:32.5762259Z * [new branch] gh/XilunWu/175/head -> origin/gh/XilunWu/175/head 2025-12-04T09:23:32.5762953Z * [new branch] gh/XilunWu/175/orig -> origin/gh/XilunWu/175/orig 2025-12-04T09:23:32.5764142Z * [new branch] gh/XilunWu/176/base -> origin/gh/XilunWu/176/base 2025-12-04T09:23:32.5764642Z * [new branch] gh/XilunWu/176/head -> origin/gh/XilunWu/176/head 2025-12-04T09:23:32.5765463Z * [new branch] gh/XilunWu/176/orig -> origin/gh/XilunWu/176/orig 2025-12-04T09:23:32.5766928Z * [new branch] gh/XuehaiPan/14/base -> origin/gh/XuehaiPan/14/base 2025-12-04T09:23:32.5769237Z * [new branch] gh/XuehaiPan/14/head -> origin/gh/XuehaiPan/14/head 2025-12-04T09:23:32.5769577Z * [new branch] gh/XuehaiPan/14/orig -> origin/gh/XuehaiPan/14/orig 2025-12-04T09:23:32.5769913Z * [new branch] gh/XuehaiPan/179/base -> origin/gh/XuehaiPan/179/base 2025-12-04T09:23:32.5771135Z * [new branch] gh/XuehaiPan/179/head -> origin/gh/XuehaiPan/179/head 2025-12-04T09:23:32.5771491Z * [new branch] gh/XuehaiPan/179/orig -> origin/gh/XuehaiPan/179/orig 2025-12-04T09:23:32.5771855Z * [new branch] gh/XuehaiPan/249/base -> origin/gh/XuehaiPan/249/base 2025-12-04T09:23:32.5772277Z * [new branch] gh/XuehaiPan/249/head -> origin/gh/XuehaiPan/249/head 2025-12-04T09:23:32.5773323Z * [new branch] gh/XuehaiPan/249/orig -> origin/gh/XuehaiPan/249/orig 2025-12-04T09:23:32.5774097Z * [new branch] gh/XuehaiPan/253/base -> origin/gh/XuehaiPan/253/base 2025-12-04T09:23:32.5774847Z * [new branch] gh/XuehaiPan/253/head -> origin/gh/XuehaiPan/253/head 2025-12-04T09:23:32.5775498Z * [new branch] gh/XuehaiPan/253/orig -> origin/gh/XuehaiPan/253/orig 2025-12-04T09:23:32.5776698Z * [new branch] gh/XuehaiPan/254/base -> origin/gh/XuehaiPan/254/base 2025-12-04T09:23:32.5777163Z * [new branch] gh/XuehaiPan/254/head -> origin/gh/XuehaiPan/254/head 2025-12-04T09:23:32.5777820Z * [new branch] gh/XuehaiPan/254/orig -> origin/gh/XuehaiPan/254/orig 2025-12-04T09:23:32.5778843Z * [new branch] gh/XuehaiPan/255/base -> origin/gh/XuehaiPan/255/base 2025-12-04T09:23:32.5779305Z * [new branch] gh/XuehaiPan/255/head -> origin/gh/XuehaiPan/255/head 2025-12-04T09:23:32.5780117Z * [new branch] gh/XuehaiPan/255/orig -> origin/gh/XuehaiPan/255/orig 2025-12-04T09:23:32.5781123Z * [new branch] gh/XuehaiPan/271/base -> origin/gh/XuehaiPan/271/base 2025-12-04T09:23:32.5781579Z * [new branch] gh/XuehaiPan/271/head -> origin/gh/XuehaiPan/271/head 2025-12-04T09:23:32.5782310Z * [new branch] gh/XuehaiPan/271/orig -> origin/gh/XuehaiPan/271/orig 2025-12-04T09:23:32.5783482Z * [new branch] gh/XuehaiPan/343/base -> origin/gh/XuehaiPan/343/base 2025-12-04T09:23:32.5783925Z * [new branch] gh/XuehaiPan/343/head -> origin/gh/XuehaiPan/343/head 2025-12-04T09:23:32.5784641Z * [new branch] gh/XuehaiPan/343/orig -> origin/gh/XuehaiPan/343/orig 2025-12-04T09:23:32.5785786Z * [new branch] gh/XuehaiPan/347/base -> origin/gh/XuehaiPan/347/base 2025-12-04T09:23:32.5786297Z * [new branch] gh/XuehaiPan/347/head -> origin/gh/XuehaiPan/347/head 2025-12-04T09:23:32.5786994Z * [new branch] gh/XuehaiPan/347/orig -> origin/gh/XuehaiPan/347/orig 2025-12-04T09:23:32.5788091Z * [new branch] gh/XuehaiPan/348/base -> origin/gh/XuehaiPan/348/base 2025-12-04T09:23:32.5788502Z * [new branch] gh/XuehaiPan/348/head -> origin/gh/XuehaiPan/348/head 2025-12-04T09:23:32.5789574Z * [new branch] gh/XuehaiPan/348/orig -> origin/gh/XuehaiPan/348/orig 2025-12-04T09:23:32.5790337Z * [new branch] gh/XuehaiPan/350/base -> origin/gh/XuehaiPan/350/base 2025-12-04T09:23:32.5791027Z * [new branch] gh/XuehaiPan/350/head -> origin/gh/XuehaiPan/350/head 2025-12-04T09:23:32.5791703Z * [new branch] gh/XuehaiPan/350/orig -> origin/gh/XuehaiPan/350/orig 2025-12-04T09:23:32.5792954Z * [new branch] gh/XuehaiPan/365/base -> origin/gh/XuehaiPan/365/base 2025-12-04T09:23:32.5793483Z * [new branch] gh/XuehaiPan/365/head -> origin/gh/XuehaiPan/365/head 2025-12-04T09:23:32.5794208Z * [new branch] gh/XuehaiPan/365/orig -> origin/gh/XuehaiPan/365/orig 2025-12-04T09:23:32.5795365Z * [new branch] gh/XuehaiPan/366/base -> origin/gh/XuehaiPan/366/base 2025-12-04T09:23:32.5795842Z * [new branch] gh/XuehaiPan/366/head -> origin/gh/XuehaiPan/366/head 2025-12-04T09:23:32.5796963Z * [new branch] gh/XuehaiPan/370/base -> origin/gh/XuehaiPan/370/base 2025-12-04T09:23:32.5797474Z * [new branch] gh/XuehaiPan/370/head -> origin/gh/XuehaiPan/370/head 2025-12-04T09:23:32.5798214Z * [new branch] gh/XuehaiPan/370/orig -> origin/gh/XuehaiPan/370/orig 2025-12-04T09:23:32.5799382Z * [new branch] gh/XuehaiPan/390/base -> origin/gh/XuehaiPan/390/base 2025-12-04T09:23:32.5799925Z * [new branch] gh/XuehaiPan/390/head -> origin/gh/XuehaiPan/390/head 2025-12-04T09:23:32.5801071Z * [new branch] gh/XuehaiPan/390/orig -> origin/gh/XuehaiPan/390/orig 2025-12-04T09:23:32.5802064Z * [new branch] gh/XuehaiPan/391/base -> origin/gh/XuehaiPan/391/base 2025-12-04T09:23:32.5802542Z * [new branch] gh/XuehaiPan/391/head -> origin/gh/XuehaiPan/391/head 2025-12-04T09:23:32.5803243Z * [new branch] gh/XuehaiPan/391/orig -> origin/gh/XuehaiPan/391/orig 2025-12-04T09:23:32.5804329Z * [new branch] gh/XuehaiPan/392/base -> origin/gh/XuehaiPan/392/base 2025-12-04T09:23:32.5804824Z * [new branch] gh/XuehaiPan/392/head -> origin/gh/XuehaiPan/392/head 2025-12-04T09:23:32.5805527Z * [new branch] gh/XuehaiPan/392/orig -> origin/gh/XuehaiPan/392/orig 2025-12-04T09:23:32.5807236Z * [new branch] gh/XuehaiPan/394/base -> origin/gh/XuehaiPan/394/base 2025-12-04T09:23:32.5807773Z * [new branch] gh/XuehaiPan/394/head -> origin/gh/XuehaiPan/394/head 2025-12-04T09:23:32.5808478Z * [new branch] gh/XuehaiPan/394/orig -> origin/gh/XuehaiPan/394/orig 2025-12-04T09:23:32.5809595Z * [new branch] gh/XuehaiPan/397/base -> origin/gh/XuehaiPan/397/base 2025-12-04T09:23:32.5810105Z * [new branch] gh/XuehaiPan/397/head -> origin/gh/XuehaiPan/397/head 2025-12-04T09:23:32.5810886Z * [new branch] gh/XuehaiPan/397/orig -> origin/gh/XuehaiPan/397/orig 2025-12-04T09:23:32.5811709Z * [new branch] gh/XuehaiPan/398/base -> origin/gh/XuehaiPan/398/base 2025-12-04T09:23:32.5812371Z * [new branch] gh/XuehaiPan/398/head -> origin/gh/XuehaiPan/398/head 2025-12-04T09:23:32.5813011Z * [new branch] gh/XuehaiPan/398/orig -> origin/gh/XuehaiPan/398/orig 2025-12-04T09:23:32.5814111Z * [new branch] gh/XuehaiPan/399/base -> origin/gh/XuehaiPan/399/base 2025-12-04T09:23:32.5814718Z * [new branch] gh/XuehaiPan/399/head -> origin/gh/XuehaiPan/399/head 2025-12-04T09:23:32.5815404Z * [new branch] gh/XuehaiPan/399/orig -> origin/gh/XuehaiPan/399/orig 2025-12-04T09:23:32.5816512Z * [new branch] gh/XuehaiPan/400/base -> origin/gh/XuehaiPan/400/base 2025-12-04T09:23:32.5817031Z * [new branch] gh/XuehaiPan/400/head -> origin/gh/XuehaiPan/400/head 2025-12-04T09:23:32.5817706Z * [new branch] gh/XuehaiPan/400/orig -> origin/gh/XuehaiPan/400/orig 2025-12-04T09:23:32.5819046Z * [new branch] gh/ZhiweiYan-96/39/base -> origin/gh/ZhiweiYan-96/39/base 2025-12-04T09:23:32.5819493Z * [new branch] gh/ZhiweiYan-96/39/head -> origin/gh/ZhiweiYan-96/39/head 2025-12-04T09:23:32.5820250Z * [new branch] gh/ZhiweiYan-96/39/orig -> origin/gh/ZhiweiYan-96/39/orig 2025-12-04T09:23:32.5821572Z * [new branch] gh/ZhiweiYan-96/44/base -> origin/gh/ZhiweiYan-96/44/base 2025-12-04T09:23:32.5821919Z * [new branch] gh/ZhiweiYan-96/44/head -> origin/gh/ZhiweiYan-96/44/head 2025-12-04T09:23:32.5823078Z * [new branch] gh/ZhiweiYan-96/45/base -> origin/gh/ZhiweiYan-96/45/base 2025-12-04T09:23:32.5823421Z * [new branch] gh/ZhiweiYan-96/45/head -> origin/gh/ZhiweiYan-96/45/head 2025-12-04T09:23:32.5824581Z * [new branch] gh/ZhiweiYan-96/49/base -> origin/gh/ZhiweiYan-96/49/base 2025-12-04T09:23:32.5825119Z * [new branch] gh/ZhiweiYan-96/49/head -> origin/gh/ZhiweiYan-96/49/head 2025-12-04T09:23:32.5826158Z * [new branch] gh/ZhiweiYan-96/62/base -> origin/gh/ZhiweiYan-96/62/base 2025-12-04T09:23:32.5826629Z * [new branch] gh/ZhiweiYan-96/62/head -> origin/gh/ZhiweiYan-96/62/head 2025-12-04T09:23:32.5827795Z * [new branch] gh/ZhiweiYan-96/66/base -> origin/gh/ZhiweiYan-96/66/base 2025-12-04T09:23:32.5828290Z * [new branch] gh/ZhiweiYan-96/66/head -> origin/gh/ZhiweiYan-96/66/head 2025-12-04T09:23:32.5829337Z * [new branch] gh/ZhiweiYan-96/67/base -> origin/gh/ZhiweiYan-96/67/base 2025-12-04T09:23:32.5829906Z * [new branch] gh/ZhiweiYan-96/67/head -> origin/gh/ZhiweiYan-96/67/head 2025-12-04T09:23:32.5830695Z * [new branch] gh/ZhiweiYan-96/68/base -> origin/gh/ZhiweiYan-96/68/base 2025-12-04T09:23:32.5831296Z * [new branch] gh/ZhiweiYan-96/68/head -> origin/gh/ZhiweiYan-96/68/head 2025-12-04T09:23:32.5832001Z * [new branch] gh/ZhiweiYan-96/68/orig -> origin/gh/ZhiweiYan-96/68/orig 2025-12-04T09:23:32.5837923Z * [new branch] gh/aakhundov/1/base -> origin/gh/aakhundov/1/base 2025-12-04T09:23:32.5838388Z * [new branch] gh/aakhundov/1/head -> origin/gh/aakhundov/1/head 2025-12-04T09:23:32.5838786Z * [new branch] gh/aakhundov/2/base -> origin/gh/aakhundov/2/base 2025-12-04T09:23:32.5839155Z * [new branch] gh/aakhundov/2/head -> origin/gh/aakhundov/2/head 2025-12-04T09:23:32.5839519Z * [new branch] gh/aditew01/openblas -> origin/gh/aditew01/openblas 2025-12-04T09:23:32.5839872Z * [new branch] gh/aditew01/sbgemm -> origin/gh/aditew01/sbgemm 2025-12-04T09:23:32.5840244Z * [new branch] gh/aditew01/vecbf16 -> origin/gh/aditew01/vecbf16 2025-12-04T09:23:32.5840598Z * [new branch] gh/albanD/4/base -> origin/gh/albanD/4/base 2025-12-04T09:23:32.5840982Z * [new branch] gh/albanD/4/head -> origin/gh/albanD/4/head 2025-12-04T09:23:32.5843474Z * [new branch] gh/albanD/4/orig -> origin/gh/albanD/4/orig 2025-12-04T09:23:32.5843953Z * [new branch] gh/alexbrauckmann/paddedtensor_faketensor_init -> origin/gh/alexbrauckmann/paddedtensor_faketensor_init 2025-12-04T09:23:32.5844426Z * [new branch] gh/alexsamardzic/12/base -> origin/gh/alexsamardzic/12/base 2025-12-04T09:23:32.5844777Z * [new branch] gh/alexsamardzic/12/head -> origin/gh/alexsamardzic/12/head 2025-12-04T09:23:32.5847248Z * [new branch] gh/alexsamardzic/12/orig -> origin/gh/alexsamardzic/12/orig 2025-12-04T09:23:32.5847608Z * [new branch] gh/alexsamardzic/14/base -> origin/gh/alexsamardzic/14/base 2025-12-04T09:23:32.5847954Z * [new branch] gh/alexsamardzic/14/head -> origin/gh/alexsamardzic/14/head 2025-12-04T09:23:32.5848474Z * [new branch] gh/alexsamardzic/14/orig -> origin/gh/alexsamardzic/14/orig 2025-12-04T09:23:32.5848807Z * [new branch] gh/alexsamardzic/15/base -> origin/gh/alexsamardzic/15/base 2025-12-04T09:23:32.5851851Z * [new branch] gh/alexsamardzic/15/head -> origin/gh/alexsamardzic/15/head 2025-12-04T09:23:32.5852485Z * [new branch] gh/alexsamardzic/15/orig -> origin/gh/alexsamardzic/15/orig 2025-12-04T09:23:32.5852836Z * [new branch] gh/amjames/18/base -> origin/gh/amjames/18/base 2025-12-04T09:23:32.5853149Z * [new branch] gh/amjames/18/head -> origin/gh/amjames/18/head 2025-12-04T09:23:32.5853466Z * [new branch] gh/amjames/18/orig -> origin/gh/amjames/18/orig 2025-12-04T09:23:32.5853801Z * [new branch] gh/andrewor14/35/base -> origin/gh/andrewor14/35/base 2025-12-04T09:23:32.5854147Z * [new branch] gh/andrewor14/35/head -> origin/gh/andrewor14/35/head 2025-12-04T09:23:32.5854475Z * [new branch] gh/andrewor14/35/orig -> origin/gh/andrewor14/35/orig 2025-12-04T09:23:32.5854876Z * [new branch] gh/andrewor14/50/base -> origin/gh/andrewor14/50/base 2025-12-04T09:23:32.5855400Z * [new branch] gh/andrewor14/50/head -> origin/gh/andrewor14/50/head 2025-12-04T09:23:32.5856871Z * [new branch] gh/andrewor14/50/orig -> origin/gh/andrewor14/50/orig 2025-12-04T09:23:32.5857372Z * [new branch] gh/andyanwang/30/base -> origin/gh/andyanwang/30/base 2025-12-04T09:23:32.5858394Z * [new branch] gh/andyanwang/30/orig -> origin/gh/andyanwang/30/orig 2025-12-04T09:23:32.5859222Z * [new branch] gh/andyanwang/31/base -> origin/gh/andyanwang/31/base 2025-12-04T09:23:32.5860066Z * [new branch] gh/andyanwang/31/orig -> origin/gh/andyanwang/31/orig 2025-12-04T09:23:32.5861607Z * [new branch] gh/andyanwang/39/base -> origin/gh/andyanwang/39/base 2025-12-04T09:23:32.5862032Z * [new branch] gh/andyanwang/39/head -> origin/gh/andyanwang/39/head 2025-12-04T09:23:32.5862443Z * [new branch] gh/andyanwang/39/orig -> origin/gh/andyanwang/39/orig 2025-12-04T09:23:32.5866725Z * [new branch] gh/andyanwang/42/base -> origin/gh/andyanwang/42/base 2025-12-04T09:23:32.5867183Z * [new branch] gh/andyanwang/42/head -> origin/gh/andyanwang/42/head 2025-12-04T09:23:32.5867518Z * [new branch] gh/andyanwang/42/orig -> origin/gh/andyanwang/42/orig 2025-12-04T09:23:32.5867878Z * [new branch] gh/andyanwang/45/base -> origin/gh/andyanwang/45/base 2025-12-04T09:23:32.5868208Z * [new branch] gh/andyanwang/45/head -> origin/gh/andyanwang/45/head 2025-12-04T09:23:32.5869126Z * [new branch] gh/andyanwang/45/orig -> origin/gh/andyanwang/45/orig 2025-12-04T09:23:32.5869526Z * [new branch] gh/angelayi/107/base -> origin/gh/angelayi/107/base 2025-12-04T09:23:32.5869857Z * [new branch] gh/angelayi/107/head -> origin/gh/angelayi/107/head 2025-12-04T09:23:32.5870491Z * [new branch] gh/angelayi/114/base -> origin/gh/angelayi/114/base 2025-12-04T09:23:32.5871218Z * [new branch] gh/angelayi/114/head -> origin/gh/angelayi/114/head 2025-12-04T09:23:32.5871892Z * [new branch] gh/angelayi/114/orig -> origin/gh/angelayi/114/orig 2025-12-04T09:23:32.5873077Z * [new branch] gh/angelayi/116/base -> origin/gh/angelayi/116/base 2025-12-04T09:23:32.5873684Z * [new branch] gh/angelayi/116/head -> origin/gh/angelayi/116/head 2025-12-04T09:23:32.5874397Z * [new branch] gh/angelayi/116/orig -> origin/gh/angelayi/116/orig 2025-12-04T09:23:32.5879950Z * [new branch] gh/angelayi/122/base -> origin/gh/angelayi/122/base 2025-12-04T09:23:32.5880513Z * [new branch] gh/angelayi/122/head -> origin/gh/angelayi/122/head 2025-12-04T09:23:32.5880844Z * [new branch] gh/angelayi/122/orig -> origin/gh/angelayi/122/orig 2025-12-04T09:23:32.5881162Z * [new branch] gh/angelayi/124/base -> origin/gh/angelayi/124/base 2025-12-04T09:23:32.5881575Z * [new branch] gh/angelayi/124/head -> origin/gh/angelayi/124/head 2025-12-04T09:23:32.5881907Z * [new branch] gh/angelayi/124/orig -> origin/gh/angelayi/124/orig 2025-12-04T09:23:32.5886917Z * [new branch] gh/angelayi/128/base -> origin/gh/angelayi/128/base 2025-12-04T09:23:32.5887308Z * [new branch] gh/angelayi/128/head -> origin/gh/angelayi/128/head 2025-12-04T09:23:32.5887644Z * [new branch] gh/angelayi/128/orig -> origin/gh/angelayi/128/orig 2025-12-04T09:23:32.5887972Z * [new branch] gh/angelayi/131/base -> origin/gh/angelayi/131/base 2025-12-04T09:23:32.5888303Z * [new branch] gh/angelayi/131/head -> origin/gh/angelayi/131/head 2025-12-04T09:23:32.5888620Z * [new branch] gh/angelayi/131/orig -> origin/gh/angelayi/131/orig 2025-12-04T09:23:32.5888926Z * [new branch] gh/angelayi/132/base -> origin/gh/angelayi/132/base 2025-12-04T09:23:32.5889248Z * [new branch] gh/angelayi/132/head -> origin/gh/angelayi/132/head 2025-12-04T09:23:32.5889562Z * [new branch] gh/angelayi/132/orig -> origin/gh/angelayi/132/orig 2025-12-04T09:23:32.5889875Z * [new branch] gh/angelayi/133/base -> origin/gh/angelayi/133/base 2025-12-04T09:23:32.5890182Z * [new branch] gh/angelayi/133/head -> origin/gh/angelayi/133/head 2025-12-04T09:23:32.5890499Z * [new branch] gh/angelayi/133/orig -> origin/gh/angelayi/133/orig 2025-12-04T09:23:32.5890815Z * [new branch] gh/angelayi/134/base -> origin/gh/angelayi/134/base 2025-12-04T09:23:32.5891130Z * [new branch] gh/angelayi/134/head -> origin/gh/angelayi/134/head 2025-12-04T09:23:32.5891440Z * [new branch] gh/angelayi/134/orig -> origin/gh/angelayi/134/orig 2025-12-04T09:23:32.5891764Z * [new branch] gh/angelayi/135/base -> origin/gh/angelayi/135/base 2025-12-04T09:23:32.5892478Z * [new branch] gh/angelayi/135/head -> origin/gh/angelayi/135/head 2025-12-04T09:23:32.5892885Z * [new branch] gh/angelayi/135/orig -> origin/gh/angelayi/135/orig 2025-12-04T09:23:32.5896628Z * [new branch] gh/angelayi/136/base -> origin/gh/angelayi/136/base 2025-12-04T09:23:32.5896979Z * [new branch] gh/angelayi/136/head -> origin/gh/angelayi/136/head 2025-12-04T09:23:32.5897315Z * [new branch] gh/angelayi/136/orig -> origin/gh/angelayi/136/orig 2025-12-04T09:23:32.5897676Z * [new branch] gh/angelayi/137/base -> origin/gh/angelayi/137/base 2025-12-04T09:23:32.5898003Z * [new branch] gh/angelayi/137/head -> origin/gh/angelayi/137/head 2025-12-04T09:23:32.5900051Z * [new branch] gh/angelayi/137/orig -> origin/gh/angelayi/137/orig 2025-12-04T09:23:32.5900392Z * [new branch] gh/angelayi/138/base -> origin/gh/angelayi/138/base 2025-12-04T09:23:32.5900719Z * [new branch] gh/angelayi/138/head -> origin/gh/angelayi/138/head 2025-12-04T09:23:32.5901039Z * [new branch] gh/angelayi/138/orig -> origin/gh/angelayi/138/orig 2025-12-04T09:23:32.5901365Z * [new branch] gh/angelayi/139/base -> origin/gh/angelayi/139/base 2025-12-04T09:23:32.5904901Z * [new branch] gh/angelayi/139/head -> origin/gh/angelayi/139/head 2025-12-04T09:23:32.5905234Z * [new branch] gh/angelayi/139/orig -> origin/gh/angelayi/139/orig 2025-12-04T09:23:32.5905830Z * [new branch] gh/angelayi/140/base -> origin/gh/angelayi/140/base 2025-12-04T09:23:32.5906169Z * [new branch] gh/angelayi/140/head -> origin/gh/angelayi/140/head 2025-12-04T09:23:32.5906496Z * [new branch] gh/angelayi/140/orig -> origin/gh/angelayi/140/orig 2025-12-04T09:23:32.5906959Z * [new branch] gh/angelayi/141/base -> origin/gh/angelayi/141/base 2025-12-04T09:23:32.5907414Z * [new branch] gh/angelayi/141/head -> origin/gh/angelayi/141/head 2025-12-04T09:23:32.5907824Z * [new branch] gh/angelayi/141/orig -> origin/gh/angelayi/141/orig 2025-12-04T09:23:32.5908213Z * [new branch] gh/angelayi/142/base -> origin/gh/angelayi/142/base 2025-12-04T09:23:32.5908918Z * [new branch] gh/angelayi/142/head -> origin/gh/angelayi/142/head 2025-12-04T09:23:32.5909926Z * [new branch] gh/angelayi/142/orig -> origin/gh/angelayi/142/orig 2025-12-04T09:23:32.5910874Z * [new branch] gh/angelayi/143/base -> origin/gh/angelayi/143/base 2025-12-04T09:23:32.5911437Z * [new branch] gh/angelayi/143/head -> origin/gh/angelayi/143/head 2025-12-04T09:23:32.5912164Z * [new branch] gh/angelayi/143/orig -> origin/gh/angelayi/143/orig 2025-12-04T09:23:32.5913242Z * [new branch] gh/angelayi/144/base -> origin/gh/angelayi/144/base 2025-12-04T09:23:32.5913964Z * [new branch] gh/angelayi/144/head -> origin/gh/angelayi/144/head 2025-12-04T09:23:32.5914674Z * [new branch] gh/angelayi/144/orig -> origin/gh/angelayi/144/orig 2025-12-04T09:23:32.5916883Z * [new branch] gh/anijain2305/753/base -> origin/gh/anijain2305/753/base 2025-12-04T09:23:32.5921542Z * [new branch] gh/anijain2305/753/head -> origin/gh/anijain2305/753/head 2025-12-04T09:23:32.5926204Z * [new branch] gh/anijain2305/753/orig -> origin/gh/anijain2305/753/orig 2025-12-04T09:23:32.5926686Z * [new branch] gh/anijain2305/810/base -> origin/gh/anijain2305/810/base 2025-12-04T09:23:32.5927032Z * [new branch] gh/anijain2305/810/head -> origin/gh/anijain2305/810/head 2025-12-04T09:23:32.5927390Z * [new branch] gh/anijain2305/810/orig -> origin/gh/anijain2305/810/orig 2025-12-04T09:23:32.5927722Z * [new branch] gh/anijain2305/854/base -> origin/gh/anijain2305/854/base 2025-12-04T09:23:32.5928054Z * [new branch] gh/anijain2305/854/head -> origin/gh/anijain2305/854/head 2025-12-04T09:23:32.5928383Z * [new branch] gh/anijain2305/854/orig -> origin/gh/anijain2305/854/orig 2025-12-04T09:23:32.5928725Z * [new branch] gh/anijain2305/864/base -> origin/gh/anijain2305/864/base 2025-12-04T09:23:32.5929057Z * [new branch] gh/anijain2305/864/head -> origin/gh/anijain2305/864/head 2025-12-04T09:23:32.5929388Z * [new branch] gh/anijain2305/864/orig -> origin/gh/anijain2305/864/orig 2025-12-04T09:23:32.5929716Z * [new branch] gh/anijain2305/870/base -> origin/gh/anijain2305/870/base 2025-12-04T09:23:32.5930048Z * [new branch] gh/anijain2305/870/head -> origin/gh/anijain2305/870/head 2025-12-04T09:23:32.5930386Z * [new branch] gh/anijain2305/870/orig -> origin/gh/anijain2305/870/orig 2025-12-04T09:23:32.5930718Z * [new branch] gh/anijain2305/873/base -> origin/gh/anijain2305/873/base 2025-12-04T09:23:32.5931477Z * [new branch] gh/anijain2305/873/head -> origin/gh/anijain2305/873/head 2025-12-04T09:23:32.5931911Z * [new branch] gh/anijain2305/873/orig -> origin/gh/anijain2305/873/orig 2025-12-04T09:23:32.5932320Z * [new branch] gh/anijain2305/894/base -> origin/gh/anijain2305/894/base 2025-12-04T09:23:32.5937034Z * [new branch] gh/anijain2305/894/head -> origin/gh/anijain2305/894/head 2025-12-04T09:23:32.5942874Z * [new branch] gh/anijain2305/894/orig -> origin/gh/anijain2305/894/orig 2025-12-04T09:23:32.5947453Z * [new branch] gh/anijain2305/895/base -> origin/gh/anijain2305/895/base 2025-12-04T09:23:32.5948104Z * [new branch] gh/anijain2305/895/head -> origin/gh/anijain2305/895/head 2025-12-04T09:23:32.5948791Z * [new branch] gh/anijain2305/895/orig -> origin/gh/anijain2305/895/orig 2025-12-04T09:23:32.5949729Z * [new branch] gh/anijain2305/910/base -> origin/gh/anijain2305/910/base 2025-12-04T09:23:32.5950297Z * [new branch] gh/anijain2305/910/head -> origin/gh/anijain2305/910/head 2025-12-04T09:23:32.5950774Z * [new branch] gh/anijain2305/910/orig -> origin/gh/anijain2305/910/orig 2025-12-04T09:23:32.5951132Z * [new branch] gh/anijain2305/919/base -> origin/gh/anijain2305/919/base 2025-12-04T09:23:32.5951506Z * [new branch] gh/anijain2305/919/head -> origin/gh/anijain2305/919/head 2025-12-04T09:23:32.5951843Z * [new branch] gh/anijain2305/919/orig -> origin/gh/anijain2305/919/orig 2025-12-04T09:23:32.5952181Z * [new branch] gh/anijain2305/922/base -> origin/gh/anijain2305/922/base 2025-12-04T09:23:32.5952549Z * [new branch] gh/anijain2305/922/head -> origin/gh/anijain2305/922/head 2025-12-04T09:23:32.5953017Z * [new branch] gh/anijain2305/922/orig -> origin/gh/anijain2305/922/orig 2025-12-04T09:23:32.5953389Z * [new branch] gh/anijain2305/932/base -> origin/gh/anijain2305/932/base 2025-12-04T09:23:32.5953772Z * [new branch] gh/anijain2305/932/head -> origin/gh/anijain2305/932/head 2025-12-04T09:23:32.5954145Z * [new branch] gh/anijain2305/932/orig -> origin/gh/anijain2305/932/orig 2025-12-04T09:23:32.5954499Z * [new branch] gh/anijain2305/940/base -> origin/gh/anijain2305/940/base 2025-12-04T09:23:32.5954866Z * [new branch] gh/anijain2305/940/head -> origin/gh/anijain2305/940/head 2025-12-04T09:23:32.5955235Z * [new branch] gh/anijain2305/940/orig -> origin/gh/anijain2305/940/orig 2025-12-04T09:23:32.5955605Z * [new branch] gh/anijain2305/941/base -> origin/gh/anijain2305/941/base 2025-12-04T09:23:32.5955964Z * [new branch] gh/anijain2305/941/head -> origin/gh/anijain2305/941/head 2025-12-04T09:23:32.5956307Z * [new branch] gh/anijain2305/941/orig -> origin/gh/anijain2305/941/orig 2025-12-04T09:23:32.5956677Z * [new branch] gh/anijain2305/942/base -> origin/gh/anijain2305/942/base 2025-12-04T09:23:32.5957044Z * [new branch] gh/anijain2305/942/head -> origin/gh/anijain2305/942/head 2025-12-04T09:23:32.5957405Z * [new branch] gh/anijain2305/942/orig -> origin/gh/anijain2305/942/orig 2025-12-04T09:23:32.5957767Z * [new branch] gh/anijain2305/943/base -> origin/gh/anijain2305/943/base 2025-12-04T09:23:32.5958140Z * [new branch] gh/anijain2305/943/head -> origin/gh/anijain2305/943/head 2025-12-04T09:23:32.5958505Z * [new branch] gh/anijain2305/943/orig -> origin/gh/anijain2305/943/orig 2025-12-04T09:23:32.5958865Z * [new branch] gh/anijain2305/944/base -> origin/gh/anijain2305/944/base 2025-12-04T09:23:32.5959220Z * [new branch] gh/anijain2305/944/head -> origin/gh/anijain2305/944/head 2025-12-04T09:23:32.5959589Z * [new branch] gh/anijain2305/944/orig -> origin/gh/anijain2305/944/orig 2025-12-04T09:23:32.5959955Z * [new branch] gh/anijain2305/945/base -> origin/gh/anijain2305/945/base 2025-12-04T09:23:32.5960312Z * [new branch] gh/anijain2305/945/head -> origin/gh/anijain2305/945/head 2025-12-04T09:23:32.5960677Z * [new branch] gh/anijain2305/945/orig -> origin/gh/anijain2305/945/orig 2025-12-04T09:23:32.5961187Z * [new branch] gh/anijain2305/946/base -> origin/gh/anijain2305/946/base 2025-12-04T09:23:32.5961587Z * [new branch] gh/anijain2305/946/head -> origin/gh/anijain2305/946/head 2025-12-04T09:23:32.5961950Z * [new branch] gh/anijain2305/946/orig -> origin/gh/anijain2305/946/orig 2025-12-04T09:23:32.5962362Z * [new branch] gh/anijain2305/947/base -> origin/gh/anijain2305/947/base 2025-12-04T09:23:32.5962736Z * [new branch] gh/anijain2305/947/head -> origin/gh/anijain2305/947/head 2025-12-04T09:23:32.5963121Z * [new branch] gh/anijain2305/947/orig -> origin/gh/anijain2305/947/orig 2025-12-04T09:23:32.5964286Z * [new branch] gh/anijain2305/948/base -> origin/gh/anijain2305/948/base 2025-12-04T09:23:32.5964713Z * [new branch] gh/anijain2305/948/head -> origin/gh/anijain2305/948/head 2025-12-04T09:23:32.5967487Z * [new branch] gh/anijain2305/948/orig -> origin/gh/anijain2305/948/orig 2025-12-04T09:23:32.5967958Z * [new branch] gh/anijain2305/949/base -> origin/gh/anijain2305/949/base 2025-12-04T09:23:32.5968359Z * [new branch] gh/anijain2305/949/head -> origin/gh/anijain2305/949/head 2025-12-04T09:23:32.5968791Z * [new branch] gh/anijain2305/949/orig -> origin/gh/anijain2305/949/orig 2025-12-04T09:23:32.5969353Z * [new branch] gh/anijain2305/950/base -> origin/gh/anijain2305/950/base 2025-12-04T09:23:32.5969734Z * [new branch] gh/anijain2305/950/head -> origin/gh/anijain2305/950/head 2025-12-04T09:23:32.5970195Z * [new branch] gh/anijain2305/950/orig -> origin/gh/anijain2305/950/orig 2025-12-04T09:23:32.5972704Z * [new branch] gh/anijain2305/951/base -> origin/gh/anijain2305/951/base 2025-12-04T09:23:32.5977404Z * [new branch] gh/anijain2305/951/head -> origin/gh/anijain2305/951/head 2025-12-04T09:23:32.5977941Z * [new branch] gh/anijain2305/951/orig -> origin/gh/anijain2305/951/orig 2025-12-04T09:23:32.5981625Z * [new branch] gh/anijain2305/952/base -> origin/gh/anijain2305/952/base 2025-12-04T09:23:32.5982462Z * [new branch] gh/anijain2305/952/head -> origin/gh/anijain2305/952/head 2025-12-04T09:23:32.5982945Z * [new branch] gh/anijain2305/952/orig -> origin/gh/anijain2305/952/orig 2025-12-04T09:23:32.5983327Z * [new branch] gh/anijain2305/953/base -> origin/gh/anijain2305/953/base 2025-12-04T09:23:32.5983693Z * [new branch] gh/anijain2305/953/head -> origin/gh/anijain2305/953/head 2025-12-04T09:23:32.5984067Z * [new branch] gh/anijain2305/953/orig -> origin/gh/anijain2305/953/orig 2025-12-04T09:23:32.5984435Z * [new branch] gh/anijain2305/954/base -> origin/gh/anijain2305/954/base 2025-12-04T09:23:32.5984821Z * [new branch] gh/anijain2305/954/head -> origin/gh/anijain2305/954/head 2025-12-04T09:23:32.5985182Z * [new branch] gh/anijain2305/954/orig -> origin/gh/anijain2305/954/orig 2025-12-04T09:23:32.5985553Z * [new branch] gh/anijain2305/955/base -> origin/gh/anijain2305/955/base 2025-12-04T09:23:32.5985930Z * [new branch] gh/anijain2305/955/head -> origin/gh/anijain2305/955/head 2025-12-04T09:23:32.5986298Z * [new branch] gh/anijain2305/955/orig -> origin/gh/anijain2305/955/orig 2025-12-04T09:23:32.5986664Z * [new branch] gh/anijain2305/956/base -> origin/gh/anijain2305/956/base 2025-12-04T09:23:32.5987030Z * [new branch] gh/anijain2305/956/head -> origin/gh/anijain2305/956/head 2025-12-04T09:23:32.5987402Z * [new branch] gh/anijain2305/956/orig -> origin/gh/anijain2305/956/orig 2025-12-04T09:23:32.5987796Z * [new branch] gh/anijain2305/957/base -> origin/gh/anijain2305/957/base 2025-12-04T09:23:32.5988332Z * [new branch] gh/anijain2305/957/head -> origin/gh/anijain2305/957/head 2025-12-04T09:23:32.5988677Z * [new branch] gh/anijain2305/957/orig -> origin/gh/anijain2305/957/orig 2025-12-04T09:23:32.5989008Z * [new branch] gh/anijain2305/958/base -> origin/gh/anijain2305/958/base 2025-12-04T09:23:32.5989436Z * [new branch] gh/anijain2305/958/head -> origin/gh/anijain2305/958/head 2025-12-04T09:23:32.5990202Z * [new branch] gh/anijain2305/958/orig -> origin/gh/anijain2305/958/orig 2025-12-04T09:23:32.5991611Z * [new branch] gh/anijain2305/959/base -> origin/gh/anijain2305/959/base 2025-12-04T09:23:32.5991985Z * [new branch] gh/anijain2305/959/head -> origin/gh/anijain2305/959/head 2025-12-04T09:23:32.5992336Z * [new branch] gh/anijain2305/959/orig -> origin/gh/anijain2305/959/orig 2025-12-04T09:23:32.5994657Z * [new branch] gh/anijain2305/960/base -> origin/gh/anijain2305/960/base 2025-12-04T09:23:32.5995045Z * [new branch] gh/anijain2305/960/head -> origin/gh/anijain2305/960/head 2025-12-04T09:23:32.5995411Z * [new branch] gh/anijain2305/960/orig -> origin/gh/anijain2305/960/orig 2025-12-04T09:23:32.5998425Z * [new branch] gh/anijain2305/961/base -> origin/gh/anijain2305/961/base 2025-12-04T09:23:32.5998799Z * [new branch] gh/anijain2305/961/head -> origin/gh/anijain2305/961/head 2025-12-04T09:23:32.5999162Z * [new branch] gh/anijain2305/961/orig -> origin/gh/anijain2305/961/orig 2025-12-04T09:23:32.5999525Z * [new branch] gh/anijain2305/962/base -> origin/gh/anijain2305/962/base 2025-12-04T09:23:32.5999897Z * [new branch] gh/anijain2305/962/head -> origin/gh/anijain2305/962/head 2025-12-04T09:23:32.6001401Z * [new branch] gh/anijain2305/962/orig -> origin/gh/anijain2305/962/orig 2025-12-04T09:23:32.6001783Z * [new branch] gh/anijain2305/963/base -> origin/gh/anijain2305/963/base 2025-12-04T09:23:32.6002145Z * [new branch] gh/anijain2305/963/head -> origin/gh/anijain2305/963/head 2025-12-04T09:23:32.6005282Z * [new branch] gh/anijain2305/963/orig -> origin/gh/anijain2305/963/orig 2025-12-04T09:23:32.6005814Z * [new branch] gh/anijain2305/964/base -> origin/gh/anijain2305/964/base 2025-12-04T09:23:32.6006184Z * [new branch] gh/anijain2305/964/head -> origin/gh/anijain2305/964/head 2025-12-04T09:23:32.6006546Z * [new branch] gh/anijain2305/964/orig -> origin/gh/anijain2305/964/orig 2025-12-04T09:23:32.6006910Z * [new branch] gh/anijain2305/965/base -> origin/gh/anijain2305/965/base 2025-12-04T09:23:32.6013839Z * [new branch] gh/anijain2305/965/head -> origin/gh/anijain2305/965/head 2025-12-04T09:23:32.6014461Z * [new branch] gh/anijain2305/965/orig -> origin/gh/anijain2305/965/orig 2025-12-04T09:23:32.6014946Z * [new branch] gh/anijain2305/966/base -> origin/gh/anijain2305/966/base 2025-12-04T09:23:32.6015303Z * [new branch] gh/anijain2305/966/head -> origin/gh/anijain2305/966/head 2025-12-04T09:23:32.6015665Z * [new branch] gh/anijain2305/966/orig -> origin/gh/anijain2305/966/orig 2025-12-04T09:23:32.6016011Z * [new branch] gh/anijain2305/967/base -> origin/gh/anijain2305/967/base 2025-12-04T09:23:32.6016344Z * [new branch] gh/anijain2305/967/head -> origin/gh/anijain2305/967/head 2025-12-04T09:23:32.6016689Z * [new branch] gh/anijain2305/967/orig -> origin/gh/anijain2305/967/orig 2025-12-04T09:23:32.6017032Z * [new branch] gh/anijain2305/968/base -> origin/gh/anijain2305/968/base 2025-12-04T09:23:32.6017436Z * [new branch] gh/anijain2305/968/head -> origin/gh/anijain2305/968/head 2025-12-04T09:23:32.6018037Z * [new branch] gh/anijain2305/968/orig -> origin/gh/anijain2305/968/orig 2025-12-04T09:23:32.6018407Z * [new branch] gh/anijain2305/969/base -> origin/gh/anijain2305/969/base 2025-12-04T09:23:32.6018782Z * [new branch] gh/anijain2305/969/head -> origin/gh/anijain2305/969/head 2025-12-04T09:23:32.6019228Z * [new branch] gh/anijain2305/969/orig -> origin/gh/anijain2305/969/orig 2025-12-04T09:23:32.6019613Z * [new branch] gh/anijain2305/970/base -> origin/gh/anijain2305/970/base 2025-12-04T09:23:32.6019950Z * [new branch] gh/anijain2305/970/head -> origin/gh/anijain2305/970/head 2025-12-04T09:23:32.6020437Z * [new branch] gh/anijain2305/970/orig -> origin/gh/anijain2305/970/orig 2025-12-04T09:23:32.6021243Z * [new branch] gh/anjali411/216/base -> origin/gh/anjali411/216/base 2025-12-04T09:23:32.6021740Z * [new branch] gh/anjali411/216/head -> origin/gh/anjali411/216/head 2025-12-04T09:23:32.6022233Z * [new branch] gh/anjali411/216/orig -> origin/gh/anjali411/216/orig 2025-12-04T09:23:32.6024500Z * [new branch] gh/anshul-si/1/base -> origin/gh/anshul-si/1/base 2025-12-04T09:23:32.6024899Z * [new branch] gh/anshul-si/1/head -> origin/gh/anshul-si/1/head 2025-12-04T09:23:32.6025275Z * [new branch] gh/anshul-si/2/base -> origin/gh/anshul-si/2/base 2025-12-04T09:23:32.6025610Z * [new branch] gh/anshul-si/2/head -> origin/gh/anshul-si/2/head 2025-12-04T09:23:32.6026280Z * [new branch] gh/anshul-si/3/base -> origin/gh/anshul-si/3/base 2025-12-04T09:23:32.6026939Z * [new branch] gh/anshul-si/3/head -> origin/gh/anshul-si/3/head 2025-12-04T09:23:32.6027762Z * [new branch] gh/anshul-si/4/base -> origin/gh/anshul-si/4/base 2025-12-04T09:23:32.6028445Z * [new branch] gh/anshul-si/4/head -> origin/gh/anshul-si/4/head 2025-12-04T09:23:32.6029308Z * [new branch] gh/anshul-si/5/base -> origin/gh/anshul-si/5/base 2025-12-04T09:23:32.6029942Z * [new branch] gh/anshul-si/5/head -> origin/gh/anshul-si/5/head 2025-12-04T09:23:32.6036710Z * [new branch] gh/anshul-si/53/base -> origin/gh/anshul-si/53/base 2025-12-04T09:23:32.6041413Z * [new branch] gh/anshul-si/53/head -> origin/gh/anshul-si/53/head 2025-12-04T09:23:32.6046102Z * [new branch] gh/anshul-si/58/base -> origin/gh/anshul-si/58/base 2025-12-04T09:23:32.6048611Z * [new branch] gh/anshul-si/58/head -> origin/gh/anshul-si/58/head 2025-12-04T09:23:32.6049187Z * [new branch] gh/anshul-si/66/base -> origin/gh/anshul-si/66/base 2025-12-04T09:23:32.6049672Z * [new branch] gh/anshul-si/66/head -> origin/gh/anshul-si/66/head 2025-12-04T09:23:32.6050178Z * [new branch] gh/anshul-si/66/orig -> origin/gh/anshul-si/66/orig 2025-12-04T09:23:32.6050569Z * [new branch] gh/anshul-si/67/base -> origin/gh/anshul-si/67/base 2025-12-04T09:23:32.6050950Z * [new branch] gh/anshul-si/67/head -> origin/gh/anshul-si/67/head 2025-12-04T09:23:32.6051313Z * [new branch] gh/anshul-si/67/orig -> origin/gh/anshul-si/67/orig 2025-12-04T09:23:32.6051649Z * [new branch] gh/anshul-si/68/base -> origin/gh/anshul-si/68/base 2025-12-04T09:23:32.6052013Z * [new branch] gh/anshul-si/68/head -> origin/gh/anshul-si/68/head 2025-12-04T09:23:32.6052375Z * [new branch] gh/anshul-si/68/orig -> origin/gh/anshul-si/68/orig 2025-12-04T09:23:32.6052742Z * [new branch] gh/anshul-si/69/base -> origin/gh/anshul-si/69/base 2025-12-04T09:23:32.6053068Z * [new branch] gh/anshul-si/69/head -> origin/gh/anshul-si/69/head 2025-12-04T09:23:32.6053546Z * [new branch] gh/anshul-si/69/orig -> origin/gh/anshul-si/69/orig 2025-12-04T09:23:32.6053939Z * [new branch] gh/anshul-si/70/base -> origin/gh/anshul-si/70/base 2025-12-04T09:23:32.6054308Z * [new branch] gh/anshul-si/70/head -> origin/gh/anshul-si/70/head 2025-12-04T09:23:32.6054723Z * [new branch] gh/anshul-si/70/orig -> origin/gh/anshul-si/70/orig 2025-12-04T09:23:32.6055076Z * [new branch] gh/anshul-si/71/base -> origin/gh/anshul-si/71/base 2025-12-04T09:23:32.6055430Z * [new branch] gh/anshul-si/71/head -> origin/gh/anshul-si/71/head 2025-12-04T09:23:32.6055741Z * [new branch] gh/anshul-si/71/orig -> origin/gh/anshul-si/71/orig 2025-12-04T09:23:32.6056062Z * [new branch] gh/anshul-si/72/base -> origin/gh/anshul-si/72/base 2025-12-04T09:23:32.6056381Z * [new branch] gh/anshul-si/72/head -> origin/gh/anshul-si/72/head 2025-12-04T09:23:32.6056707Z * [new branch] gh/anshul-si/72/orig -> origin/gh/anshul-si/72/orig 2025-12-04T09:23:32.6057023Z * [new branch] gh/anshul-si/73/base -> origin/gh/anshul-si/73/base 2025-12-04T09:23:32.6057345Z * [new branch] gh/anshul-si/73/head -> origin/gh/anshul-si/73/head 2025-12-04T09:23:32.6057669Z * [new branch] gh/anshul-si/73/orig -> origin/gh/anshul-si/73/orig 2025-12-04T09:23:32.6057999Z * [new branch] gh/aorenste/132/base -> origin/gh/aorenste/132/base 2025-12-04T09:23:32.6058317Z * [new branch] gh/aorenste/132/head -> origin/gh/aorenste/132/head 2025-12-04T09:23:32.6059058Z * [new branch] gh/aorenste/134/base -> origin/gh/aorenste/134/base 2025-12-04T09:23:32.6059446Z * [new branch] gh/aorenste/134/head -> origin/gh/aorenste/134/head 2025-12-04T09:23:32.6059788Z * [new branch] gh/aorenste/134/orig -> origin/gh/aorenste/134/orig 2025-12-04T09:23:32.6060128Z * [new branch] gh/aorenste/139/base -> origin/gh/aorenste/139/base 2025-12-04T09:23:32.6060458Z * [new branch] gh/aorenste/139/head -> origin/gh/aorenste/139/head 2025-12-04T09:23:32.6062751Z * [new branch] gh/aorenste/139/orig -> origin/gh/aorenste/139/orig 2025-12-04T09:23:32.6063161Z * [new branch] gh/aorenste/141/base -> origin/gh/aorenste/141/base 2025-12-04T09:23:32.6063556Z * [new branch] gh/aorenste/141/head -> origin/gh/aorenste/141/head 2025-12-04T09:23:32.6063894Z * [new branch] gh/aorenste/145/base -> origin/gh/aorenste/145/base 2025-12-04T09:23:32.6064275Z * [new branch] gh/aorenste/145/head -> origin/gh/aorenste/145/head 2025-12-04T09:23:32.6066893Z * [new branch] gh/aorenste/145/orig -> origin/gh/aorenste/145/orig 2025-12-04T09:23:32.6067221Z * [new branch] gh/aorenste/146/base -> origin/gh/aorenste/146/base 2025-12-04T09:23:32.6067546Z * [new branch] gh/aorenste/146/head -> origin/gh/aorenste/146/head 2025-12-04T09:23:32.6067865Z * [new branch] gh/aorenste/146/orig -> origin/gh/aorenste/146/orig 2025-12-04T09:23:32.6068201Z * [new branch] gh/aorenste/147/base -> origin/gh/aorenste/147/base 2025-12-04T09:23:32.6068678Z * [new branch] gh/aorenste/147/head -> origin/gh/aorenste/147/head 2025-12-04T09:23:32.6071638Z * [new branch] gh/aorenste/147/orig -> origin/gh/aorenste/147/orig 2025-12-04T09:23:32.6071967Z * [new branch] gh/aorenste/148/base -> origin/gh/aorenste/148/base 2025-12-04T09:23:32.6072293Z * [new branch] gh/aorenste/148/head -> origin/gh/aorenste/148/head 2025-12-04T09:23:32.6072728Z * [new branch] gh/aorenste/148/orig -> origin/gh/aorenste/148/orig 2025-12-04T09:23:32.6073701Z * [new branch] gh/aorenste/149/base -> origin/gh/aorenste/149/base 2025-12-04T09:23:32.6076811Z * [new branch] gh/aorenste/149/head -> origin/gh/aorenste/149/head 2025-12-04T09:23:32.6077151Z * [new branch] gh/aorenste/149/orig -> origin/gh/aorenste/149/orig 2025-12-04T09:23:32.6077639Z * [new branch] gh/aorenste/150/base -> origin/gh/aorenste/150/base 2025-12-04T09:23:32.6078000Z * [new branch] gh/aorenste/150/head -> origin/gh/aorenste/150/head 2025-12-04T09:23:32.6078343Z * [new branch] gh/aorenste/150/orig -> origin/gh/aorenste/150/orig 2025-12-04T09:23:32.6078667Z * [new branch] gh/aorenste/151/base -> origin/gh/aorenste/151/base 2025-12-04T09:23:32.6082983Z * [new branch] gh/aorenste/151/head -> origin/gh/aorenste/151/head 2025-12-04T09:23:32.6085441Z * [new branch] gh/aorenste/151/orig -> origin/gh/aorenste/151/orig 2025-12-04T09:23:32.6085880Z * [new branch] gh/aorenste/152/base -> origin/gh/aorenste/152/base 2025-12-04T09:23:32.6086217Z * [new branch] gh/aorenste/152/head -> origin/gh/aorenste/152/head 2025-12-04T09:23:32.6086584Z * [new branch] gh/aorenste/152/orig -> origin/gh/aorenste/152/orig 2025-12-04T09:23:32.6086935Z * [new branch] gh/aorenste/153/base -> origin/gh/aorenste/153/base 2025-12-04T09:23:32.6087256Z * [new branch] gh/aorenste/153/head -> origin/gh/aorenste/153/head 2025-12-04T09:23:32.6087569Z * [new branch] gh/aorenste/153/orig -> origin/gh/aorenste/153/orig 2025-12-04T09:23:32.6087888Z * [new branch] gh/aorenste/154/base -> origin/gh/aorenste/154/base 2025-12-04T09:23:32.6088207Z * [new branch] gh/aorenste/154/head -> origin/gh/aorenste/154/head 2025-12-04T09:23:32.6088532Z * [new branch] gh/aorenste/154/orig -> origin/gh/aorenste/154/orig 2025-12-04T09:23:32.6088885Z * [new branch] gh/aorenste/155/base -> origin/gh/aorenste/155/base 2025-12-04T09:23:32.6089211Z * [new branch] gh/aorenste/155/head -> origin/gh/aorenste/155/head 2025-12-04T09:23:32.6089533Z * [new branch] gh/aorenste/155/orig -> origin/gh/aorenste/155/orig 2025-12-04T09:23:32.6089857Z * [new branch] gh/aorenste/156/base -> origin/gh/aorenste/156/base 2025-12-04T09:23:32.6090178Z * [new branch] gh/aorenste/156/head -> origin/gh/aorenste/156/head 2025-12-04T09:23:32.6090510Z * [new branch] gh/aorenste/156/orig -> origin/gh/aorenste/156/orig 2025-12-04T09:23:32.6092682Z * [new branch] gh/aorenste/157/base -> origin/gh/aorenste/157/base 2025-12-04T09:23:32.6093021Z * [new branch] gh/aorenste/157/head -> origin/gh/aorenste/157/head 2025-12-04T09:23:32.6093356Z * [new branch] gh/aorenste/157/orig -> origin/gh/aorenste/157/orig 2025-12-04T09:23:32.6093695Z * [new branch] gh/aorenste/158/base -> origin/gh/aorenste/158/base 2025-12-04T09:23:32.6096233Z * [new branch] gh/aorenste/158/head -> origin/gh/aorenste/158/head 2025-12-04T09:23:32.6096578Z * [new branch] gh/aorenste/158/orig -> origin/gh/aorenste/158/orig 2025-12-04T09:23:32.6096905Z * [new branch] gh/aorenste/159/base -> origin/gh/aorenste/159/base 2025-12-04T09:23:32.6097246Z * [new branch] gh/aorenste/159/head -> origin/gh/aorenste/159/head 2025-12-04T09:23:32.6097582Z * [new branch] gh/aorenste/159/orig -> origin/gh/aorenste/159/orig 2025-12-04T09:23:32.6101182Z * [new branch] gh/avikchaudhuri/1/base -> origin/gh/avikchaudhuri/1/base 2025-12-04T09:23:32.6101550Z * [new branch] gh/avikchaudhuri/1/head -> origin/gh/avikchaudhuri/1/head 2025-12-04T09:23:32.6102372Z * [new branch] gh/avikchaudhuri/2/base -> origin/gh/avikchaudhuri/2/base 2025-12-04T09:23:32.6102732Z * [new branch] gh/avikchaudhuri/2/head -> origin/gh/avikchaudhuri/2/head 2025-12-04T09:23:32.6103094Z * [new branch] gh/avikchaudhuri/2/orig -> origin/gh/avikchaudhuri/2/orig 2025-12-04T09:23:32.6106425Z * [new branch] gh/bdhirsh/666/base -> origin/gh/bdhirsh/666/base 2025-12-04T09:23:32.6106782Z * [new branch] gh/bdhirsh/666/head -> origin/gh/bdhirsh/666/head 2025-12-04T09:23:32.6107113Z * [new branch] gh/bdhirsh/666/orig -> origin/gh/bdhirsh/666/orig 2025-12-04T09:23:32.6107445Z * [new branch] gh/bdhirsh/668/base -> origin/gh/bdhirsh/668/base 2025-12-04T09:23:32.6107780Z * [new branch] gh/bdhirsh/668/head -> origin/gh/bdhirsh/668/head 2025-12-04T09:23:32.6108121Z * [new branch] gh/bdhirsh/668/orig -> origin/gh/bdhirsh/668/orig 2025-12-04T09:23:32.6108468Z * [new branch] gh/bdhirsh/669/base -> origin/gh/bdhirsh/669/base 2025-12-04T09:23:32.6108820Z * [new branch] gh/bdhirsh/669/head -> origin/gh/bdhirsh/669/head 2025-12-04T09:23:32.6109156Z * [new branch] gh/bdhirsh/669/orig -> origin/gh/bdhirsh/669/orig 2025-12-04T09:23:32.6109507Z * [new branch] gh/bdhirsh/670/base -> origin/gh/bdhirsh/670/base 2025-12-04T09:23:32.6110013Z * [new branch] gh/bdhirsh/670/head -> origin/gh/bdhirsh/670/head 2025-12-04T09:23:32.6110744Z * [new branch] gh/bdhirsh/670/orig -> origin/gh/bdhirsh/670/orig 2025-12-04T09:23:32.6112124Z * [new branch] gh/bdhirsh/672/base -> origin/gh/bdhirsh/672/base 2025-12-04T09:23:32.6112898Z * [new branch] gh/bdhirsh/672/head -> origin/gh/bdhirsh/672/head 2025-12-04T09:23:32.6117471Z * [new branch] gh/bdhirsh/672/orig -> origin/gh/bdhirsh/672/orig 2025-12-04T09:23:32.6117851Z * [new branch] gh/bdhirsh/675/base -> origin/gh/bdhirsh/675/base 2025-12-04T09:23:32.6122577Z * [new branch] gh/bdhirsh/675/head -> origin/gh/bdhirsh/675/head 2025-12-04T09:23:32.6122962Z * [new branch] gh/bdhirsh/675/orig -> origin/gh/bdhirsh/675/orig 2025-12-04T09:23:32.6123321Z * [new branch] gh/bdhirsh/676/base -> origin/gh/bdhirsh/676/base 2025-12-04T09:23:32.6123672Z * [new branch] gh/bdhirsh/676/head -> origin/gh/bdhirsh/676/head 2025-12-04T09:23:32.6124023Z * [new branch] gh/bdhirsh/676/orig -> origin/gh/bdhirsh/676/orig 2025-12-04T09:23:32.6124170Z * [new branch] gh/bdhirsh/677/base -> origin/gh/bdhirsh/677/base 2025-12-04T09:23:32.6124318Z * [new branch] gh/bdhirsh/677/head -> origin/gh/bdhirsh/677/head 2025-12-04T09:23:32.6124464Z * [new branch] gh/bdhirsh/677/orig -> origin/gh/bdhirsh/677/orig 2025-12-04T09:23:32.6124611Z * [new branch] gh/bdhirsh/678/base -> origin/gh/bdhirsh/678/base 2025-12-04T09:23:32.6124758Z * [new branch] gh/bdhirsh/678/head -> origin/gh/bdhirsh/678/head 2025-12-04T09:23:32.6129475Z * [new branch] gh/bdhirsh/678/orig -> origin/gh/bdhirsh/678/orig 2025-12-04T09:23:32.6134325Z * [new branch] gh/bdhirsh/679/base -> origin/gh/bdhirsh/679/base 2025-12-04T09:23:32.6138859Z * [new branch] gh/bdhirsh/679/head -> origin/gh/bdhirsh/679/head 2025-12-04T09:23:32.6141062Z * [new branch] gh/bdhirsh/679/orig -> origin/gh/bdhirsh/679/orig 2025-12-04T09:23:32.6146673Z * [new branch] gh/bdhirsh/680/base -> origin/gh/bdhirsh/680/base 2025-12-04T09:23:32.6146986Z * [new branch] gh/bdhirsh/680/head -> origin/gh/bdhirsh/680/head 2025-12-04T09:23:32.6147550Z * [new branch] gh/bdhirsh/680/orig -> origin/gh/bdhirsh/680/orig 2025-12-04T09:23:32.6147867Z * [new branch] gh/bdhirsh/681/base -> origin/gh/bdhirsh/681/base 2025-12-04T09:23:32.6148565Z * [new branch] gh/bdhirsh/681/head -> origin/gh/bdhirsh/681/head 2025-12-04T09:23:32.6148921Z * [new branch] gh/bdhirsh/681/orig -> origin/gh/bdhirsh/681/orig 2025-12-04T09:23:32.6149129Z * [new branch] gh/benjaminglass1/101/base -> origin/gh/benjaminglass1/101/base 2025-12-04T09:23:32.6149314Z * [new branch] gh/benjaminglass1/101/head -> origin/gh/benjaminglass1/101/head 2025-12-04T09:23:32.6149487Z * [new branch] gh/benjaminglass1/101/orig -> origin/gh/benjaminglass1/101/orig 2025-12-04T09:23:32.6149665Z * [new branch] gh/benjaminglass1/102/base -> origin/gh/benjaminglass1/102/base 2025-12-04T09:23:32.6149853Z * [new branch] gh/benjaminglass1/102/head -> origin/gh/benjaminglass1/102/head 2025-12-04T09:23:32.6150050Z * [new branch] gh/benjaminglass1/102/orig -> origin/gh/benjaminglass1/102/orig 2025-12-04T09:23:32.6150224Z * [new branch] gh/benjaminglass1/106/base -> origin/gh/benjaminglass1/106/base 2025-12-04T09:23:32.6150393Z * [new branch] gh/benjaminglass1/106/head -> origin/gh/benjaminglass1/106/head 2025-12-04T09:23:32.6150565Z * [new branch] gh/benjaminglass1/106/orig -> origin/gh/benjaminglass1/106/orig 2025-12-04T09:23:32.6150735Z * [new branch] gh/benjaminglass1/107/base -> origin/gh/benjaminglass1/107/base 2025-12-04T09:23:32.6150903Z * [new branch] gh/benjaminglass1/107/head -> origin/gh/benjaminglass1/107/head 2025-12-04T09:23:32.6151080Z * [new branch] gh/benjaminglass1/107/orig -> origin/gh/benjaminglass1/107/orig 2025-12-04T09:23:32.6151249Z * [new branch] gh/benjaminglass1/108/base -> origin/gh/benjaminglass1/108/base 2025-12-04T09:23:32.6151433Z * [new branch] gh/benjaminglass1/108/head -> origin/gh/benjaminglass1/108/head 2025-12-04T09:23:32.6151601Z * [new branch] gh/benjaminglass1/108/orig -> origin/gh/benjaminglass1/108/orig 2025-12-04T09:23:32.6151766Z * [new branch] gh/benjaminglass1/109/base -> origin/gh/benjaminglass1/109/base 2025-12-04T09:23:32.6151954Z * [new branch] gh/benjaminglass1/109/head -> origin/gh/benjaminglass1/109/head 2025-12-04T09:23:32.6152136Z * [new branch] gh/benjaminglass1/109/orig -> origin/gh/benjaminglass1/109/orig 2025-12-04T09:23:32.6152300Z * [new branch] gh/benjaminglass1/97/base -> origin/gh/benjaminglass1/97/base 2025-12-04T09:23:32.6152468Z * [new branch] gh/benjaminglass1/97/head -> origin/gh/benjaminglass1/97/head 2025-12-04T09:23:32.6152624Z * [new branch] gh/benjaminglass1/97/orig -> origin/gh/benjaminglass1/97/orig 2025-12-04T09:23:32.6152965Z * [new branch] gh/bobrenjc93/570/base -> origin/gh/bobrenjc93/570/base 2025-12-04T09:23:32.6153133Z * [new branch] gh/bobrenjc93/570/head -> origin/gh/bobrenjc93/570/head 2025-12-04T09:23:32.6153290Z * [new branch] gh/bobrenjc93/570/orig -> origin/gh/bobrenjc93/570/orig 2025-12-04T09:23:32.6153455Z * [new branch] gh/bobrenjc93/604/base -> origin/gh/bobrenjc93/604/base 2025-12-04T09:23:32.6153607Z * [new branch] gh/bobrenjc93/604/head -> origin/gh/bobrenjc93/604/head 2025-12-04T09:23:32.6153762Z * [new branch] gh/bobrenjc93/604/orig -> origin/gh/bobrenjc93/604/orig 2025-12-04T09:23:32.6158544Z * [new branch] gh/bobrenjc93/638/base -> origin/gh/bobrenjc93/638/base 2025-12-04T09:23:32.6161400Z * [new branch] gh/bobrenjc93/638/head -> origin/gh/bobrenjc93/638/head 2025-12-04T09:23:32.6161688Z * [new branch] gh/bobrenjc93/638/orig -> origin/gh/bobrenjc93/638/orig 2025-12-04T09:23:32.6167138Z * [new branch] gh/bobrenjc93/653/base -> origin/gh/bobrenjc93/653/base 2025-12-04T09:23:32.6169249Z * [new branch] gh/bobrenjc93/653/head -> origin/gh/bobrenjc93/653/head 2025-12-04T09:23:32.6169575Z * [new branch] gh/bobrenjc93/653/orig -> origin/gh/bobrenjc93/653/orig 2025-12-04T09:23:32.6169922Z * [new branch] gh/bobrenjc93/654/base -> origin/gh/bobrenjc93/654/base 2025-12-04T09:23:32.6170188Z * [new branch] gh/bobrenjc93/654/head -> origin/gh/bobrenjc93/654/head 2025-12-04T09:23:32.6170371Z * [new branch] gh/bobrenjc93/654/orig -> origin/gh/bobrenjc93/654/orig 2025-12-04T09:23:32.6170507Z * [new branch] gh/bobrenjc93/657/base -> origin/gh/bobrenjc93/657/base 2025-12-04T09:23:32.6170645Z * [new branch] gh/bobrenjc93/657/head -> origin/gh/bobrenjc93/657/head 2025-12-04T09:23:32.6170797Z * [new branch] gh/bobrenjc93/657/orig -> origin/gh/bobrenjc93/657/orig 2025-12-04T09:23:32.6170936Z * [new branch] gh/bobrenjc93/672/base -> origin/gh/bobrenjc93/672/base 2025-12-04T09:23:32.6171070Z * [new branch] gh/bobrenjc93/672/head -> origin/gh/bobrenjc93/672/head 2025-12-04T09:23:32.6171226Z * [new branch] gh/bobrenjc93/672/orig -> origin/gh/bobrenjc93/672/orig 2025-12-04T09:23:32.6171483Z * [new branch] gh/bobrenjc93/679/base -> origin/gh/bobrenjc93/679/base 2025-12-04T09:23:32.6171973Z * [new branch] gh/bobrenjc93/679/head -> origin/gh/bobrenjc93/679/head 2025-12-04T09:23:32.6172154Z * [new branch] gh/bobrenjc93/679/orig -> origin/gh/bobrenjc93/679/orig 2025-12-04T09:23:32.6172300Z * [new branch] gh/bobrenjc93/680/base -> origin/gh/bobrenjc93/680/base 2025-12-04T09:23:32.6172449Z * [new branch] gh/bobrenjc93/680/head -> origin/gh/bobrenjc93/680/head 2025-12-04T09:23:32.6172608Z * [new branch] gh/bobrenjc93/680/orig -> origin/gh/bobrenjc93/680/orig 2025-12-04T09:23:32.6172745Z * [new branch] gh/bobrenjc93/681/base -> origin/gh/bobrenjc93/681/base 2025-12-04T09:23:32.6172886Z * [new branch] gh/bobrenjc93/681/head -> origin/gh/bobrenjc93/681/head 2025-12-04T09:23:32.6173024Z * [new branch] gh/bobrenjc93/681/orig -> origin/gh/bobrenjc93/681/orig 2025-12-04T09:23:32.6173343Z * [new branch] gh/bobrenjc93/682/base -> origin/gh/bobrenjc93/682/base 2025-12-04T09:23:32.6173683Z * [new branch] gh/bobrenjc93/682/head -> origin/gh/bobrenjc93/682/head 2025-12-04T09:23:32.6176820Z * [new branch] gh/bobrenjc93/682/orig -> origin/gh/bobrenjc93/682/orig 2025-12-04T09:23:32.6177000Z * [new branch] gh/bobrenjc93/683/base -> origin/gh/bobrenjc93/683/base 2025-12-04T09:23:32.6177154Z * [new branch] gh/bobrenjc93/683/head -> origin/gh/bobrenjc93/683/head 2025-12-04T09:23:32.6177318Z * [new branch] gh/bobrenjc93/683/orig -> origin/gh/bobrenjc93/683/orig 2025-12-04T09:23:32.6177713Z * [new branch] gh/bobrenjc93/684/base -> origin/gh/bobrenjc93/684/base 2025-12-04T09:23:32.6178705Z * [new branch] gh/bobrenjc93/684/head -> origin/gh/bobrenjc93/684/head 2025-12-04T09:23:32.6179288Z * [new branch] gh/bobrenjc93/684/orig -> origin/gh/bobrenjc93/684/orig 2025-12-04T09:23:32.6183241Z * [new branch] gh/bobrenjc93/685/base -> origin/gh/bobrenjc93/685/base 2025-12-04T09:23:32.6183422Z * [new branch] gh/bobrenjc93/685/head -> origin/gh/bobrenjc93/685/head 2025-12-04T09:23:32.6183569Z * [new branch] gh/bobrenjc93/685/orig -> origin/gh/bobrenjc93/685/orig 2025-12-04T09:23:32.6183720Z * [new branch] gh/bobrenjc93/686/base -> origin/gh/bobrenjc93/686/base 2025-12-04T09:23:32.6184008Z * [new branch] gh/bobrenjc93/686/head -> origin/gh/bobrenjc93/686/head 2025-12-04T09:23:32.6184190Z * [new branch] gh/bobrenjc93/686/orig -> origin/gh/bobrenjc93/686/orig 2025-12-04T09:23:32.6188649Z * [new branch] gh/bobrenjc93/687/base -> origin/gh/bobrenjc93/687/base 2025-12-04T09:23:32.6188971Z * [new branch] gh/bobrenjc93/687/head -> origin/gh/bobrenjc93/687/head 2025-12-04T09:23:32.6189121Z * [new branch] gh/bobrenjc93/687/orig -> origin/gh/bobrenjc93/687/orig 2025-12-04T09:23:32.6189265Z * [new branch] gh/bobrenjc93/688/base -> origin/gh/bobrenjc93/688/base 2025-12-04T09:23:32.6189412Z * [new branch] gh/bobrenjc93/688/head -> origin/gh/bobrenjc93/688/head 2025-12-04T09:23:32.6189901Z * [new branch] gh/bobrenjc93/688/orig -> origin/gh/bobrenjc93/688/orig 2025-12-04T09:23:32.6190397Z * [new branch] gh/bobrenjc93/689/base -> origin/gh/bobrenjc93/689/base 2025-12-04T09:23:32.6193639Z * [new branch] gh/bobrenjc93/689/head -> origin/gh/bobrenjc93/689/head 2025-12-04T09:23:32.6193846Z * [new branch] gh/bobrenjc93/689/orig -> origin/gh/bobrenjc93/689/orig 2025-12-04T09:23:32.6194006Z * [new branch] gh/bobrenjc93/690/base -> origin/gh/bobrenjc93/690/base 2025-12-04T09:23:32.6194178Z * [new branch] gh/bobrenjc93/690/head -> origin/gh/bobrenjc93/690/head 2025-12-04T09:23:32.6194378Z * [new branch] gh/bobrenjc93/690/orig -> origin/gh/bobrenjc93/690/orig 2025-12-04T09:23:32.6198529Z * [new branch] gh/bobrenjc93/691/base -> origin/gh/bobrenjc93/691/base 2025-12-04T09:23:32.6198728Z * [new branch] gh/bobrenjc93/691/head -> origin/gh/bobrenjc93/691/head 2025-12-04T09:23:32.6199287Z * [new branch] gh/bobrenjc93/691/orig -> origin/gh/bobrenjc93/691/orig 2025-12-04T09:23:32.6199502Z * [new branch] gh/bobrenjc93/692/base -> origin/gh/bobrenjc93/692/base 2025-12-04T09:23:32.6203931Z * [new branch] gh/bobrenjc93/692/head -> origin/gh/bobrenjc93/692/head 2025-12-04T09:23:32.6204206Z * [new branch] gh/bobrenjc93/692/orig -> origin/gh/bobrenjc93/692/orig 2025-12-04T09:23:32.6204416Z * [new branch] gh/bobrenjc93/693/base -> origin/gh/bobrenjc93/693/base 2025-12-04T09:23:32.6204693Z * [new branch] gh/bobrenjc93/693/head -> origin/gh/bobrenjc93/693/head 2025-12-04T09:23:32.6204864Z * [new branch] gh/bobrenjc93/693/orig -> origin/gh/bobrenjc93/693/orig 2025-12-04T09:23:32.6205072Z * [new branch] gh/bobrenjc93/694/base -> origin/gh/bobrenjc93/694/base 2025-12-04T09:23:32.6205241Z * [new branch] gh/bobrenjc93/694/head -> origin/gh/bobrenjc93/694/head 2025-12-04T09:23:32.6208811Z * [new branch] gh/bobrenjc93/694/orig -> origin/gh/bobrenjc93/694/orig 2025-12-04T09:23:32.6209084Z * [new branch] gh/bobrenjc93/695/base -> origin/gh/bobrenjc93/695/base 2025-12-04T09:23:32.6209257Z * [new branch] gh/bobrenjc93/695/head -> origin/gh/bobrenjc93/695/head 2025-12-04T09:23:32.6209420Z * [new branch] gh/bobrenjc93/695/orig -> origin/gh/bobrenjc93/695/orig 2025-12-04T09:23:32.6209704Z * [new branch] gh/c00w/23/base -> origin/gh/c00w/23/base 2025-12-04T09:23:32.6209858Z * [new branch] gh/c00w/23/head -> origin/gh/c00w/23/head 2025-12-04T09:23:32.6215420Z * [new branch] gh/c00w/53/base -> origin/gh/c00w/53/base 2025-12-04T09:23:32.6215596Z * [new branch] gh/c00w/53/head -> origin/gh/c00w/53/head 2025-12-04T09:23:32.6215742Z * [new branch] gh/c00w/53/orig -> origin/gh/c00w/53/orig 2025-12-04T09:23:32.6215869Z * [new branch] gh/c00w/54/base -> origin/gh/c00w/54/base 2025-12-04T09:23:32.6216304Z * [new branch] gh/c00w/54/head -> origin/gh/c00w/54/head 2025-12-04T09:23:32.6216433Z * [new branch] gh/c00w/54/orig -> origin/gh/c00w/54/orig 2025-12-04T09:23:32.6221294Z * [new branch] gh/c00w/56/base -> origin/gh/c00w/56/base 2025-12-04T09:23:32.6221652Z * [new branch] gh/c00w/56/head -> origin/gh/c00w/56/head 2025-12-04T09:23:32.6221781Z * [new branch] gh/c00w/56/orig -> origin/gh/c00w/56/orig 2025-12-04T09:23:32.6221909Z * [new branch] gh/c00w/57/base -> origin/gh/c00w/57/base 2025-12-04T09:23:32.6222024Z * [new branch] gh/c00w/57/head -> origin/gh/c00w/57/head 2025-12-04T09:23:32.6222137Z * [new branch] gh/c00w/57/orig -> origin/gh/c00w/57/orig 2025-12-04T09:23:32.6222265Z * [new branch] gh/c00w/58/base -> origin/gh/c00w/58/base 2025-12-04T09:23:32.6222384Z * [new branch] gh/c00w/58/head -> origin/gh/c00w/58/head 2025-12-04T09:23:32.6222507Z * [new branch] gh/c00w/58/orig -> origin/gh/c00w/58/orig 2025-12-04T09:23:32.6224198Z * [new branch] gh/clee2000/1/base -> origin/gh/clee2000/1/base 2025-12-04T09:23:32.6224371Z * [new branch] gh/clee2000/1/head -> origin/gh/clee2000/1/head 2025-12-04T09:23:32.6224510Z * [new branch] gh/clee2000/1/orig -> origin/gh/clee2000/1/orig 2025-12-04T09:23:32.6224673Z * [new branch] gh/coconutruben/1/base -> origin/gh/coconutruben/1/base 2025-12-04T09:23:32.6224828Z * [new branch] gh/coconutruben/1/head -> origin/gh/coconutruben/1/head 2025-12-04T09:23:32.6230336Z * [new branch] gh/coconutruben/55/base -> origin/gh/coconutruben/55/base 2025-12-04T09:23:32.6230526Z * [new branch] gh/coconutruben/55/head -> origin/gh/coconutruben/55/head 2025-12-04T09:23:32.6230685Z * [new branch] gh/coconutruben/55/orig -> origin/gh/coconutruben/55/orig 2025-12-04T09:23:32.6230827Z * [new branch] gh/coconutruben/57/base -> origin/gh/coconutruben/57/base 2025-12-04T09:23:32.6230973Z * [new branch] gh/coconutruben/57/head -> origin/gh/coconutruben/57/head 2025-12-04T09:23:32.6231120Z * [new branch] gh/coconutruben/57/orig -> origin/gh/coconutruben/57/orig 2025-12-04T09:23:32.6231257Z * [new branch] gh/coconutruben/70/base -> origin/gh/coconutruben/70/base 2025-12-04T09:23:32.6231403Z * [new branch] gh/coconutruben/70/head -> origin/gh/coconutruben/70/head 2025-12-04T09:23:32.6231691Z * [new branch] gh/coconutruben/70/orig -> origin/gh/coconutruben/70/orig 2025-12-04T09:23:32.6231999Z * [new branch] gh/coconutruben/71/base -> origin/gh/coconutruben/71/base 2025-12-04T09:23:32.6232514Z * [new branch] gh/coconutruben/71/head -> origin/gh/coconutruben/71/head 2025-12-04T09:23:32.6234060Z * [new branch] gh/coconutruben/71/orig -> origin/gh/coconutruben/71/orig 2025-12-04T09:23:32.6245500Z * [new branch] gh/coconutruben/72/base -> origin/gh/coconutruben/72/base 2025-12-04T09:23:32.6247674Z * [new branch] gh/coconutruben/72/head -> origin/gh/coconutruben/72/head 2025-12-04T09:23:32.6247944Z * [new branch] gh/coconutruben/72/orig -> origin/gh/coconutruben/72/orig 2025-12-04T09:23:32.6251725Z * [new branch] gh/coconutruben/73/base -> origin/gh/coconutruben/73/base 2025-12-04T09:23:32.6251908Z * [new branch] gh/coconutruben/73/head -> origin/gh/coconutruben/73/head 2025-12-04T09:23:32.6252059Z * [new branch] gh/coconutruben/73/orig -> origin/gh/coconutruben/73/orig 2025-12-04T09:23:32.6252213Z * [new branch] gh/coconutruben/74/base -> origin/gh/coconutruben/74/base 2025-12-04T09:23:32.6252548Z * [new branch] gh/coconutruben/74/head -> origin/gh/coconutruben/74/head 2025-12-04T09:23:32.6252696Z * [new branch] gh/coconutruben/74/orig -> origin/gh/coconutruben/74/orig 2025-12-04T09:23:32.6252849Z * [new branch] gh/coconutruben/79/base -> origin/gh/coconutruben/79/base 2025-12-04T09:23:32.6253047Z * [new branch] gh/coconutruben/79/head -> origin/gh/coconutruben/79/head 2025-12-04T09:23:32.6253217Z * [new branch] gh/coconutruben/79/orig -> origin/gh/coconutruben/79/orig 2025-12-04T09:23:32.6253767Z * [new branch] gh/coconutruben/80/base -> origin/gh/coconutruben/80/base 2025-12-04T09:23:32.6259946Z * [new branch] gh/coconutruben/80/head -> origin/gh/coconutruben/80/head 2025-12-04T09:23:32.6265265Z * [new branch] gh/coconutruben/80/orig -> origin/gh/coconutruben/80/orig 2025-12-04T09:23:32.6267672Z * [new branch] gh/coconutruben/82/base -> origin/gh/coconutruben/82/base 2025-12-04T09:23:32.6267912Z * [new branch] gh/coconutruben/82/head -> origin/gh/coconutruben/82/head 2025-12-04T09:23:32.6268086Z * [new branch] gh/coconutruben/82/orig -> origin/gh/coconutruben/82/orig 2025-12-04T09:23:32.6268264Z * [new branch] gh/coconutruben/83/base -> origin/gh/coconutruben/83/base 2025-12-04T09:23:32.6268460Z * [new branch] gh/coconutruben/83/head -> origin/gh/coconutruben/83/head 2025-12-04T09:23:32.6268631Z * [new branch] gh/coconutruben/83/orig -> origin/gh/coconutruben/83/orig 2025-12-04T09:23:32.6268793Z * [new branch] gh/coconutruben/84/base -> origin/gh/coconutruben/84/base 2025-12-04T09:23:32.6268962Z * [new branch] gh/coconutruben/84/head -> origin/gh/coconutruben/84/head 2025-12-04T09:23:32.6269146Z * [new branch] gh/coconutruben/84/orig -> origin/gh/coconutruben/84/orig 2025-12-04T09:23:32.6269316Z * [new branch] gh/coconutruben/85/base -> origin/gh/coconutruben/85/base 2025-12-04T09:23:32.6269518Z * [new branch] gh/coconutruben/85/head -> origin/gh/coconutruben/85/head 2025-12-04T09:23:32.6269683Z * [new branch] gh/coconutruben/85/orig -> origin/gh/coconutruben/85/orig 2025-12-04T09:23:32.6269851Z * [new branch] gh/coconutruben/86/base -> origin/gh/coconutruben/86/base 2025-12-04T09:23:32.6270026Z * [new branch] gh/coconutruben/86/head -> origin/gh/coconutruben/86/head 2025-12-04T09:23:32.6270216Z * [new branch] gh/coconutruben/86/orig -> origin/gh/coconutruben/86/orig 2025-12-04T09:23:32.6270383Z * [new branch] gh/colinchan15/1/base -> origin/gh/colinchan15/1/base 2025-12-04T09:23:32.6270534Z * [new branch] gh/colinchan15/1/head -> origin/gh/colinchan15/1/head 2025-12-04T09:23:32.6270679Z * [new branch] gh/colinchan15/2/base -> origin/gh/colinchan15/2/base 2025-12-04T09:23:32.6270852Z * [new branch] gh/colinchan15/2/head -> origin/gh/colinchan15/2/head 2025-12-04T09:23:32.6270991Z * [new branch] gh/colinchan15/3/base -> origin/gh/colinchan15/3/base 2025-12-04T09:23:32.6271138Z * [new branch] gh/colinchan15/3/head -> origin/gh/colinchan15/3/head 2025-12-04T09:23:32.6271283Z * [new branch] gh/colinchan15/6/base -> origin/gh/colinchan15/6/base 2025-12-04T09:23:32.6271422Z * [new branch] gh/colinchan15/6/head -> origin/gh/colinchan15/6/head 2025-12-04T09:23:32.6271565Z * [new branch] gh/d4l3k/1/base -> origin/gh/d4l3k/1/base 2025-12-04T09:23:32.6271692Z * [new branch] gh/d4l3k/1/head -> origin/gh/d4l3k/1/head 2025-12-04T09:23:32.6271826Z * [new branch] gh/d4l3k/2/base -> origin/gh/d4l3k/2/base 2025-12-04T09:23:32.6271963Z * [new branch] gh/d4l3k/2/head -> origin/gh/d4l3k/2/head 2025-12-04T09:23:32.6272243Z * [new branch] gh/d4l3k/2/orig -> origin/gh/d4l3k/2/orig 2025-12-04T09:23:32.6272383Z * [new branch] gh/d4l3k/3/base -> origin/gh/d4l3k/3/base 2025-12-04T09:23:32.6272510Z * [new branch] gh/d4l3k/3/head -> origin/gh/d4l3k/3/head 2025-12-04T09:23:32.6272857Z * [new branch] gh/d4l3k/3/orig -> origin/gh/d4l3k/3/orig 2025-12-04T09:23:32.6273003Z * [new branch] gh/d4l3k/4/base -> origin/gh/d4l3k/4/base 2025-12-04T09:23:32.6273128Z * [new branch] gh/d4l3k/4/head -> origin/gh/d4l3k/4/head 2025-12-04T09:23:32.6273265Z * [new branch] gh/d4l3k/4/orig -> origin/gh/d4l3k/4/orig 2025-12-04T09:23:32.6273388Z * [new branch] gh/d4l3k/5/base -> origin/gh/d4l3k/5/base 2025-12-04T09:23:32.6273513Z * [new branch] gh/d4l3k/5/orig -> origin/gh/d4l3k/5/orig 2025-12-04T09:23:32.6273701Z * [new branch] gh/davidberard98/392/base -> origin/gh/davidberard98/392/base 2025-12-04T09:23:32.6277460Z * [new branch] gh/davidberard98/392/head -> origin/gh/davidberard98/392/head 2025-12-04T09:23:32.6277719Z * [new branch] gh/davidberard98/392/orig -> origin/gh/davidberard98/392/orig 2025-12-04T09:23:32.6277933Z * [new branch] gh/davidberard98/399/base -> origin/gh/davidberard98/399/base 2025-12-04T09:23:32.6284885Z * [new branch] gh/davidberard98/399/head -> origin/gh/davidberard98/399/head 2025-12-04T09:23:32.6285102Z * [new branch] gh/davidberard98/399/orig -> origin/gh/davidberard98/399/orig 2025-12-04T09:23:32.6285287Z * [new branch] gh/desertfire/605/base -> origin/gh/desertfire/605/base 2025-12-04T09:23:32.6285472Z * [new branch] gh/desertfire/605/head -> origin/gh/desertfire/605/head 2025-12-04T09:23:32.6285655Z * [new branch] gh/desertfire/605/orig -> origin/gh/desertfire/605/orig 2025-12-04T09:23:32.6285820Z * [new branch] gh/desertfire/606/base -> origin/gh/desertfire/606/base 2025-12-04T09:23:32.6285974Z * [new branch] gh/desertfire/606/head -> origin/gh/desertfire/606/head 2025-12-04T09:23:32.6286157Z * [new branch] gh/desertfire/606/orig -> origin/gh/desertfire/606/orig 2025-12-04T09:23:32.6286311Z * [new branch] gh/desertfire/607/base -> origin/gh/desertfire/607/base 2025-12-04T09:23:32.6286463Z * [new branch] gh/desertfire/607/head -> origin/gh/desertfire/607/head 2025-12-04T09:23:32.6286620Z * [new branch] gh/desertfire/607/orig -> origin/gh/desertfire/607/orig 2025-12-04T09:23:32.6286772Z * [new branch] gh/desertfire/608/base -> origin/gh/desertfire/608/base 2025-12-04T09:23:32.6286922Z * [new branch] gh/desertfire/608/head -> origin/gh/desertfire/608/head 2025-12-04T09:23:32.6287085Z * [new branch] gh/desertfire/608/orig -> origin/gh/desertfire/608/orig 2025-12-04T09:23:32.6291602Z * [new branch] gh/desertfire/609/base -> origin/gh/desertfire/609/base 2025-12-04T09:23:32.6291808Z * [new branch] gh/desertfire/609/head -> origin/gh/desertfire/609/head 2025-12-04T09:23:32.6291986Z * [new branch] gh/desertfire/609/orig -> origin/gh/desertfire/609/orig 2025-12-04T09:23:32.6292150Z * [new branch] gh/desertfire/610/base -> origin/gh/desertfire/610/base 2025-12-04T09:23:32.6292307Z * [new branch] gh/desertfire/610/head -> origin/gh/desertfire/610/head 2025-12-04T09:23:32.6292461Z * [new branch] gh/desertfire/610/orig -> origin/gh/desertfire/610/orig 2025-12-04T09:23:32.6296308Z * [new branch] gh/desertfire/611/base -> origin/gh/desertfire/611/base 2025-12-04T09:23:32.6296669Z * [new branch] gh/desertfire/611/head -> origin/gh/desertfire/611/head 2025-12-04T09:23:32.6296830Z * [new branch] gh/desertfire/611/orig -> origin/gh/desertfire/611/orig 2025-12-04T09:23:32.6296989Z * [new branch] gh/desertfire/612/base -> origin/gh/desertfire/612/base 2025-12-04T09:23:32.6297208Z * [new branch] gh/desertfire/612/head -> origin/gh/desertfire/612/head 2025-12-04T09:23:32.6297372Z * [new branch] gh/desertfire/612/orig -> origin/gh/desertfire/612/orig 2025-12-04T09:23:32.6301698Z * [new branch] gh/desertfire/613/base -> origin/gh/desertfire/613/base 2025-12-04T09:23:32.6301892Z * [new branch] gh/desertfire/613/head -> origin/gh/desertfire/613/head 2025-12-04T09:23:32.6302041Z * [new branch] gh/desertfire/613/orig -> origin/gh/desertfire/613/orig 2025-12-04T09:23:32.6302183Z * [new branch] gh/desertfire/614/base -> origin/gh/desertfire/614/base 2025-12-04T09:23:32.6302346Z * [new branch] gh/desertfire/614/head -> origin/gh/desertfire/614/head 2025-12-04T09:23:32.6302486Z * [new branch] gh/desertfire/614/orig -> origin/gh/desertfire/614/orig 2025-12-04T09:23:32.6302633Z * [new branch] gh/desertfire/615/base -> origin/gh/desertfire/615/base 2025-12-04T09:23:32.6302779Z * [new branch] gh/desertfire/615/head -> origin/gh/desertfire/615/head 2025-12-04T09:23:32.6302916Z * [new branch] gh/desertfire/615/orig -> origin/gh/desertfire/615/orig 2025-12-04T09:23:32.6306819Z * [new branch] gh/desertfire/616/base -> origin/gh/desertfire/616/base 2025-12-04T09:23:32.6307504Z * [new branch] gh/desertfire/616/head -> origin/gh/desertfire/616/head 2025-12-04T09:23:32.6307682Z * [new branch] gh/desertfire/616/orig -> origin/gh/desertfire/616/orig 2025-12-04T09:23:32.6307969Z * [new branch] gh/desertfire/617/base -> origin/gh/desertfire/617/base 2025-12-04T09:23:32.6308140Z * [new branch] gh/desertfire/617/head -> origin/gh/desertfire/617/head 2025-12-04T09:23:32.6308290Z * [new branch] gh/desertfire/617/orig -> origin/gh/desertfire/617/orig 2025-12-04T09:23:32.6308449Z * [new branch] gh/dharakk/1/base -> origin/gh/dharakk/1/base 2025-12-04T09:23:32.6308601Z * [new branch] gh/dharakk/1/head -> origin/gh/dharakk/1/head 2025-12-04T09:23:32.6308754Z * [new branch] gh/drisspg/170/base -> origin/gh/drisspg/170/base 2025-12-04T09:23:32.6309183Z * [new branch] gh/drisspg/170/head -> origin/gh/drisspg/170/head 2025-12-04T09:23:32.6310177Z * [new branch] gh/drisspg/170/orig -> origin/gh/drisspg/170/orig 2025-12-04T09:23:32.6310924Z * [new branch] gh/drisspg/182/base -> origin/gh/drisspg/182/base 2025-12-04T09:23:32.6311518Z * [new branch] gh/drisspg/182/head -> origin/gh/drisspg/182/head 2025-12-04T09:23:32.6312838Z * [new branch] gh/drisspg/183/base -> origin/gh/drisspg/183/base 2025-12-04T09:23:32.6313101Z * [new branch] gh/drisspg/183/head -> origin/gh/drisspg/183/head 2025-12-04T09:23:32.6314494Z * [new branch] gh/drisspg/184/base -> origin/gh/drisspg/184/base 2025-12-04T09:23:32.6314644Z * [new branch] gh/drisspg/184/head -> origin/gh/drisspg/184/head 2025-12-04T09:23:32.6316950Z * [new branch] gh/drisspg/185/base -> origin/gh/drisspg/185/base 2025-12-04T09:23:32.6317132Z * [new branch] gh/drisspg/185/head -> origin/gh/drisspg/185/head 2025-12-04T09:23:32.6317516Z * [new branch] gh/drisspg/194/base -> origin/gh/drisspg/194/base 2025-12-04T09:23:32.6318127Z * [new branch] gh/drisspg/194/head -> origin/gh/drisspg/194/head 2025-12-04T09:23:32.6318865Z * [new branch] gh/drisspg/194/orig -> origin/gh/drisspg/194/orig 2025-12-04T09:23:32.6324078Z * [new branch] gh/drisspg/200/base -> origin/gh/drisspg/200/base 2025-12-04T09:23:32.6324250Z * [new branch] gh/drisspg/200/head -> origin/gh/drisspg/200/head 2025-12-04T09:23:32.6324596Z * [new branch] gh/drisspg/200/orig -> origin/gh/drisspg/200/orig 2025-12-04T09:23:32.6324746Z * [new branch] gh/drisspg/218/base -> origin/gh/drisspg/218/base 2025-12-04T09:23:32.6324879Z * [new branch] gh/drisspg/218/head -> origin/gh/drisspg/218/head 2025-12-04T09:23:32.6325020Z * [new branch] gh/drisspg/218/orig -> origin/gh/drisspg/218/orig 2025-12-04T09:23:32.6325156Z * [new branch] gh/drisspg/219/base -> origin/gh/drisspg/219/base 2025-12-04T09:23:32.6325587Z * [new branch] gh/drisspg/219/head -> origin/gh/drisspg/219/head 2025-12-04T09:23:32.6326945Z * [new branch] gh/drisspg/219/orig -> origin/gh/drisspg/219/orig 2025-12-04T09:23:32.6327254Z * [new branch] gh/drisspg/220/base -> origin/gh/drisspg/220/base 2025-12-04T09:23:32.6333556Z * [new branch] gh/drisspg/220/head -> origin/gh/drisspg/220/head 2025-12-04T09:23:32.6335639Z * [new branch] gh/drisspg/220/orig -> origin/gh/drisspg/220/orig 2025-12-04T09:23:32.6335884Z * [new branch] gh/drisspg/221/base -> origin/gh/drisspg/221/base 2025-12-04T09:23:32.6336237Z * [new branch] gh/drisspg/221/head -> origin/gh/drisspg/221/head 2025-12-04T09:23:32.6336435Z * [new branch] gh/drisspg/221/orig -> origin/gh/drisspg/221/orig 2025-12-04T09:23:32.6336601Z * [new branch] gh/drisspg/222/base -> origin/gh/drisspg/222/base 2025-12-04T09:23:32.6336765Z * [new branch] gh/drisspg/222/head -> origin/gh/drisspg/222/head 2025-12-04T09:23:32.6336920Z * [new branch] gh/drisspg/222/orig -> origin/gh/drisspg/222/orig 2025-12-04T09:23:32.6337105Z * [new branch] gh/drisspg/223/base -> origin/gh/drisspg/223/base 2025-12-04T09:23:32.6337334Z * [new branch] gh/drisspg/223/head -> origin/gh/drisspg/223/head 2025-12-04T09:23:32.6337991Z * [new branch] gh/drisspg/223/orig -> origin/gh/drisspg/223/orig 2025-12-04T09:23:32.6338177Z * [new branch] gh/drisspg/224/base -> origin/gh/drisspg/224/base 2025-12-04T09:23:32.6338485Z * [new branch] gh/drisspg/224/head -> origin/gh/drisspg/224/head 2025-12-04T09:23:32.6341122Z * [new branch] gh/drisspg/224/orig -> origin/gh/drisspg/224/orig 2025-12-04T09:23:32.6343781Z * [new branch] gh/drisspg/225/base -> origin/gh/drisspg/225/base 2025-12-04T09:23:32.6343969Z * [new branch] gh/drisspg/225/head -> origin/gh/drisspg/225/head 2025-12-04T09:23:32.6344129Z * [new branch] gh/drisspg/225/orig -> origin/gh/drisspg/225/orig 2025-12-04T09:23:32.6344264Z * [new branch] gh/drisspg/226/base -> origin/gh/drisspg/226/base 2025-12-04T09:23:32.6344406Z * [new branch] gh/drisspg/226/head -> origin/gh/drisspg/226/head 2025-12-04T09:23:32.6344545Z * [new branch] gh/drisspg/226/orig -> origin/gh/drisspg/226/orig 2025-12-04T09:23:32.6344686Z * [new branch] gh/drisspg/227/base -> origin/gh/drisspg/227/base 2025-12-04T09:23:32.6344818Z * [new branch] gh/drisspg/227/head -> origin/gh/drisspg/227/head 2025-12-04T09:23:32.6344955Z * [new branch] gh/drisspg/227/orig -> origin/gh/drisspg/227/orig 2025-12-04T09:23:32.6345359Z * [new branch] gh/drisspg/228/base -> origin/gh/drisspg/228/base 2025-12-04T09:23:32.6345928Z * [new branch] gh/drisspg/228/head -> origin/gh/drisspg/228/head 2025-12-04T09:23:32.6346632Z * [new branch] gh/drisspg/228/orig -> origin/gh/drisspg/228/orig 2025-12-04T09:23:32.6347769Z * [new branch] gh/drisspg/229/base -> origin/gh/drisspg/229/base 2025-12-04T09:23:32.6348034Z * [new branch] gh/drisspg/229/head -> origin/gh/drisspg/229/head 2025-12-04T09:23:32.6349137Z * [new branch] gh/drisspg/229/orig -> origin/gh/drisspg/229/orig 2025-12-04T09:23:32.6349993Z * [new branch] gh/drisspg/230/base -> origin/gh/drisspg/230/base 2025-12-04T09:23:32.6350461Z * [new branch] gh/drisspg/230/head -> origin/gh/drisspg/230/head 2025-12-04T09:23:32.6351342Z * [new branch] gh/drisspg/230/orig -> origin/gh/drisspg/230/orig 2025-12-04T09:23:32.6352606Z * [new branch] gh/dsjohns2/1/base -> origin/gh/dsjohns2/1/base 2025-12-04T09:23:32.6353784Z * [new branch] gh/dsjohns2/1/head -> origin/gh/dsjohns2/1/head 2025-12-04T09:23:32.6355085Z * [new branch] gh/dzmitry-huba/1/base -> origin/gh/dzmitry-huba/1/base 2025-12-04T09:23:32.6358870Z * [new branch] gh/dzmitry-huba/1/head -> origin/gh/dzmitry-huba/1/head 2025-12-04T09:23:32.6359050Z * [new branch] gh/dzmitry-huba/12/base -> origin/gh/dzmitry-huba/12/base 2025-12-04T09:23:32.6359364Z * [new branch] gh/dzmitry-huba/12/head -> origin/gh/dzmitry-huba/12/head 2025-12-04T09:23:32.6359515Z * [new branch] gh/dzmitry-huba/12/orig -> origin/gh/dzmitry-huba/12/orig 2025-12-04T09:23:32.6359669Z * [new branch] gh/dzmitry-huba/13/base -> origin/gh/dzmitry-huba/13/base 2025-12-04T09:23:32.6359830Z * [new branch] gh/dzmitry-huba/13/head -> origin/gh/dzmitry-huba/13/head 2025-12-04T09:23:32.6365891Z * [new branch] gh/dzmitry-huba/13/orig -> origin/gh/dzmitry-huba/13/orig 2025-12-04T09:23:32.6366108Z * [new branch] gh/dzmitry-huba/14/base -> origin/gh/dzmitry-huba/14/base 2025-12-04T09:23:32.6366277Z * [new branch] gh/dzmitry-huba/14/head -> origin/gh/dzmitry-huba/14/head 2025-12-04T09:23:32.6366438Z * [new branch] gh/dzmitry-huba/14/orig -> origin/gh/dzmitry-huba/14/orig 2025-12-04T09:23:32.6366616Z * [new branch] gh/dzmitry-huba/15/base -> origin/gh/dzmitry-huba/15/base 2025-12-04T09:23:32.6366778Z * [new branch] gh/dzmitry-huba/15/head -> origin/gh/dzmitry-huba/15/head 2025-12-04T09:23:32.6366930Z * [new branch] gh/dzmitry-huba/15/orig -> origin/gh/dzmitry-huba/15/orig 2025-12-04T09:23:32.6367448Z * [new branch] gh/dzmitry-huba/16/base -> origin/gh/dzmitry-huba/16/base 2025-12-04T09:23:32.6367591Z * [new branch] gh/dzmitry-huba/16/head -> origin/gh/dzmitry-huba/16/head 2025-12-04T09:23:32.6367729Z * [new branch] gh/dzmitry-huba/16/orig -> origin/gh/dzmitry-huba/16/orig 2025-12-04T09:23:32.6368036Z * [new branch] gh/dzmitry-huba/17/base -> origin/gh/dzmitry-huba/17/base 2025-12-04T09:23:32.6368178Z * [new branch] gh/dzmitry-huba/17/head -> origin/gh/dzmitry-huba/17/head 2025-12-04T09:23:32.6373789Z * [new branch] gh/dzmitry-huba/17/orig -> origin/gh/dzmitry-huba/17/orig 2025-12-04T09:23:32.6374135Z * [new branch] gh/dzmitry-huba/2/base -> origin/gh/dzmitry-huba/2/base 2025-12-04T09:23:32.6374307Z * [new branch] gh/dzmitry-huba/2/head -> origin/gh/dzmitry-huba/2/head 2025-12-04T09:23:32.6374530Z * [new branch] gh/dzmitry-huba/3/base -> origin/gh/dzmitry-huba/3/base 2025-12-04T09:23:32.6374689Z * [new branch] gh/dzmitry-huba/3/head -> origin/gh/dzmitry-huba/3/head 2025-12-04T09:23:32.6374863Z * [new branch] gh/eellison/808/base -> origin/gh/eellison/808/base 2025-12-04T09:23:32.6379166Z * [new branch] gh/eellison/808/head -> origin/gh/eellison/808/head 2025-12-04T09:23:32.6379332Z * [new branch] gh/eellison/808/orig -> origin/gh/eellison/808/orig 2025-12-04T09:23:32.6379487Z * [new branch] gh/eellison/822/base -> origin/gh/eellison/822/base 2025-12-04T09:23:32.6379717Z * [new branch] gh/eellison/822/head -> origin/gh/eellison/822/head 2025-12-04T09:23:32.6379961Z * [new branch] gh/eellison/822/orig -> origin/gh/eellison/822/orig 2025-12-04T09:23:32.6380116Z * [new branch] gh/eellison/823/base -> origin/gh/eellison/823/base 2025-12-04T09:23:32.6380273Z * [new branch] gh/eellison/823/head -> origin/gh/eellison/823/head 2025-12-04T09:23:32.6383853Z * [new branch] gh/eellison/823/orig -> origin/gh/eellison/823/orig 2025-12-04T09:23:32.6384007Z * [new branch] gh/eellison/862/base -> origin/gh/eellison/862/base 2025-12-04T09:23:32.6384163Z * [new branch] gh/eellison/862/head -> origin/gh/eellison/862/head 2025-12-04T09:23:32.6384351Z * [new branch] gh/eellison/862/orig -> origin/gh/eellison/862/orig 2025-12-04T09:23:32.6384495Z * [new branch] gh/eellison/863/base -> origin/gh/eellison/863/base 2025-12-04T09:23:32.6384640Z * [new branch] gh/eellison/863/head -> origin/gh/eellison/863/head 2025-12-04T09:23:32.6390795Z * [new branch] gh/eellison/863/orig -> origin/gh/eellison/863/orig 2025-12-04T09:23:32.6395744Z * [new branch] gh/eellison/864/base -> origin/gh/eellison/864/base 2025-12-04T09:23:32.6399857Z * [new branch] gh/eellison/864/head -> origin/gh/eellison/864/head 2025-12-04T09:23:32.6404314Z * [new branch] gh/eellison/864/orig -> origin/gh/eellison/864/orig 2025-12-04T09:23:32.6404519Z * [new branch] gh/eellison/865/base -> origin/gh/eellison/865/base 2025-12-04T09:23:32.6404678Z * [new branch] gh/eellison/865/head -> origin/gh/eellison/865/head 2025-12-04T09:23:32.6404825Z * [new branch] gh/eellison/865/orig -> origin/gh/eellison/865/orig 2025-12-04T09:23:32.6404966Z * [new branch] gh/eellison/866/base -> origin/gh/eellison/866/base 2025-12-04T09:23:32.6405123Z * [new branch] gh/eellison/866/head -> origin/gh/eellison/866/head 2025-12-04T09:23:32.6405269Z * [new branch] gh/eellison/866/orig -> origin/gh/eellison/866/orig 2025-12-04T09:23:32.6405403Z * [new branch] gh/eellison/867/base -> origin/gh/eellison/867/base 2025-12-04T09:23:32.6405549Z * [new branch] gh/eellison/867/head -> origin/gh/eellison/867/head 2025-12-04T09:23:32.6405837Z * [new branch] gh/eellison/867/orig -> origin/gh/eellison/867/orig 2025-12-04T09:23:32.6405987Z * [new branch] gh/eellison/868/base -> origin/gh/eellison/868/base 2025-12-04T09:23:32.6406130Z * [new branch] gh/eellison/868/head -> origin/gh/eellison/868/head 2025-12-04T09:23:32.6406262Z * [new branch] gh/eellison/868/orig -> origin/gh/eellison/868/orig 2025-12-04T09:23:32.6406397Z * [new branch] gh/eellison/869/base -> origin/gh/eellison/869/base 2025-12-04T09:23:32.6406543Z * [new branch] gh/eellison/869/head -> origin/gh/eellison/869/head 2025-12-04T09:23:32.6406674Z * [new branch] gh/eellison/869/orig -> origin/gh/eellison/869/orig 2025-12-04T09:23:32.6406816Z * [new branch] gh/eellison/870/base -> origin/gh/eellison/870/base 2025-12-04T09:23:32.6406949Z * [new branch] gh/eellison/870/head -> origin/gh/eellison/870/head 2025-12-04T09:23:32.6407083Z * [new branch] gh/eellison/870/orig -> origin/gh/eellison/870/orig 2025-12-04T09:23:32.6407435Z * [new branch] gh/eellison/871/base -> origin/gh/eellison/871/base 2025-12-04T09:23:32.6407569Z * [new branch] gh/eellison/871/head -> origin/gh/eellison/871/head 2025-12-04T09:23:32.6407714Z * [new branch] gh/eellison/871/orig -> origin/gh/eellison/871/orig 2025-12-04T09:23:32.6407908Z * [new branch] gh/eellison/872/base -> origin/gh/eellison/872/base 2025-12-04T09:23:32.6408045Z * [new branch] gh/eellison/872/head -> origin/gh/eellison/872/head 2025-12-04T09:23:32.6408186Z * [new branch] gh/eellison/872/orig -> origin/gh/eellison/872/orig 2025-12-04T09:23:32.6408931Z * [new branch] gh/eellison/873/base -> origin/gh/eellison/873/base 2025-12-04T09:23:32.6409078Z * [new branch] gh/eellison/873/head -> origin/gh/eellison/873/head 2025-12-04T09:23:32.6409212Z * [new branch] gh/eellison/873/orig -> origin/gh/eellison/873/orig 2025-12-04T09:23:32.6409353Z * [new branch] gh/eellison/874/base -> origin/gh/eellison/874/base 2025-12-04T09:23:32.6409497Z * [new branch] gh/eellison/874/head -> origin/gh/eellison/874/head 2025-12-04T09:23:32.6409634Z * [new branch] gh/eellison/874/orig -> origin/gh/eellison/874/orig 2025-12-04T09:23:32.6416061Z * [new branch] gh/eellison/875/base -> origin/gh/eellison/875/base 2025-12-04T09:23:32.6418238Z * [new branch] gh/eellison/875/head -> origin/gh/eellison/875/head 2025-12-04T09:23:32.6418487Z * [new branch] gh/eellison/875/orig -> origin/gh/eellison/875/orig 2025-12-04T09:23:32.6423443Z * [new branch] gh/eellison/876/base -> origin/gh/eellison/876/base 2025-12-04T09:23:32.6426168Z * [new branch] gh/eellison/876/head -> origin/gh/eellison/876/head 2025-12-04T09:23:32.6426459Z * [new branch] gh/eellison/876/orig -> origin/gh/eellison/876/orig 2025-12-04T09:23:32.6426703Z * [new branch] gh/eellison/877/base -> origin/gh/eellison/877/base 2025-12-04T09:23:32.6426856Z * [new branch] gh/eellison/877/head -> origin/gh/eellison/877/head 2025-12-04T09:23:32.6427004Z * [new branch] gh/eellison/877/orig -> origin/gh/eellison/877/orig 2025-12-04T09:23:32.6427167Z * [new branch] gh/eellison/878/base -> origin/gh/eellison/878/base 2025-12-04T09:23:32.6427310Z * [new branch] gh/eellison/878/head -> origin/gh/eellison/878/head 2025-12-04T09:23:32.6427452Z * [new branch] gh/eellison/878/orig -> origin/gh/eellison/878/orig 2025-12-04T09:23:32.6427602Z * [new branch] gh/eellison/879/base -> origin/gh/eellison/879/base 2025-12-04T09:23:32.6428167Z * [new branch] gh/eellison/879/head -> origin/gh/eellison/879/head 2025-12-04T09:23:32.6428547Z * [new branch] gh/eellison/879/orig -> origin/gh/eellison/879/orig 2025-12-04T09:23:32.6428702Z * [new branch] gh/eellison/880/base -> origin/gh/eellison/880/base 2025-12-04T09:23:32.6429432Z * [new branch] gh/eellison/880/head -> origin/gh/eellison/880/head 2025-12-04T09:23:32.6429712Z * [new branch] gh/eellison/880/orig -> origin/gh/eellison/880/orig 2025-12-04T09:23:32.6430011Z * [new branch] gh/eellison/881/base -> origin/gh/eellison/881/base 2025-12-04T09:23:32.6430209Z * [new branch] gh/eellison/881/head -> origin/gh/eellison/881/head 2025-12-04T09:23:32.6430358Z * [new branch] gh/eellison/881/orig -> origin/gh/eellison/881/orig 2025-12-04T09:23:32.6430511Z * [new branch] gh/eellison/882/base -> origin/gh/eellison/882/base 2025-12-04T09:23:32.6430660Z * [new branch] gh/eellison/882/head -> origin/gh/eellison/882/head 2025-12-04T09:23:32.6431182Z * [new branch] gh/eellison/882/orig -> origin/gh/eellison/882/orig 2025-12-04T09:23:32.6431353Z * [new branch] gh/eellison/883/base -> origin/gh/eellison/883/base 2025-12-04T09:23:32.6431510Z * [new branch] gh/eellison/883/head -> origin/gh/eellison/883/head 2025-12-04T09:23:32.6431754Z * [new branch] gh/eellison/883/orig -> origin/gh/eellison/883/orig 2025-12-04T09:23:32.6431923Z * [new branch] gh/eellison/884/base -> origin/gh/eellison/884/base 2025-12-04T09:23:32.6432072Z * [new branch] gh/eellison/884/head -> origin/gh/eellison/884/head 2025-12-04T09:23:32.6432212Z * [new branch] gh/eellison/884/orig -> origin/gh/eellison/884/orig 2025-12-04T09:23:32.6432355Z * [new branch] gh/etaf/147/base -> origin/gh/etaf/147/base 2025-12-04T09:23:32.6432502Z * [new branch] gh/etaf/147/head -> origin/gh/etaf/147/head 2025-12-04T09:23:32.6432803Z * [new branch] gh/etaf/154/base -> origin/gh/etaf/154/base 2025-12-04T09:23:32.6432963Z * [new branch] gh/etaf/154/head -> origin/gh/etaf/154/head 2025-12-04T09:23:32.6433309Z * [new branch] gh/etaf/154/orig -> origin/gh/etaf/154/orig 2025-12-04T09:23:32.6437810Z * [new branch] gh/etaf/156/base -> origin/gh/etaf/156/base 2025-12-04T09:23:32.6438108Z * [new branch] gh/etaf/156/head -> origin/gh/etaf/156/head 2025-12-04T09:23:32.6443215Z * [new branch] gh/etaf/156/orig -> origin/gh/etaf/156/orig 2025-12-04T09:23:32.6448132Z * [new branch] gh/etaf/157/base -> origin/gh/etaf/157/base 2025-12-04T09:23:32.6450044Z * [new branch] gh/etaf/157/head -> origin/gh/etaf/157/head 2025-12-04T09:23:32.6450329Z * [new branch] gh/etaf/157/orig -> origin/gh/etaf/157/orig 2025-12-04T09:23:32.6450483Z * [new branch] gh/etaf/158/base -> origin/gh/etaf/158/base 2025-12-04T09:23:32.6450603Z * [new branch] gh/etaf/158/head -> origin/gh/etaf/158/head 2025-12-04T09:23:32.6450727Z * [new branch] gh/etaf/158/orig -> origin/gh/etaf/158/orig 2025-12-04T09:23:32.6450867Z * [new branch] gh/etaf/159/base -> origin/gh/etaf/159/base 2025-12-04T09:23:32.6450993Z * [new branch] gh/etaf/159/head -> origin/gh/etaf/159/head 2025-12-04T09:23:32.6451121Z * [new branch] gh/etaf/159/orig -> origin/gh/etaf/159/orig 2025-12-04T09:23:32.6451239Z * [new branch] gh/etaf/160/base -> origin/gh/etaf/160/base 2025-12-04T09:23:32.6451369Z * [new branch] gh/etaf/160/head -> origin/gh/etaf/160/head 2025-12-04T09:23:32.6451492Z * [new branch] gh/etaf/160/orig -> origin/gh/etaf/160/orig 2025-12-04T09:23:32.6451616Z * [new branch] gh/etaf/161/base -> origin/gh/etaf/161/base 2025-12-04T09:23:32.6451750Z * [new branch] gh/etaf/161/head -> origin/gh/etaf/161/head 2025-12-04T09:23:32.6451875Z * [new branch] gh/etaf/161/orig -> origin/gh/etaf/161/orig 2025-12-04T09:23:32.6452014Z * [new branch] gh/etaf/166/base -> origin/gh/etaf/166/base 2025-12-04T09:23:32.6452135Z * [new branch] gh/etaf/166/head -> origin/gh/etaf/166/head 2025-12-04T09:23:32.6452256Z * [new branch] gh/etaf/166/orig -> origin/gh/etaf/166/orig 2025-12-04T09:23:32.6452386Z * [new branch] gh/etaf/167/base -> origin/gh/etaf/167/base 2025-12-04T09:23:32.6454043Z * [new branch] gh/etaf/167/head -> origin/gh/etaf/167/head 2025-12-04T09:23:32.6454188Z * [new branch] gh/etaf/167/orig -> origin/gh/etaf/167/orig 2025-12-04T09:23:32.6454508Z * [new branch] gh/etaf/168/base -> origin/gh/etaf/168/base 2025-12-04T09:23:32.6454699Z * [new branch] gh/etaf/168/head -> origin/gh/etaf/168/head 2025-12-04T09:23:32.6454838Z * [new branch] gh/etaf/168/orig -> origin/gh/etaf/168/orig 2025-12-04T09:23:32.6461148Z * [new branch] gh/etaf/172/base -> origin/gh/etaf/172/base 2025-12-04T09:23:32.6466395Z * [new branch] gh/etaf/172/head -> origin/gh/etaf/172/head 2025-12-04T09:23:32.6468221Z * [new branch] gh/etaf/172/orig -> origin/gh/etaf/172/orig 2025-12-04T09:23:32.6468454Z * [new branch] gh/etaf/173/base -> origin/gh/etaf/173/base 2025-12-04T09:23:32.6468594Z * [new branch] gh/etaf/173/head -> origin/gh/etaf/173/head 2025-12-04T09:23:32.6468714Z * [new branch] gh/etaf/173/orig -> origin/gh/etaf/173/orig 2025-12-04T09:23:32.6468847Z * [new branch] gh/etaf/174/base -> origin/gh/etaf/174/base 2025-12-04T09:23:32.6468965Z * [new branch] gh/etaf/174/head -> origin/gh/etaf/174/head 2025-12-04T09:23:32.6469083Z * [new branch] gh/etaf/175/base -> origin/gh/etaf/175/base 2025-12-04T09:23:32.6469216Z * [new branch] gh/etaf/175/head -> origin/gh/etaf/175/head 2025-12-04T09:23:32.6469335Z * [new branch] gh/etaf/175/orig -> origin/gh/etaf/175/orig 2025-12-04T09:23:32.6469454Z * [new branch] gh/etaf/176/base -> origin/gh/etaf/176/base 2025-12-04T09:23:32.6469579Z * [new branch] gh/etaf/176/head -> origin/gh/etaf/176/head 2025-12-04T09:23:32.6469699Z * [new branch] gh/etaf/176/orig -> origin/gh/etaf/176/orig 2025-12-04T09:23:32.6469825Z * [new branch] gh/etaf/177/base -> origin/gh/etaf/177/base 2025-12-04T09:23:32.6469951Z * [new branch] gh/etaf/177/head -> origin/gh/etaf/177/head 2025-12-04T09:23:32.6470072Z * [new branch] gh/etaf/177/orig -> origin/gh/etaf/177/orig 2025-12-04T09:23:32.6470770Z * [new branch] gh/etaf/178/base -> origin/gh/etaf/178/base 2025-12-04T09:23:32.6471039Z * [new branch] gh/etaf/178/head -> origin/gh/etaf/178/head 2025-12-04T09:23:32.6471167Z * [new branch] gh/etaf/178/orig -> origin/gh/etaf/178/orig 2025-12-04T09:23:32.6472397Z * [new branch] gh/etaf/179/base -> origin/gh/etaf/179/base 2025-12-04T09:23:32.6472902Z * [new branch] gh/etaf/179/head -> origin/gh/etaf/179/head 2025-12-04T09:23:32.6476057Z * [new branch] gh/etaf/179/orig -> origin/gh/etaf/179/orig 2025-12-04T09:23:32.6476370Z * [new branch] gh/etaf/180/base -> origin/gh/etaf/180/base 2025-12-04T09:23:32.6476536Z * [new branch] gh/etaf/180/head -> origin/gh/etaf/180/head 2025-12-04T09:23:32.6476665Z * [new branch] gh/etaf/180/orig -> origin/gh/etaf/180/orig 2025-12-04T09:23:32.6477607Z * [new branch] gh/exclamaforte/1/base -> origin/gh/exclamaforte/1/base 2025-12-04T09:23:32.6477965Z * [new branch] gh/exclamaforte/1/head -> origin/gh/exclamaforte/1/head 2025-12-04T09:23:32.6479091Z * [new branch] gh/exclamaforte/2/base -> origin/gh/exclamaforte/2/base 2025-12-04T09:23:32.6479479Z * [new branch] gh/exclamaforte/2/head -> origin/gh/exclamaforte/2/head 2025-12-04T09:23:32.6480672Z * [new branch] gh/exclamaforte/3/base -> origin/gh/exclamaforte/3/base 2025-12-04T09:23:32.6481201Z * [new branch] gh/exclamaforte/3/head -> origin/gh/exclamaforte/3/head 2025-12-04T09:23:32.6482891Z * [new branch] gh/exclamaforte/4/base -> origin/gh/exclamaforte/4/base 2025-12-04T09:23:32.6483197Z * [new branch] gh/exclamaforte/4/head -> origin/gh/exclamaforte/4/head 2025-12-04T09:23:32.6484170Z * [new branch] gh/ezyang/2374/base -> origin/gh/ezyang/2374/base 2025-12-04T09:23:32.6484730Z * [new branch] gh/ezyang/2374/head -> origin/gh/ezyang/2374/head 2025-12-04T09:23:32.6485703Z * [new branch] gh/ezyang/2374/orig -> origin/gh/ezyang/2374/orig 2025-12-04T09:23:32.6486526Z * [new branch] gh/ezyang/2973/base -> origin/gh/ezyang/2973/base 2025-12-04T09:23:32.6486900Z * [new branch] gh/ezyang/2973/head -> origin/gh/ezyang/2973/head 2025-12-04T09:23:32.6487865Z * [new branch] gh/ezyang/2973/orig -> origin/gh/ezyang/2973/orig 2025-12-04T09:23:32.6488832Z * [new branch] gh/ezyang/2974/base -> origin/gh/ezyang/2974/base 2025-12-04T09:23:32.6489311Z * [new branch] gh/ezyang/2974/head -> origin/gh/ezyang/2974/head 2025-12-04T09:23:32.6490212Z * [new branch] gh/ezyang/2974/orig -> origin/gh/ezyang/2974/orig 2025-12-04T09:23:32.6491179Z * [new branch] gh/ezyang/3131/base -> origin/gh/ezyang/3131/base 2025-12-04T09:23:32.6491656Z * [new branch] gh/ezyang/3131/head -> origin/gh/ezyang/3131/head 2025-12-04T09:23:32.6492575Z * [new branch] gh/ezyang/3131/orig -> origin/gh/ezyang/3131/orig 2025-12-04T09:23:32.6493457Z * [new branch] gh/ezyang/3139/base -> origin/gh/ezyang/3139/base 2025-12-04T09:23:32.6493954Z * [new branch] gh/ezyang/3139/head -> origin/gh/ezyang/3139/head 2025-12-04T09:23:32.6494834Z * [new branch] gh/ezyang/3139/orig -> origin/gh/ezyang/3139/orig 2025-12-04T09:23:32.6495716Z * [new branch] gh/ezyang/3140/base -> origin/gh/ezyang/3140/base 2025-12-04T09:23:32.6496223Z * [new branch] gh/ezyang/3140/head -> origin/gh/ezyang/3140/head 2025-12-04T09:23:32.6497087Z * [new branch] gh/ezyang/3140/orig -> origin/gh/ezyang/3140/orig 2025-12-04T09:23:32.6498057Z * [new branch] gh/ezyang/3143/base -> origin/gh/ezyang/3143/base 2025-12-04T09:23:32.6498592Z * [new branch] gh/ezyang/3143/head -> origin/gh/ezyang/3143/head 2025-12-04T09:23:32.6499439Z * [new branch] gh/ezyang/3143/orig -> origin/gh/ezyang/3143/orig 2025-12-04T09:23:32.6500419Z * [new branch] gh/ezyang/3144/base -> origin/gh/ezyang/3144/base 2025-12-04T09:23:32.6500875Z * [new branch] gh/ezyang/3144/head -> origin/gh/ezyang/3144/head 2025-12-04T09:23:32.6501732Z * [new branch] gh/ezyang/3144/orig -> origin/gh/ezyang/3144/orig 2025-12-04T09:23:32.6502633Z * [new branch] gh/ezyang/3167/base -> origin/gh/ezyang/3167/base 2025-12-04T09:23:32.6503123Z * [new branch] gh/ezyang/3167/head -> origin/gh/ezyang/3167/head 2025-12-04T09:23:32.6504005Z * [new branch] gh/ezyang/3167/orig -> origin/gh/ezyang/3167/orig 2025-12-04T09:23:32.6504963Z * [new branch] gh/ezyang/3173/base -> origin/gh/ezyang/3173/base 2025-12-04T09:23:32.6505344Z * [new branch] gh/ezyang/3173/head -> origin/gh/ezyang/3173/head 2025-12-04T09:23:32.6506451Z * [new branch] gh/ezyang/3173/orig -> origin/gh/ezyang/3173/orig 2025-12-04T09:23:32.6507674Z * [new branch] gh/ezyang/3175/base -> origin/gh/ezyang/3175/base 2025-12-04T09:23:32.6508124Z * [new branch] gh/ezyang/3175/head -> origin/gh/ezyang/3175/head 2025-12-04T09:23:32.6509042Z * [new branch] gh/ezyang/3175/orig -> origin/gh/ezyang/3175/orig 2025-12-04T09:23:32.6509991Z * [new branch] gh/ezyang/3182/base -> origin/gh/ezyang/3182/base 2025-12-04T09:23:32.6510414Z * [new branch] gh/ezyang/3182/head -> origin/gh/ezyang/3182/head 2025-12-04T09:23:32.6511451Z * [new branch] gh/ezyang/3182/orig -> origin/gh/ezyang/3182/orig 2025-12-04T09:23:32.6512313Z * [new branch] gh/ezyang/3185/base -> origin/gh/ezyang/3185/base 2025-12-04T09:23:32.6513541Z * [new branch] gh/ezyang/3185/head -> origin/gh/ezyang/3185/head 2025-12-04T09:23:32.6513690Z * [new branch] gh/ezyang/3185/orig -> origin/gh/ezyang/3185/orig 2025-12-04T09:23:32.6515426Z * [new branch] gh/ezyang/3189/base -> origin/gh/ezyang/3189/base 2025-12-04T09:23:32.6515642Z * [new branch] gh/ezyang/3189/head -> origin/gh/ezyang/3189/head 2025-12-04T09:23:32.6516112Z * [new branch] gh/ezyang/3189/orig -> origin/gh/ezyang/3189/orig 2025-12-04T09:23:32.6517436Z * [new branch] gh/ezyang/3191/base -> origin/gh/ezyang/3191/base 2025-12-04T09:23:32.6517747Z * [new branch] gh/ezyang/3191/head -> origin/gh/ezyang/3191/head 2025-12-04T09:23:32.6518723Z * [new branch] gh/ezyang/3191/orig -> origin/gh/ezyang/3191/orig 2025-12-04T09:23:32.6519904Z * [new branch] gh/ezyang/3192/base -> origin/gh/ezyang/3192/base 2025-12-04T09:23:32.6520302Z * [new branch] gh/ezyang/3192/head -> origin/gh/ezyang/3192/head 2025-12-04T09:23:32.6521491Z * [new branch] gh/ezyang/3192/orig -> origin/gh/ezyang/3192/orig 2025-12-04T09:23:32.6522247Z * [new branch] gh/ezyang/3193/base -> origin/gh/ezyang/3193/base 2025-12-04T09:23:32.6522759Z * [new branch] gh/ezyang/3193/head -> origin/gh/ezyang/3193/head 2025-12-04T09:23:32.6523604Z * [new branch] gh/ezyang/3193/orig -> origin/gh/ezyang/3193/orig 2025-12-04T09:23:32.6524650Z * [new branch] gh/ezyang/3194/base -> origin/gh/ezyang/3194/base 2025-12-04T09:23:32.6524950Z * [new branch] gh/ezyang/3194/head -> origin/gh/ezyang/3194/head 2025-12-04T09:23:32.6526000Z * [new branch] gh/ezyang/3194/orig -> origin/gh/ezyang/3194/orig 2025-12-04T09:23:32.6527017Z * [new branch] gh/ezyang/3195/base -> origin/gh/ezyang/3195/base 2025-12-04T09:23:32.6528124Z * [new branch] gh/ezyang/3195/head -> origin/gh/ezyang/3195/head 2025-12-04T09:23:32.6528621Z * [new branch] gh/ezyang/3195/orig -> origin/gh/ezyang/3195/orig 2025-12-04T09:23:32.6529720Z * [new branch] gh/ezyang/3196/base -> origin/gh/ezyang/3196/base 2025-12-04T09:23:32.6530172Z * [new branch] gh/ezyang/3196/head -> origin/gh/ezyang/3196/head 2025-12-04T09:23:32.6531154Z * [new branch] gh/ezyang/3196/orig -> origin/gh/ezyang/3196/orig 2025-12-04T09:23:32.6532341Z * [new branch] gh/ezyang/3197/base -> origin/gh/ezyang/3197/base 2025-12-04T09:23:32.6532736Z * [new branch] gh/ezyang/3197/head -> origin/gh/ezyang/3197/head 2025-12-04T09:23:32.6534145Z * [new branch] gh/ezyang/3197/orig -> origin/gh/ezyang/3197/orig 2025-12-04T09:23:32.6534324Z * [new branch] gh/ezyang/3198/base -> origin/gh/ezyang/3198/base 2025-12-04T09:23:32.6540208Z * [new branch] gh/ezyang/3198/head -> origin/gh/ezyang/3198/head 2025-12-04T09:23:32.6540403Z * [new branch] gh/ezyang/3198/orig -> origin/gh/ezyang/3198/orig 2025-12-04T09:23:32.6540551Z * [new branch] gh/ezyang/3199/base -> origin/gh/ezyang/3199/base 2025-12-04T09:23:32.6540688Z * [new branch] gh/ezyang/3199/head -> origin/gh/ezyang/3199/head 2025-12-04T09:23:32.6540832Z * [new branch] gh/ezyang/3199/orig -> origin/gh/ezyang/3199/orig 2025-12-04T09:23:32.6540965Z * [new branch] gh/ezyang/3200/base -> origin/gh/ezyang/3200/base 2025-12-04T09:23:32.6541499Z * [new branch] gh/ezyang/3200/head -> origin/gh/ezyang/3200/head 2025-12-04T09:23:32.6541633Z * [new branch] gh/ezyang/3200/orig -> origin/gh/ezyang/3200/orig 2025-12-04T09:23:32.6541939Z * [new branch] gh/ezyang/3201/base -> origin/gh/ezyang/3201/base 2025-12-04T09:23:32.6542167Z * [new branch] gh/ezyang/3201/head -> origin/gh/ezyang/3201/head 2025-12-04T09:23:32.6542676Z * [new branch] gh/ezyang/3201/orig -> origin/gh/ezyang/3201/orig 2025-12-04T09:23:32.6546615Z * [new branch] gh/ezyang/3202/base -> origin/gh/ezyang/3202/base 2025-12-04T09:23:32.6546899Z * [new branch] gh/ezyang/3202/head -> origin/gh/ezyang/3202/head 2025-12-04T09:23:32.6547063Z * [new branch] gh/ezyang/3202/orig -> origin/gh/ezyang/3202/orig 2025-12-04T09:23:32.6547229Z * [new branch] gh/ezyang/3203/base -> origin/gh/ezyang/3203/base 2025-12-04T09:23:32.6547398Z * [new branch] gh/ezyang/3203/head -> origin/gh/ezyang/3203/head 2025-12-04T09:23:32.6547550Z * [new branch] gh/ezyang/3203/orig -> origin/gh/ezyang/3203/orig 2025-12-04T09:23:32.6548559Z * [new branch] gh/ezyang/3204/base -> origin/gh/ezyang/3204/base 2025-12-04T09:23:32.6548955Z * [new branch] gh/ezyang/3204/head -> origin/gh/ezyang/3204/head 2025-12-04T09:23:32.6551087Z * [new branch] gh/ezyang/3204/orig -> origin/gh/ezyang/3204/orig 2025-12-04T09:23:32.6551265Z * [new branch] gh/ezyang/3205/base -> origin/gh/ezyang/3205/base 2025-12-04T09:23:32.6551405Z * [new branch] gh/ezyang/3205/head -> origin/gh/ezyang/3205/head 2025-12-04T09:23:32.6551923Z * [new branch] gh/ezyang/3205/orig -> origin/gh/ezyang/3205/orig 2025-12-04T09:23:32.6553525Z * [new branch] gh/ezyang/3206/base -> origin/gh/ezyang/3206/base 2025-12-04T09:23:32.6553736Z * [new branch] gh/ezyang/3206/head -> origin/gh/ezyang/3206/head 2025-12-04T09:23:32.6556469Z * [new branch] gh/ezyang/3206/orig -> origin/gh/ezyang/3206/orig 2025-12-04T09:23:32.6559017Z * [new branch] gh/ezyang/3207/base -> origin/gh/ezyang/3207/base 2025-12-04T09:23:32.6563756Z * [new branch] gh/ezyang/3207/head -> origin/gh/ezyang/3207/head 2025-12-04T09:23:32.6566021Z * [new branch] gh/ezyang/3207/orig -> origin/gh/ezyang/3207/orig 2025-12-04T09:23:32.6566321Z * [new branch] gh/ezyang/3208/base -> origin/gh/ezyang/3208/base 2025-12-04T09:23:32.6566498Z * [new branch] gh/ezyang/3208/head -> origin/gh/ezyang/3208/head 2025-12-04T09:23:32.6566637Z * [new branch] gh/ezyang/3208/orig -> origin/gh/ezyang/3208/orig 2025-12-04T09:23:32.6566936Z * [new branch] gh/ezyang/3209/base -> origin/gh/ezyang/3209/base 2025-12-04T09:23:32.6567086Z * [new branch] gh/ezyang/3209/head -> origin/gh/ezyang/3209/head 2025-12-04T09:23:32.6567242Z * [new branch] gh/ezyang/3209/orig -> origin/gh/ezyang/3209/orig 2025-12-04T09:23:32.6567473Z * [new branch] gh/fadara01/3/base -> origin/gh/fadara01/3/base 2025-12-04T09:23:32.6567627Z * [new branch] gh/fadara01/3/head -> origin/gh/fadara01/3/head 2025-12-04T09:23:32.6567811Z * [new branch] gh/fadara01/3/orig -> origin/gh/fadara01/3/orig 2025-12-04T09:23:32.6567956Z * [new branch] gh/fadara01/5/base -> origin/gh/fadara01/5/base 2025-12-04T09:23:32.6568171Z * [new branch] gh/fadara01/5/head -> origin/gh/fadara01/5/head 2025-12-04T09:23:32.6568804Z * [new branch] gh/fadara01/5/orig -> origin/gh/fadara01/5/orig 2025-12-04T09:23:32.6569169Z * [new branch] gh/fadara01/6/base -> origin/gh/fadara01/6/base 2025-12-04T09:23:32.6569304Z * [new branch] gh/fadara01/6/head -> origin/gh/fadara01/6/head 2025-12-04T09:23:32.6569435Z * [new branch] gh/fadara01/6/orig -> origin/gh/fadara01/6/orig 2025-12-04T09:23:32.6569632Z * [new branch] gh/fadara01/7/base -> origin/gh/fadara01/7/base 2025-12-04T09:23:32.6569931Z * [new branch] gh/fadara01/7/head -> origin/gh/fadara01/7/head 2025-12-04T09:23:32.6570084Z * [new branch] gh/fadara01/7/orig -> origin/gh/fadara01/7/orig 2025-12-04T09:23:32.6575099Z * [new branch] gh/fadara01/8/base -> origin/gh/fadara01/8/base 2025-12-04T09:23:32.6575282Z * [new branch] gh/fadara01/8/head -> origin/gh/fadara01/8/head 2025-12-04T09:23:32.6575432Z * [new branch] gh/fadara01/8/orig -> origin/gh/fadara01/8/orig 2025-12-04T09:23:32.6575587Z * [new branch] gh/fadara01/9/base -> origin/gh/fadara01/9/base 2025-12-04T09:23:32.6575727Z * [new branch] gh/fadara01/9/head -> origin/gh/fadara01/9/head 2025-12-04T09:23:32.6575853Z * [new branch] gh/fadara01/9/orig -> origin/gh/fadara01/9/orig 2025-12-04T09:23:32.6575990Z * [new branch] gh/fduwjj/182/base -> origin/gh/fduwjj/182/base 2025-12-04T09:23:32.6577363Z * [new branch] gh/fduwjj/182/head -> origin/gh/fduwjj/182/head 2025-12-04T09:23:32.6577702Z * [new branch] gh/fduwjj/182/orig -> origin/gh/fduwjj/182/orig 2025-12-04T09:23:32.6580495Z * [new branch] gh/fduwjj/211/base -> origin/gh/fduwjj/211/base 2025-12-04T09:23:32.6580816Z * [new branch] gh/fduwjj/211/head -> origin/gh/fduwjj/211/head 2025-12-04T09:23:32.6580976Z * [new branch] gh/fduwjj/211/orig -> origin/gh/fduwjj/211/orig 2025-12-04T09:23:32.6585348Z * [new branch] gh/fduwjj/212/base -> origin/gh/fduwjj/212/base 2025-12-04T09:23:32.6585664Z * [new branch] gh/fduwjj/212/head -> origin/gh/fduwjj/212/head 2025-12-04T09:23:32.6585855Z * [new branch] gh/fduwjj/212/orig -> origin/gh/fduwjj/212/orig 2025-12-04T09:23:32.6586022Z * [new branch] gh/fduwjj/213/base -> origin/gh/fduwjj/213/base 2025-12-04T09:23:32.6586265Z * [new branch] gh/fduwjj/213/head -> origin/gh/fduwjj/213/head 2025-12-04T09:23:32.6590615Z * [new branch] gh/fduwjj/213/orig -> origin/gh/fduwjj/213/orig 2025-12-04T09:23:32.6590803Z * [new branch] gh/fduwjj/226/base -> origin/gh/fduwjj/226/base 2025-12-04T09:23:32.6591101Z * [new branch] gh/fduwjj/226/head -> origin/gh/fduwjj/226/head 2025-12-04T09:23:32.6591265Z * [new branch] gh/fduwjj/226/orig -> origin/gh/fduwjj/226/orig 2025-12-04T09:23:32.6591416Z * [new branch] gh/fduwjj/229/base -> origin/gh/fduwjj/229/base 2025-12-04T09:23:32.6591564Z * [new branch] gh/fduwjj/229/head -> origin/gh/fduwjj/229/head 2025-12-04T09:23:32.6592144Z * [new branch] gh/fduwjj/229/orig -> origin/gh/fduwjj/229/orig 2025-12-04T09:23:32.6592455Z * [new branch] gh/fduwjj/233/base -> origin/gh/fduwjj/233/base 2025-12-04T09:23:32.6592618Z * [new branch] gh/fduwjj/233/head -> origin/gh/fduwjj/233/head 2025-12-04T09:23:32.6592870Z * [new branch] gh/fduwjj/233/orig -> origin/gh/fduwjj/233/orig 2025-12-04T09:23:32.6593010Z * [new branch] gh/fduwjj/234/base -> origin/gh/fduwjj/234/base 2025-12-04T09:23:32.6593158Z * [new branch] gh/fduwjj/234/head -> origin/gh/fduwjj/234/head 2025-12-04T09:23:32.6593401Z * [new branch] gh/fduwjj/234/orig -> origin/gh/fduwjj/234/orig 2025-12-04T09:23:32.6597767Z * [new branch] gh/fduwjj/235/base -> origin/gh/fduwjj/235/base 2025-12-04T09:23:32.6598068Z * [new branch] gh/fduwjj/235/head -> origin/gh/fduwjj/235/head 2025-12-04T09:23:32.6598235Z * [new branch] gh/fduwjj/235/orig -> origin/gh/fduwjj/235/orig 2025-12-04T09:23:32.6598551Z * [new branch] gh/fduwjj/236/base -> origin/gh/fduwjj/236/base 2025-12-04T09:23:32.6598721Z * [new branch] gh/fduwjj/236/head -> origin/gh/fduwjj/236/head 2025-12-04T09:23:32.6598866Z * [new branch] gh/fduwjj/236/orig -> origin/gh/fduwjj/236/orig 2025-12-04T09:23:32.6599350Z * [new branch] gh/fduwjj/237/base -> origin/gh/fduwjj/237/base 2025-12-04T09:23:32.6604794Z * [new branch] gh/fduwjj/237/head -> origin/gh/fduwjj/237/head 2025-12-04T09:23:32.6605138Z * [new branch] gh/fduwjj/237/orig -> origin/gh/fduwjj/237/orig 2025-12-04T09:23:32.6605319Z * [new branch] gh/fduwjj/238/base -> origin/gh/fduwjj/238/base 2025-12-04T09:23:32.6605506Z * [new branch] gh/fduwjj/238/head -> origin/gh/fduwjj/238/head 2025-12-04T09:23:32.6605833Z * [new branch] gh/fduwjj/238/orig -> origin/gh/fduwjj/238/orig 2025-12-04T09:23:32.6606007Z * [new branch] gh/fduwjj/239/base -> origin/gh/fduwjj/239/base 2025-12-04T09:23:32.6606149Z * [new branch] gh/fduwjj/239/head -> origin/gh/fduwjj/239/head 2025-12-04T09:23:32.6608866Z * [new branch] gh/fduwjj/239/orig -> origin/gh/fduwjj/239/orig 2025-12-04T09:23:32.6609084Z * [new branch] gh/fegin/332/base -> origin/gh/fegin/332/base 2025-12-04T09:23:32.6609315Z * [new branch] gh/fegin/332/head -> origin/gh/fegin/332/head 2025-12-04T09:23:32.6609484Z * [new branch] gh/fegin/332/orig -> origin/gh/fegin/332/orig 2025-12-04T09:23:32.6609626Z * [new branch] gh/fegin/333/base -> origin/gh/fegin/333/base 2025-12-04T09:23:32.6609838Z * [new branch] gh/fegin/333/head -> origin/gh/fegin/333/head 2025-12-04T09:23:32.6609987Z * [new branch] gh/fegin/333/orig -> origin/gh/fegin/333/orig 2025-12-04T09:23:32.6616584Z * [new branch] gh/fegin/334/base -> origin/gh/fegin/334/base 2025-12-04T09:23:32.6621283Z * [new branch] gh/fegin/334/head -> origin/gh/fegin/334/head 2025-12-04T09:23:32.6625791Z * [new branch] gh/fegin/334/orig -> origin/gh/fegin/334/orig 2025-12-04T09:23:32.6628308Z * [new branch] gh/fegin/335/base -> origin/gh/fegin/335/base 2025-12-04T09:23:32.6628485Z * [new branch] gh/fegin/335/head -> origin/gh/fegin/335/head 2025-12-04T09:23:32.6628641Z * [new branch] gh/fegin/335/orig -> origin/gh/fegin/335/orig 2025-12-04T09:23:32.6628777Z * [new branch] gh/fffrog/160/base -> origin/gh/fffrog/160/base 2025-12-04T09:23:32.6628909Z * [new branch] gh/fffrog/160/head -> origin/gh/fffrog/160/head 2025-12-04T09:23:32.6629055Z * [new branch] gh/fffrog/177/base -> origin/gh/fffrog/177/base 2025-12-04T09:23:32.6629184Z * [new branch] gh/fffrog/177/head -> origin/gh/fffrog/177/head 2025-12-04T09:23:32.6629324Z * [new branch] gh/fffrog/177/orig -> origin/gh/fffrog/177/orig 2025-12-04T09:23:32.6629448Z * [new branch] gh/fffrog/178/base -> origin/gh/fffrog/178/base 2025-12-04T09:23:32.6629574Z * [new branch] gh/fffrog/178/head -> origin/gh/fffrog/178/head 2025-12-04T09:23:32.6629709Z * [new branch] gh/fffrog/178/orig -> origin/gh/fffrog/178/orig 2025-12-04T09:23:32.6630057Z * [new branch] gh/fffrog/181/base -> origin/gh/fffrog/181/base 2025-12-04T09:23:32.6630189Z * [new branch] gh/fffrog/181/head -> origin/gh/fffrog/181/head 2025-12-04T09:23:32.6630325Z * [new branch] gh/fffrog/181/orig -> origin/gh/fffrog/181/orig 2025-12-04T09:23:32.6630526Z * [new branch] gh/fffrog/183/base -> origin/gh/fffrog/183/base 2025-12-04T09:23:32.6630664Z * [new branch] gh/fffrog/183/head -> origin/gh/fffrog/183/head 2025-12-04T09:23:32.6630790Z * [new branch] gh/fffrog/183/orig -> origin/gh/fffrog/183/orig 2025-12-04T09:23:32.6630918Z * [new branch] gh/fxdawnn/10/base -> origin/gh/fxdawnn/10/base 2025-12-04T09:23:32.6631056Z * [new branch] gh/fxdawnn/10/head -> origin/gh/fxdawnn/10/head 2025-12-04T09:23:32.6631182Z * [new branch] gh/fxdawnn/10/orig -> origin/gh/fxdawnn/10/orig 2025-12-04T09:23:32.6631318Z * [new branch] gh/fxdawnn/11/base -> origin/gh/fxdawnn/11/base 2025-12-04T09:23:32.6631442Z * [new branch] gh/fxdawnn/11/head -> origin/gh/fxdawnn/11/head 2025-12-04T09:23:32.6631567Z * [new branch] gh/fxdawnn/11/orig -> origin/gh/fxdawnn/11/orig 2025-12-04T09:23:32.6631713Z * [new branch] gh/fxdawnn/12/base -> origin/gh/fxdawnn/12/base 2025-12-04T09:23:32.6631839Z * [new branch] gh/fxdawnn/12/head -> origin/gh/fxdawnn/12/head 2025-12-04T09:23:32.6631974Z * [new branch] gh/fxdawnn/12/orig -> origin/gh/fxdawnn/12/orig 2025-12-04T09:23:32.6632108Z * [new branch] gh/fxdawnn/13/base -> origin/gh/fxdawnn/13/base 2025-12-04T09:23:32.6632235Z * [new branch] gh/fxdawnn/13/head -> origin/gh/fxdawnn/13/head 2025-12-04T09:23:32.6632370Z * [new branch] gh/fxdawnn/13/orig -> origin/gh/fxdawnn/13/orig 2025-12-04T09:23:32.6637666Z * [new branch] gh/fxdawnn/14/base -> origin/gh/fxdawnn/14/base 2025-12-04T09:23:32.6638017Z * [new branch] gh/fxdawnn/14/head -> origin/gh/fxdawnn/14/head 2025-12-04T09:23:32.6638180Z * [new branch] gh/fxdawnn/14/orig -> origin/gh/fxdawnn/14/orig 2025-12-04T09:23:32.6638424Z * [new branch] gh/fxdawnn/15/base -> origin/gh/fxdawnn/15/base 2025-12-04T09:23:32.6638596Z * [new branch] gh/fxdawnn/15/head -> origin/gh/fxdawnn/15/head 2025-12-04T09:23:32.6638728Z * [new branch] gh/fxdawnn/15/orig -> origin/gh/fxdawnn/15/orig 2025-12-04T09:23:32.6638885Z * [new branch] gh/fxdawnn/6/base -> origin/gh/fxdawnn/6/base 2025-12-04T09:23:32.6639021Z * [new branch] gh/fxdawnn/6/head -> origin/gh/fxdawnn/6/head 2025-12-04T09:23:32.6639153Z * [new branch] gh/fxdawnn/6/orig -> origin/gh/fxdawnn/6/orig 2025-12-04T09:23:32.6639300Z * [new branch] gh/fxdawnn/7/base -> origin/gh/fxdawnn/7/base 2025-12-04T09:23:32.6639439Z * [new branch] gh/fxdawnn/7/head -> origin/gh/fxdawnn/7/head 2025-12-04T09:23:32.6644898Z * [new branch] gh/fxdawnn/7/orig -> origin/gh/fxdawnn/7/orig 2025-12-04T09:23:32.6645231Z * [new branch] gh/fxdawnn/9/base -> origin/gh/fxdawnn/9/base 2025-12-04T09:23:32.6645385Z * [new branch] gh/fxdawnn/9/head -> origin/gh/fxdawnn/9/head 2025-12-04T09:23:32.6645512Z * [new branch] gh/fxdawnn/9/orig -> origin/gh/fxdawnn/9/orig 2025-12-04T09:23:32.6645768Z * [new branch] gh/galv/1/base -> origin/gh/galv/1/base 2025-12-04T09:23:32.6645907Z * [new branch] gh/galv/1/head -> origin/gh/galv/1/head 2025-12-04T09:23:32.6646120Z * [new branch] gh/galv/1/orig -> origin/gh/galv/1/orig 2025-12-04T09:23:32.6651022Z * [new branch] gh/galv/2/base -> origin/gh/galv/2/base 2025-12-04T09:23:32.6651334Z * [new branch] gh/galv/2/head -> origin/gh/galv/2/head 2025-12-04T09:23:32.6651518Z * [new branch] gh/galv/2/orig -> origin/gh/galv/2/orig 2025-12-04T09:23:32.6651780Z * [new branch] gh/galv/3/base -> origin/gh/galv/3/base 2025-12-04T09:23:32.6652037Z * [new branch] gh/galv/3/head -> origin/gh/galv/3/head 2025-12-04T09:23:32.6652174Z * [new branch] gh/galv/3/orig -> origin/gh/galv/3/orig 2025-12-04T09:23:32.6652325Z * [new branch] gh/guangyey/134/base -> origin/gh/guangyey/134/base 2025-12-04T09:23:32.6656510Z * [new branch] gh/guangyey/134/head -> origin/gh/guangyey/134/head 2025-12-04T09:23:32.6661273Z * [new branch] gh/guangyey/134/orig -> origin/gh/guangyey/134/orig 2025-12-04T09:23:32.6666077Z * [new branch] gh/guangyey/163/base -> origin/gh/guangyey/163/base 2025-12-04T09:23:32.6668080Z * [new branch] gh/guangyey/163/head -> origin/gh/guangyey/163/head 2025-12-04T09:23:32.6668440Z * [new branch] gh/guangyey/163/orig -> origin/gh/guangyey/163/orig 2025-12-04T09:23:32.6668608Z * [new branch] gh/guangyey/168/base -> origin/gh/guangyey/168/base 2025-12-04T09:23:32.6668745Z * [new branch] gh/guangyey/168/head -> origin/gh/guangyey/168/head 2025-12-04T09:23:32.6668907Z * [new branch] gh/guangyey/168/orig -> origin/gh/guangyey/168/orig 2025-12-04T09:23:32.6669059Z * [new branch] gh/guangyey/169/base -> origin/gh/guangyey/169/base 2025-12-04T09:23:32.6669188Z * [new branch] gh/guangyey/169/head -> origin/gh/guangyey/169/head 2025-12-04T09:23:32.6669317Z * [new branch] gh/guangyey/169/orig -> origin/gh/guangyey/169/orig 2025-12-04T09:23:32.6669456Z * [new branch] gh/guangyey/170/base -> origin/gh/guangyey/170/base 2025-12-04T09:23:32.6669584Z * [new branch] gh/guangyey/170/head -> origin/gh/guangyey/170/head 2025-12-04T09:23:32.6669719Z * [new branch] gh/guangyey/170/orig -> origin/gh/guangyey/170/orig 2025-12-04T09:23:32.6669850Z * [new branch] gh/guangyey/171/base -> origin/gh/guangyey/171/base 2025-12-04T09:23:32.6669975Z * [new branch] gh/guangyey/171/head -> origin/gh/guangyey/171/head 2025-12-04T09:23:32.6670111Z * [new branch] gh/guangyey/171/orig -> origin/gh/guangyey/171/orig 2025-12-04T09:23:32.6670249Z * [new branch] gh/guangyey/178/base -> origin/gh/guangyey/178/base 2025-12-04T09:23:32.6670387Z * [new branch] gh/guangyey/178/head -> origin/gh/guangyey/178/head 2025-12-04T09:23:32.6670518Z * [new branch] gh/guangyey/178/orig -> origin/gh/guangyey/178/orig 2025-12-04T09:23:32.6670649Z * [new branch] gh/guangyey/182/base -> origin/gh/guangyey/182/base 2025-12-04T09:23:32.6670787Z * [new branch] gh/guangyey/182/head -> origin/gh/guangyey/182/head 2025-12-04T09:23:32.6670918Z * [new branch] gh/guangyey/182/orig -> origin/gh/guangyey/182/orig 2025-12-04T09:23:32.6671058Z * [new branch] gh/guangyey/183/base -> origin/gh/guangyey/183/base 2025-12-04T09:23:32.6671188Z * [new branch] gh/guangyey/183/head -> origin/gh/guangyey/183/head 2025-12-04T09:23:32.6673575Z * [new branch] gh/guangyey/183/orig -> origin/gh/guangyey/183/orig 2025-12-04T09:23:32.6673760Z * [new branch] gh/guangyey/185/base -> origin/gh/guangyey/185/base 2025-12-04T09:23:32.6673924Z * [new branch] gh/guangyey/185/head -> origin/gh/guangyey/185/head 2025-12-04T09:23:32.6674292Z * [new branch] gh/guangyey/185/orig -> origin/gh/guangyey/185/orig 2025-12-04T09:23:32.6674455Z * [new branch] gh/guangyey/186/base -> origin/gh/guangyey/186/base 2025-12-04T09:23:32.6674615Z * [new branch] gh/guangyey/186/head -> origin/gh/guangyey/186/head 2025-12-04T09:23:32.6684336Z * [new branch] gh/guangyey/186/orig -> origin/gh/guangyey/186/orig 2025-12-04T09:23:32.6688318Z * [new branch] gh/guangyey/187/base -> origin/gh/guangyey/187/base 2025-12-04T09:23:32.6691783Z * [new branch] gh/guangyey/187/head -> origin/gh/guangyey/187/head 2025-12-04T09:23:32.6695996Z * [new branch] gh/guangyey/187/orig -> origin/gh/guangyey/187/orig 2025-12-04T09:23:32.6700196Z * [new branch] gh/guangyey/188/base -> origin/gh/guangyey/188/base 2025-12-04T09:23:32.6702124Z * [new branch] gh/guangyey/188/head -> origin/gh/guangyey/188/head 2025-12-04T09:23:32.6702300Z * [new branch] gh/guangyey/188/orig -> origin/gh/guangyey/188/orig 2025-12-04T09:23:32.6702437Z * [new branch] gh/guangyey/190/base -> origin/gh/guangyey/190/base 2025-12-04T09:23:32.6702582Z * [new branch] gh/guangyey/190/head -> origin/gh/guangyey/190/head 2025-12-04T09:23:32.6702723Z * [new branch] gh/guangyey/190/orig -> origin/gh/guangyey/190/orig 2025-12-04T09:23:32.6702852Z * [new branch] gh/guangyey/208/base -> origin/gh/guangyey/208/base 2025-12-04T09:23:32.6703001Z * [new branch] gh/guangyey/208/head -> origin/gh/guangyey/208/head 2025-12-04T09:23:32.6703204Z * [new branch] gh/guangyey/208/orig -> origin/gh/guangyey/208/orig 2025-12-04T09:23:32.6703356Z * [new branch] gh/guangyey/228/base -> origin/gh/guangyey/228/base 2025-12-04T09:23:32.6703487Z * [new branch] gh/guangyey/228/head -> origin/gh/guangyey/228/head 2025-12-04T09:23:32.6703626Z * [new branch] gh/guangyey/228/orig -> origin/gh/guangyey/228/orig 2025-12-04T09:23:32.6703764Z * [new branch] gh/guangyey/230/base -> origin/gh/guangyey/230/base 2025-12-04T09:23:32.6703898Z * [new branch] gh/guangyey/230/head -> origin/gh/guangyey/230/head 2025-12-04T09:23:32.6704037Z * [new branch] gh/guangyey/230/orig -> origin/gh/guangyey/230/orig 2025-12-04T09:23:32.6704165Z * [new branch] gh/guangyey/231/base -> origin/gh/guangyey/231/base 2025-12-04T09:23:32.6704295Z * [new branch] gh/guangyey/231/head -> origin/gh/guangyey/231/head 2025-12-04T09:23:32.6704428Z * [new branch] gh/guangyey/231/orig -> origin/gh/guangyey/231/orig 2025-12-04T09:23:32.6704561Z * [new branch] gh/guangyey/232/base -> origin/gh/guangyey/232/base 2025-12-04T09:23:32.6704698Z * [new branch] gh/guangyey/232/head -> origin/gh/guangyey/232/head 2025-12-04T09:23:32.6704831Z * [new branch] gh/guangyey/232/orig -> origin/gh/guangyey/232/orig 2025-12-04T09:23:32.6704962Z * [new branch] gh/guangyey/233/base -> origin/gh/guangyey/233/base 2025-12-04T09:23:32.6705098Z * [new branch] gh/guangyey/233/head -> origin/gh/guangyey/233/head 2025-12-04T09:23:32.6705231Z * [new branch] gh/guangyey/233/orig -> origin/gh/guangyey/233/orig 2025-12-04T09:23:32.6705360Z * [new branch] gh/guangyey/234/base -> origin/gh/guangyey/234/base 2025-12-04T09:23:32.6705499Z * [new branch] gh/guangyey/234/head -> origin/gh/guangyey/234/head 2025-12-04T09:23:32.6705632Z * [new branch] gh/guangyey/234/orig -> origin/gh/guangyey/234/orig 2025-12-04T09:23:32.6705893Z * [new branch] gh/guangyey/235/base -> origin/gh/guangyey/235/base 2025-12-04T09:23:32.6706249Z * [new branch] gh/guangyey/235/head -> origin/gh/guangyey/235/head 2025-12-04T09:23:32.6706395Z * [new branch] gh/guangyey/235/orig -> origin/gh/guangyey/235/orig 2025-12-04T09:23:32.6706549Z * [new branch] gh/guangyey/236/base -> origin/gh/guangyey/236/base 2025-12-04T09:23:32.6706755Z * [new branch] gh/guangyey/236/head -> origin/gh/guangyey/236/head 2025-12-04T09:23:32.6706917Z * [new branch] gh/guangyey/236/orig -> origin/gh/guangyey/236/orig 2025-12-04T09:23:32.6707069Z * [new branch] gh/guangyey/237/base -> origin/gh/guangyey/237/base 2025-12-04T09:23:32.6707220Z * [new branch] gh/guangyey/237/head -> origin/gh/guangyey/237/head 2025-12-04T09:23:32.6707540Z * [new branch] gh/guangyey/237/orig -> origin/gh/guangyey/237/orig 2025-12-04T09:23:32.6708061Z * [new branch] gh/guangyey/238/base -> origin/gh/guangyey/238/base 2025-12-04T09:23:32.6708269Z * [new branch] gh/guangyey/238/head -> origin/gh/guangyey/238/head 2025-12-04T09:23:32.6708425Z * [new branch] gh/guangyey/239/base -> origin/gh/guangyey/239/base 2025-12-04T09:23:32.6708574Z * [new branch] gh/guangyey/239/head -> origin/gh/guangyey/239/head 2025-12-04T09:23:32.6708754Z * [new branch] gh/guangyey/239/orig -> origin/gh/guangyey/239/orig 2025-12-04T09:23:32.6708892Z * [new branch] gh/guangyey/240/base -> origin/gh/guangyey/240/base 2025-12-04T09:23:32.6709040Z * [new branch] gh/guangyey/240/head -> origin/gh/guangyey/240/head 2025-12-04T09:23:32.6709186Z * [new branch] gh/guangyey/240/orig -> origin/gh/guangyey/240/orig 2025-12-04T09:23:32.6709331Z * [new branch] gh/guangyey/241/base -> origin/gh/guangyey/241/base 2025-12-04T09:23:32.6709475Z * [new branch] gh/guangyey/241/head -> origin/gh/guangyey/241/head 2025-12-04T09:23:32.6709619Z * [new branch] gh/guangyey/241/orig -> origin/gh/guangyey/241/orig 2025-12-04T09:23:32.6714160Z * [new branch] gh/guangyey/242/base -> origin/gh/guangyey/242/base 2025-12-04T09:23:32.6714457Z * [new branch] gh/guangyey/242/head -> origin/gh/guangyey/242/head 2025-12-04T09:23:32.6714642Z * [new branch] gh/guangyey/242/orig -> origin/gh/guangyey/242/orig 2025-12-04T09:23:32.6714811Z * [new branch] gh/guangyey/243/base -> origin/gh/guangyey/243/base 2025-12-04T09:23:32.6714965Z * [new branch] gh/guangyey/243/head -> origin/gh/guangyey/243/head 2025-12-04T09:23:32.6719622Z * [new branch] gh/guangyey/243/orig -> origin/gh/guangyey/243/orig 2025-12-04T09:23:32.6724248Z * [new branch] gh/guangyey/244/base -> origin/gh/guangyey/244/base 2025-12-04T09:23:32.6728348Z * [new branch] gh/guangyey/244/head -> origin/gh/guangyey/244/head 2025-12-04T09:23:32.6731152Z * [new branch] gh/guangyey/244/orig -> origin/gh/guangyey/244/orig 2025-12-04T09:23:32.6733739Z * [new branch] gh/guangyey/245/base -> origin/gh/guangyey/245/base 2025-12-04T09:23:32.6733933Z * [new branch] gh/guangyey/245/head -> origin/gh/guangyey/245/head 2025-12-04T09:23:32.6734099Z * [new branch] gh/guangyey/245/orig -> origin/gh/guangyey/245/orig 2025-12-04T09:23:32.6734250Z * [new branch] gh/guangyey/246/base -> origin/gh/guangyey/246/base 2025-12-04T09:23:32.6734399Z * [new branch] gh/guangyey/246/head -> origin/gh/guangyey/246/head 2025-12-04T09:23:32.6734560Z * [new branch] gh/guangyey/246/orig -> origin/gh/guangyey/246/orig 2025-12-04T09:23:32.6734706Z * [new branch] gh/guangyey/247/base -> origin/gh/guangyey/247/base 2025-12-04T09:23:32.6735014Z * [new branch] gh/guangyey/247/head -> origin/gh/guangyey/247/head 2025-12-04T09:23:32.6735177Z * [new branch] gh/guangyey/247/orig -> origin/gh/guangyey/247/orig 2025-12-04T09:23:32.6735324Z * [new branch] gh/guangyey/248/base -> origin/gh/guangyey/248/base 2025-12-04T09:23:32.6735536Z * [new branch] gh/guangyey/248/head -> origin/gh/guangyey/248/head 2025-12-04T09:23:32.6735688Z * [new branch] gh/guangyey/248/orig -> origin/gh/guangyey/248/orig 2025-12-04T09:23:32.6735837Z * [new branch] gh/guangyey/249/base -> origin/gh/guangyey/249/base 2025-12-04T09:23:32.6735978Z * [new branch] gh/guangyey/249/head -> origin/gh/guangyey/249/head 2025-12-04T09:23:32.6736110Z * [new branch] gh/guangyey/249/orig -> origin/gh/guangyey/249/orig 2025-12-04T09:23:32.6736252Z * [new branch] gh/guangyey/250/base -> origin/gh/guangyey/250/base 2025-12-04T09:23:32.6736386Z * [new branch] gh/guangyey/250/head -> origin/gh/guangyey/250/head 2025-12-04T09:23:32.6736519Z * [new branch] gh/guangyey/250/orig -> origin/gh/guangyey/250/orig 2025-12-04T09:23:32.6736661Z * [new branch] gh/guangyey/251/base -> origin/gh/guangyey/251/base 2025-12-04T09:23:32.6736795Z * [new branch] gh/guangyey/251/head -> origin/gh/guangyey/251/head 2025-12-04T09:23:32.6736939Z * [new branch] gh/guangyey/251/orig -> origin/gh/guangyey/251/orig 2025-12-04T09:23:32.6737072Z * [new branch] gh/guangyey/252/base -> origin/gh/guangyey/252/base 2025-12-04T09:23:32.6737204Z * [new branch] gh/guangyey/252/head -> origin/gh/guangyey/252/head 2025-12-04T09:23:32.6737346Z * [new branch] gh/guangyey/252/orig -> origin/gh/guangyey/252/orig 2025-12-04T09:23:32.6737476Z * [new branch] gh/guangyey/253/base -> origin/gh/guangyey/253/base 2025-12-04T09:23:32.6737615Z * [new branch] gh/guangyey/253/head -> origin/gh/guangyey/253/head 2025-12-04T09:23:32.6737936Z * [new branch] gh/guangyey/253/orig -> origin/gh/guangyey/253/orig 2025-12-04T09:23:32.6738069Z * [new branch] gh/guangyey/254/base -> origin/gh/guangyey/254/base 2025-12-04T09:23:32.6738213Z * [new branch] gh/guangyey/254/head -> origin/gh/guangyey/254/head 2025-12-04T09:23:32.6738345Z * [new branch] gh/guangyey/254/orig -> origin/gh/guangyey/254/orig 2025-12-04T09:23:32.6738485Z * [new branch] gh/guangyey/255/base -> origin/gh/guangyey/255/base 2025-12-04T09:23:32.6738942Z * [new branch] gh/guangyey/255/head -> origin/gh/guangyey/255/head 2025-12-04T09:23:32.6739154Z * [new branch] gh/guangyey/255/orig -> origin/gh/guangyey/255/orig 2025-12-04T09:23:32.6743259Z * [new branch] gh/guilhermeleobas/107/base -> origin/gh/guilhermeleobas/107/base 2025-12-04T09:23:32.6743504Z * [new branch] gh/guilhermeleobas/107/head -> origin/gh/guilhermeleobas/107/head 2025-12-04T09:23:32.6743683Z * [new branch] gh/guilhermeleobas/107/orig -> origin/gh/guilhermeleobas/107/orig 2025-12-04T09:23:32.6743967Z * [new branch] gh/guilhermeleobas/108/base -> origin/gh/guilhermeleobas/108/base 2025-12-04T09:23:32.6744146Z * [new branch] gh/guilhermeleobas/108/head -> origin/gh/guilhermeleobas/108/head 2025-12-04T09:23:32.6744404Z * [new branch] gh/guilhermeleobas/108/orig -> origin/gh/guilhermeleobas/108/orig 2025-12-04T09:23:32.6744614Z * [new branch] gh/guilhermeleobas/150/base -> origin/gh/guilhermeleobas/150/base 2025-12-04T09:23:32.6748898Z * [new branch] gh/guilhermeleobas/150/head -> origin/gh/guilhermeleobas/150/head 2025-12-04T09:23:32.6749087Z * [new branch] gh/guilhermeleobas/150/orig -> origin/gh/guilhermeleobas/150/orig 2025-12-04T09:23:32.6749444Z * [new branch] gh/guilhermeleobas/168/base -> origin/gh/guilhermeleobas/168/base 2025-12-04T09:23:32.6749601Z * [new branch] gh/guilhermeleobas/168/head -> origin/gh/guilhermeleobas/168/head 2025-12-04T09:23:32.6749756Z * [new branch] gh/guilhermeleobas/168/orig -> origin/gh/guilhermeleobas/168/orig 2025-12-04T09:23:32.6750006Z * [new branch] gh/guilhermeleobas/169/base -> origin/gh/guilhermeleobas/169/base 2025-12-04T09:23:32.6750331Z * [new branch] gh/guilhermeleobas/169/head -> origin/gh/guilhermeleobas/169/head 2025-12-04T09:23:32.6750529Z * [new branch] gh/guilhermeleobas/169/orig -> origin/gh/guilhermeleobas/169/orig 2025-12-04T09:23:32.6751942Z * [new branch] gh/guilhermeleobas/170/base -> origin/gh/guilhermeleobas/170/base 2025-12-04T09:23:32.6752587Z * [new branch] gh/guilhermeleobas/170/head -> origin/gh/guilhermeleobas/170/head 2025-12-04T09:23:32.6753392Z * [new branch] gh/guilhermeleobas/170/orig -> origin/gh/guilhermeleobas/170/orig 2025-12-04T09:23:32.6757061Z * [new branch] gh/guilhermeleobas/171/base -> origin/gh/guilhermeleobas/171/base 2025-12-04T09:23:32.6757416Z * [new branch] gh/guilhermeleobas/171/head -> origin/gh/guilhermeleobas/171/head 2025-12-04T09:23:32.6757614Z * [new branch] gh/guilhermeleobas/171/orig -> origin/gh/guilhermeleobas/171/orig 2025-12-04T09:23:32.6757783Z * [new branch] gh/guilhermeleobas/173/base -> origin/gh/guilhermeleobas/173/base 2025-12-04T09:23:32.6758068Z * [new branch] gh/guilhermeleobas/173/head -> origin/gh/guilhermeleobas/173/head 2025-12-04T09:23:32.6758864Z * [new branch] gh/guilhermeleobas/173/orig -> origin/gh/guilhermeleobas/173/orig 2025-12-04T09:23:32.6759762Z * [new branch] gh/guilhermeleobas/193/base -> origin/gh/guilhermeleobas/193/base 2025-12-04T09:23:32.6760251Z * [new branch] gh/guilhermeleobas/193/head -> origin/gh/guilhermeleobas/193/head 2025-12-04T09:23:32.6763093Z * [new branch] gh/guilhermeleobas/193/orig -> origin/gh/guilhermeleobas/193/orig 2025-12-04T09:23:32.6763413Z * [new branch] gh/guilhermeleobas/204/base -> origin/gh/guilhermeleobas/204/base 2025-12-04T09:23:32.6763718Z * [new branch] gh/guilhermeleobas/204/head -> origin/gh/guilhermeleobas/204/head 2025-12-04T09:23:32.6763919Z * [new branch] gh/guilhermeleobas/204/orig -> origin/gh/guilhermeleobas/204/orig 2025-12-04T09:23:32.6765550Z * [new branch] gh/guilhermeleobas/211/base -> origin/gh/guilhermeleobas/211/base 2025-12-04T09:23:32.6765741Z * [new branch] gh/guilhermeleobas/211/head -> origin/gh/guilhermeleobas/211/head 2025-12-04T09:23:32.6766002Z * [new branch] gh/guilhermeleobas/211/orig -> origin/gh/guilhermeleobas/211/orig 2025-12-04T09:23:32.6767965Z * [new branch] gh/guilhermeleobas/226/base -> origin/gh/guilhermeleobas/226/base 2025-12-04T09:23:32.6768190Z * [new branch] gh/guilhermeleobas/226/head -> origin/gh/guilhermeleobas/226/head 2025-12-04T09:23:32.6768355Z * [new branch] gh/guilhermeleobas/226/orig -> origin/gh/guilhermeleobas/226/orig 2025-12-04T09:23:32.6770918Z * [new branch] gh/guilhermeleobas/236/base -> origin/gh/guilhermeleobas/236/base 2025-12-04T09:23:32.6771263Z * [new branch] gh/guilhermeleobas/236/head -> origin/gh/guilhermeleobas/236/head 2025-12-04T09:23:32.6771448Z * [new branch] gh/guilhermeleobas/236/orig -> origin/gh/guilhermeleobas/236/orig 2025-12-04T09:23:32.6771694Z * [new branch] gh/guilhermeleobas/247/base -> origin/gh/guilhermeleobas/247/base 2025-12-04T09:23:32.6772032Z * [new branch] gh/guilhermeleobas/247/head -> origin/gh/guilhermeleobas/247/head 2025-12-04T09:23:32.6773060Z * [new branch] gh/guilhermeleobas/247/orig -> origin/gh/guilhermeleobas/247/orig 2025-12-04T09:23:32.6773666Z * [new branch] gh/guilhermeleobas/248/base -> origin/gh/guilhermeleobas/248/base 2025-12-04T09:23:32.6774400Z * [new branch] gh/guilhermeleobas/248/head -> origin/gh/guilhermeleobas/248/head 2025-12-04T09:23:32.6775141Z * [new branch] gh/guilhermeleobas/248/orig -> origin/gh/guilhermeleobas/248/orig 2025-12-04T09:23:32.6776246Z * [new branch] gh/guilhermeleobas/250/base -> origin/gh/guilhermeleobas/250/base 2025-12-04T09:23:32.6776486Z * [new branch] gh/guilhermeleobas/250/head -> origin/gh/guilhermeleobas/250/head 2025-12-04T09:23:32.6777601Z * [new branch] gh/guilhermeleobas/250/orig -> origin/gh/guilhermeleobas/250/orig 2025-12-04T09:23:32.6780061Z * [new branch] gh/guilhermeleobas/253/base -> origin/gh/guilhermeleobas/253/base 2025-12-04T09:23:32.6780301Z * [new branch] gh/guilhermeleobas/253/head -> origin/gh/guilhermeleobas/253/head 2025-12-04T09:23:32.6780508Z * [new branch] gh/guilhermeleobas/253/orig -> origin/gh/guilhermeleobas/253/orig 2025-12-04T09:23:32.6780970Z * [new branch] gh/guilhermeleobas/254/base -> origin/gh/guilhermeleobas/254/base 2025-12-04T09:23:32.6781542Z * [new branch] gh/guilhermeleobas/254/head -> origin/gh/guilhermeleobas/254/head 2025-12-04T09:23:32.6782396Z * [new branch] gh/guilhermeleobas/254/orig -> origin/gh/guilhermeleobas/254/orig 2025-12-04T09:23:32.6783509Z * [new branch] gh/guilhermeleobas/255/base -> origin/gh/guilhermeleobas/255/base 2025-12-04T09:23:32.6783700Z * [new branch] gh/guilhermeleobas/255/head -> origin/gh/guilhermeleobas/255/head 2025-12-04T09:23:32.6785920Z * [new branch] gh/guilhermeleobas/255/orig -> origin/gh/guilhermeleobas/255/orig 2025-12-04T09:23:32.6786276Z * [new branch] gh/guilhermeleobas/256/base -> origin/gh/guilhermeleobas/256/base 2025-12-04T09:23:32.6786486Z * [new branch] gh/guilhermeleobas/256/head -> origin/gh/guilhermeleobas/256/head 2025-12-04T09:23:32.6786894Z * [new branch] gh/guilhermeleobas/256/orig -> origin/gh/guilhermeleobas/256/orig 2025-12-04T09:23:32.6791294Z * [new branch] gh/guilhermeleobas/257/base -> origin/gh/guilhermeleobas/257/base 2025-12-04T09:23:32.6791511Z * [new branch] gh/guilhermeleobas/257/head -> origin/gh/guilhermeleobas/257/head 2025-12-04T09:23:32.6791986Z * [new branch] gh/guilhermeleobas/257/orig -> origin/gh/guilhermeleobas/257/orig 2025-12-04T09:23:32.6792184Z * [new branch] gh/guilhermeleobas/258/base -> origin/gh/guilhermeleobas/258/base 2025-12-04T09:23:32.6792374Z * [new branch] gh/guilhermeleobas/258/head -> origin/gh/guilhermeleobas/258/head 2025-12-04T09:23:32.6792567Z * [new branch] gh/guilhermeleobas/258/orig -> origin/gh/guilhermeleobas/258/orig 2025-12-04T09:23:32.6793103Z * [new branch] gh/guilhermeleobas/259/base -> origin/gh/guilhermeleobas/259/base 2025-12-04T09:23:32.6793942Z * [new branch] gh/guilhermeleobas/259/head -> origin/gh/guilhermeleobas/259/head 2025-12-04T09:23:32.6795376Z * [new branch] gh/guilhermeleobas/259/orig -> origin/gh/guilhermeleobas/259/orig 2025-12-04T09:23:32.6795559Z * [new branch] gh/guilhermeleobas/260/base -> origin/gh/guilhermeleobas/260/base 2025-12-04T09:23:32.6796156Z * [new branch] gh/guilhermeleobas/260/head -> origin/gh/guilhermeleobas/260/head 2025-12-04T09:23:32.6799499Z * [new branch] gh/guilhermeleobas/260/orig -> origin/gh/guilhermeleobas/260/orig 2025-12-04T09:23:32.6799730Z * [new branch] gh/guilhermeleobas/261/base -> origin/gh/guilhermeleobas/261/base 2025-12-04T09:23:32.6799926Z * [new branch] gh/guilhermeleobas/261/head -> origin/gh/guilhermeleobas/261/head 2025-12-04T09:23:32.6800113Z * [new branch] gh/guilhermeleobas/261/orig -> origin/gh/guilhermeleobas/261/orig 2025-12-04T09:23:32.6800467Z * [new branch] gh/guilhermeleobas/262/base -> origin/gh/guilhermeleobas/262/base 2025-12-04T09:23:32.6801070Z * [new branch] gh/guilhermeleobas/262/head -> origin/gh/guilhermeleobas/262/head 2025-12-04T09:23:32.6802005Z * [new branch] gh/guilhermeleobas/262/orig -> origin/gh/guilhermeleobas/262/orig 2025-12-04T09:23:32.6803161Z * [new branch] gh/guilhermeleobas/263/base -> origin/gh/guilhermeleobas/263/base 2025-12-04T09:23:32.6803428Z * [new branch] gh/guilhermeleobas/263/head -> origin/gh/guilhermeleobas/263/head 2025-12-04T09:23:32.6804422Z * [new branch] gh/guilhermeleobas/263/orig -> origin/gh/guilhermeleobas/263/orig 2025-12-04T09:23:32.6805456Z * [new branch] gh/guilhermeleobas/264/base -> origin/gh/guilhermeleobas/264/base 2025-12-04T09:23:32.6806283Z * [new branch] gh/guilhermeleobas/264/head -> origin/gh/guilhermeleobas/264/head 2025-12-04T09:23:32.6806865Z * [new branch] gh/guilhermeleobas/264/orig -> origin/gh/guilhermeleobas/264/orig 2025-12-04T09:23:32.6808230Z * [new branch] gh/guilhermeleobas/265/base -> origin/gh/guilhermeleobas/265/base 2025-12-04T09:23:32.6808502Z * [new branch] gh/guilhermeleobas/265/head -> origin/gh/guilhermeleobas/265/head 2025-12-04T09:23:32.6811191Z * [new branch] gh/guilhermeleobas/265/orig -> origin/gh/guilhermeleobas/265/orig 2025-12-04T09:23:32.6811409Z * [new branch] gh/guilhermeleobas/266/base -> origin/gh/guilhermeleobas/266/base 2025-12-04T09:23:32.6811587Z * [new branch] gh/guilhermeleobas/266/head -> origin/gh/guilhermeleobas/266/head 2025-12-04T09:23:32.6811760Z * [new branch] gh/guilhermeleobas/266/orig -> origin/gh/guilhermeleobas/266/orig 2025-12-04T09:23:32.6813120Z * [new branch] gh/guilhermeleobas/267/base -> origin/gh/guilhermeleobas/267/base 2025-12-04T09:23:32.6813279Z * [new branch] gh/guilhermeleobas/267/head -> origin/gh/guilhermeleobas/267/head 2025-12-04T09:23:32.6815958Z * [new branch] gh/guilhermeleobas/267/orig -> origin/gh/guilhermeleobas/267/orig 2025-12-04T09:23:32.6816342Z * [new branch] gh/hameerabbasi/1/base -> origin/gh/hameerabbasi/1/base 2025-12-04T09:23:32.6816639Z * [new branch] gh/hameerabbasi/1/head -> origin/gh/hameerabbasi/1/head 2025-12-04T09:23:32.6816801Z * [new branch] gh/hameerabbasi/2/base -> origin/gh/hameerabbasi/2/base 2025-12-04T09:23:32.6817296Z * [new branch] gh/hameerabbasi/2/head -> origin/gh/hameerabbasi/2/head 2025-12-04T09:23:32.6818313Z * [new branch] gh/hameerabbasi/2/orig -> origin/gh/hameerabbasi/2/orig 2025-12-04T09:23:32.6818831Z * [new branch] gh/hameerabbasi/3/base -> origin/gh/hameerabbasi/3/base 2025-12-04T09:23:32.6822258Z * [new branch] gh/hameerabbasi/3/head -> origin/gh/hameerabbasi/3/head 2025-12-04T09:23:32.6822458Z * [new branch] gh/hameerabbasi/3/orig -> origin/gh/hameerabbasi/3/orig 2025-12-04T09:23:32.6822599Z * [new branch] gh/hameerabbasi/4/base -> origin/gh/hameerabbasi/4/base 2025-12-04T09:23:32.6822746Z * [new branch] gh/hameerabbasi/4/head -> origin/gh/hameerabbasi/4/head 2025-12-04T09:23:32.6822897Z * [new branch] gh/hameerabbasi/4/orig -> origin/gh/hameerabbasi/4/orig 2025-12-04T09:23:32.6825095Z * [new branch] gh/huydhn/1/next -> origin/gh/huydhn/1/next 2025-12-04T09:23:32.6825260Z * [new branch] gh/huydhn/2/next -> origin/gh/huydhn/2/next 2025-12-04T09:23:32.6825541Z * [new branch] gh/huydhn/3/next -> origin/gh/huydhn/3/next 2025-12-04T09:23:32.6826817Z * [new branch] gh/huydhn/4/next -> origin/gh/huydhn/4/next 2025-12-04T09:23:32.6827835Z * [new branch] gh/huydhn/5/next -> origin/gh/huydhn/5/next 2025-12-04T09:23:32.6828346Z * [new branch] gh/huydhn/6/next -> origin/gh/huydhn/6/next 2025-12-04T09:23:32.6829649Z * [new branch] gh/int3/97/base -> origin/gh/int3/97/base 2025-12-04T09:23:32.6830260Z * [new branch] gh/int3/97/head -> origin/gh/int3/97/head 2025-12-04T09:23:32.6831960Z * [new branch] gh/isuruf/101/base -> origin/gh/isuruf/101/base 2025-12-04T09:23:32.6832283Z * [new branch] gh/isuruf/101/head -> origin/gh/isuruf/101/head 2025-12-04T09:23:32.6832938Z * [new branch] gh/isuruf/146/base -> origin/gh/isuruf/146/base 2025-12-04T09:23:32.6833884Z * [new branch] gh/isuruf/146/head -> origin/gh/isuruf/146/head 2025-12-04T09:23:32.6834202Z * [new branch] gh/isuruf/146/orig -> origin/gh/isuruf/146/orig 2025-12-04T09:23:32.6836936Z * [new branch] gh/isuruf/158/base -> origin/gh/isuruf/158/base 2025-12-04T09:23:32.6837153Z * [new branch] gh/isuruf/158/head -> origin/gh/isuruf/158/head 2025-12-04T09:23:32.6837324Z * [new branch] gh/isuruf/159/base -> origin/gh/isuruf/159/base 2025-12-04T09:23:32.6837605Z * [new branch] gh/isuruf/159/head -> origin/gh/isuruf/159/head 2025-12-04T09:23:32.6839353Z * [new branch] gh/isuruf/160/base -> origin/gh/isuruf/160/base 2025-12-04T09:23:32.6839711Z * [new branch] gh/isuruf/160/head -> origin/gh/isuruf/160/head 2025-12-04T09:23:32.6839894Z * [new branch] gh/isuruf/160/orig -> origin/gh/isuruf/160/orig 2025-12-04T09:23:32.6841581Z * [new branch] gh/isuruf/81/base -> origin/gh/isuruf/81/base 2025-12-04T09:23:32.6841924Z * [new branch] gh/isuruf/81/head -> origin/gh/isuruf/81/head 2025-12-04T09:23:32.6842088Z * [new branch] gh/isuruf/81/orig -> origin/gh/isuruf/81/orig 2025-12-04T09:23:32.6844551Z * [new branch] gh/jamesjwu/176/base -> origin/gh/jamesjwu/176/base 2025-12-04T09:23:32.6844921Z * [new branch] gh/jamesjwu/176/head -> origin/gh/jamesjwu/176/head 2025-12-04T09:23:32.6845178Z * [new branch] gh/jamesjwu/176/orig -> origin/gh/jamesjwu/176/orig 2025-12-04T09:23:32.6845367Z * [new branch] gh/jamesjwu/187/base -> origin/gh/jamesjwu/187/base 2025-12-04T09:23:32.6845706Z * [new branch] gh/jamesjwu/187/head -> origin/gh/jamesjwu/187/head 2025-12-04T09:23:32.6846726Z * [new branch] gh/jamesjwu/187/orig -> origin/gh/jamesjwu/187/orig 2025-12-04T09:23:32.6847240Z * [new branch] gh/jamesjwu/196/base -> origin/gh/jamesjwu/196/base 2025-12-04T09:23:32.6849093Z * [new branch] gh/jamesjwu/196/head -> origin/gh/jamesjwu/196/head 2025-12-04T09:23:32.6849435Z * [new branch] gh/jamesjwu/196/orig -> origin/gh/jamesjwu/196/orig 2025-12-04T09:23:32.6849602Z * [new branch] gh/jamesjwu/198/base -> origin/gh/jamesjwu/198/base 2025-12-04T09:23:32.6850029Z * [new branch] gh/jamesjwu/198/head -> origin/gh/jamesjwu/198/head 2025-12-04T09:23:32.6850910Z * [new branch] gh/jamesjwu/198/orig -> origin/gh/jamesjwu/198/orig 2025-12-04T09:23:32.6853405Z * [new branch] gh/jamesjwu/207/base -> origin/gh/jamesjwu/207/base 2025-12-04T09:23:32.6853748Z * [new branch] gh/jamesjwu/207/head -> origin/gh/jamesjwu/207/head 2025-12-04T09:23:32.6853901Z * [new branch] gh/jamesjwu/207/orig -> origin/gh/jamesjwu/207/orig 2025-12-04T09:23:32.6854124Z * [new branch] gh/jamesjwu/208/base -> origin/gh/jamesjwu/208/base 2025-12-04T09:23:32.6854946Z * [new branch] gh/jamesjwu/208/head -> origin/gh/jamesjwu/208/head 2025-12-04T09:23:32.6855399Z * [new branch] gh/jamesjwu/208/orig -> origin/gh/jamesjwu/208/orig 2025-12-04T09:23:32.6858630Z * [new branch] gh/jamesjwu/52/base -> origin/gh/jamesjwu/52/base 2025-12-04T09:23:32.6858945Z * [new branch] gh/jamesjwu/52/head -> origin/gh/jamesjwu/52/head 2025-12-04T09:23:32.6859285Z * [new branch] gh/jamesjwu/53/base -> origin/gh/jamesjwu/53/base 2025-12-04T09:23:32.6859554Z * [new branch] gh/jamesjwu/53/head -> origin/gh/jamesjwu/53/head 2025-12-04T09:23:32.6859814Z * [new branch] gh/jamesjwu/54/base -> origin/gh/jamesjwu/54/base 2025-12-04T09:23:32.6859945Z * [new branch] gh/jamesjwu/54/head -> origin/gh/jamesjwu/54/head 2025-12-04T09:23:32.6862149Z * [new branch] gh/jamesjwu/55/base -> origin/gh/jamesjwu/55/base 2025-12-04T09:23:32.6862432Z * [new branch] gh/jamesjwu/55/head -> origin/gh/jamesjwu/55/head 2025-12-04T09:23:32.6862690Z * [new branch] gh/jamesjwu/56/base -> origin/gh/jamesjwu/56/base 2025-12-04T09:23:32.6862876Z * [new branch] gh/jamesjwu/56/head -> origin/gh/jamesjwu/56/head 2025-12-04T09:23:32.6864663Z * [new branch] gh/jamesjwu/57/base -> origin/gh/jamesjwu/57/base 2025-12-04T09:23:32.6865017Z * [new branch] gh/jamesjwu/57/head -> origin/gh/jamesjwu/57/head 2025-12-04T09:23:32.6865168Z * [new branch] gh/jamesjwu/58/base -> origin/gh/jamesjwu/58/base 2025-12-04T09:23:32.6865651Z * [new branch] gh/jamesjwu/58/head -> origin/gh/jamesjwu/58/head 2025-12-04T09:23:32.6867260Z * [new branch] gh/jamesjwu/59/base -> origin/gh/jamesjwu/59/base 2025-12-04T09:23:32.6867426Z * [new branch] gh/jamesjwu/59/head -> origin/gh/jamesjwu/59/head 2025-12-04T09:23:32.6868086Z * [new branch] gh/jamesjwu/60/base -> origin/gh/jamesjwu/60/base 2025-12-04T09:23:32.6868615Z * [new branch] gh/jamesjwu/60/head -> origin/gh/jamesjwu/60/head 2025-12-04T09:23:32.6869795Z * [new branch] gh/jamesjwu/61/base -> origin/gh/jamesjwu/61/base 2025-12-04T09:23:32.6870064Z * [new branch] gh/jamesjwu/61/head -> origin/gh/jamesjwu/61/head 2025-12-04T09:23:32.6871188Z * [new branch] gh/jamesjwu/62/base -> origin/gh/jamesjwu/62/base 2025-12-04T09:23:32.6871507Z * [new branch] gh/jamesjwu/62/head -> origin/gh/jamesjwu/62/head 2025-12-04T09:23:32.6872555Z * [new branch] gh/jamesjwu/63/base -> origin/gh/jamesjwu/63/base 2025-12-04T09:23:32.6873043Z * [new branch] gh/jamesjwu/63/head -> origin/gh/jamesjwu/63/head 2025-12-04T09:23:32.6877387Z * [new branch] gh/jamesjwu/64/base -> origin/gh/jamesjwu/64/base 2025-12-04T09:23:32.6877689Z * [new branch] gh/jamesjwu/64/head -> origin/gh/jamesjwu/64/head 2025-12-04T09:23:32.6877861Z * [new branch] gh/jamesjwu/65/base -> origin/gh/jamesjwu/65/base 2025-12-04T09:23:32.6878003Z * [new branch] gh/jamesjwu/65/head -> origin/gh/jamesjwu/65/head 2025-12-04T09:23:32.6878277Z * [new branch] gh/janeyx99/165/base -> origin/gh/janeyx99/165/base 2025-12-04T09:23:32.6878492Z * [new branch] gh/janeyx99/165/head -> origin/gh/janeyx99/165/head 2025-12-04T09:23:32.6882536Z * [new branch] gh/janeyx99/165/orig -> origin/gh/janeyx99/165/orig 2025-12-04T09:23:32.6882713Z * [new branch] gh/janeyx99/201/base -> origin/gh/janeyx99/201/base 2025-12-04T09:23:32.6883009Z * [new branch] gh/janeyx99/201/head -> origin/gh/janeyx99/201/head 2025-12-04T09:23:32.6883148Z * [new branch] gh/janeyx99/201/orig -> origin/gh/janeyx99/201/orig 2025-12-04T09:23:32.6887360Z * [new branch] gh/janeyx99/225/base -> origin/gh/janeyx99/225/base 2025-12-04T09:23:32.6887694Z * [new branch] gh/janeyx99/225/head -> origin/gh/janeyx99/225/head 2025-12-04T09:23:32.6887831Z * [new branch] gh/janeyx99/225/orig -> origin/gh/janeyx99/225/orig 2025-12-04T09:23:32.6887976Z * [new branch] gh/janeyx99/299/base -> origin/gh/janeyx99/299/base 2025-12-04T09:23:32.6888160Z * [new branch] gh/janeyx99/299/head -> origin/gh/janeyx99/299/head 2025-12-04T09:23:32.6888303Z * [new branch] gh/janeyx99/299/orig -> origin/gh/janeyx99/299/orig 2025-12-04T09:23:32.6891443Z * [new branch] gh/janeyx99/302/base -> origin/gh/janeyx99/302/base 2025-12-04T09:23:32.6891786Z * [new branch] gh/janeyx99/302/head -> origin/gh/janeyx99/302/head 2025-12-04T09:23:32.6891991Z * [new branch] gh/janeyx99/303/base -> origin/gh/janeyx99/303/base 2025-12-04T09:23:32.6892245Z * [new branch] gh/janeyx99/303/head -> origin/gh/janeyx99/303/head 2025-12-04T09:23:32.6897520Z * [new branch] gh/janeyx99/305/base -> origin/gh/janeyx99/305/base 2025-12-04T09:23:32.6897876Z * [new branch] gh/janeyx99/305/head -> origin/gh/janeyx99/305/head 2025-12-04T09:23:32.6898132Z * [new branch] gh/janeyx99/306/base -> origin/gh/janeyx99/306/base 2025-12-04T09:23:32.6898296Z * [new branch] gh/janeyx99/306/head -> origin/gh/janeyx99/306/head 2025-12-04T09:23:32.6898581Z * [new branch] gh/janeyx99/314/base -> origin/gh/janeyx99/314/base 2025-12-04T09:23:32.6898753Z * [new branch] gh/janeyx99/314/head -> origin/gh/janeyx99/314/head 2025-12-04T09:23:32.6899032Z * [new branch] gh/janeyx99/314/orig -> origin/gh/janeyx99/314/orig 2025-12-04T09:23:32.6900416Z * [new branch] gh/janeyx99/315/base -> origin/gh/janeyx99/315/base 2025-12-04T09:23:32.6900597Z * [new branch] gh/janeyx99/315/head -> origin/gh/janeyx99/315/head 2025-12-04T09:23:32.6900733Z * [new branch] gh/janeyx99/315/orig -> origin/gh/janeyx99/315/orig 2025-12-04T09:23:32.6900876Z * [new branch] gh/janeyx99/316/base -> origin/gh/janeyx99/316/base 2025-12-04T09:23:32.6901015Z * [new branch] gh/janeyx99/316/head -> origin/gh/janeyx99/316/head 2025-12-04T09:23:32.6901281Z * [new branch] gh/janeyx99/316/orig -> origin/gh/janeyx99/316/orig 2025-12-04T09:23:32.6907089Z * [new branch] gh/janeyx99/317/base -> origin/gh/janeyx99/317/base 2025-12-04T09:23:32.6907297Z * [new branch] gh/janeyx99/317/head -> origin/gh/janeyx99/317/head 2025-12-04T09:23:32.6907453Z * [new branch] gh/janeyx99/317/orig -> origin/gh/janeyx99/317/orig 2025-12-04T09:23:32.6907588Z * [new branch] gh/janeyx99/325/base -> origin/gh/janeyx99/325/base 2025-12-04T09:23:32.6907769Z * [new branch] gh/janeyx99/325/head -> origin/gh/janeyx99/325/head 2025-12-04T09:23:32.6907913Z * [new branch] gh/janeyx99/325/orig -> origin/gh/janeyx99/325/orig 2025-12-04T09:23:32.6908059Z * [new branch] gh/janeyx99/327/base -> origin/gh/janeyx99/327/base 2025-12-04T09:23:32.6926164Z * [new branch] gh/janeyx99/327/head -> origin/gh/janeyx99/327/head 2025-12-04T09:23:32.6931472Z * [new branch] gh/janeyx99/327/orig -> origin/gh/janeyx99/327/orig 2025-12-04T09:23:32.6931814Z * [new branch] gh/janeyx99/328/base -> origin/gh/janeyx99/328/base 2025-12-04T09:23:32.6932058Z * [new branch] gh/janeyx99/328/head -> origin/gh/janeyx99/328/head 2025-12-04T09:23:32.6932207Z * [new branch] gh/janeyx99/328/orig -> origin/gh/janeyx99/328/orig 2025-12-04T09:23:32.6932365Z * [new branch] gh/janeyx99/329/base -> origin/gh/janeyx99/329/base 2025-12-04T09:23:32.6932884Z * [new branch] gh/janeyx99/329/head -> origin/gh/janeyx99/329/head 2025-12-04T09:23:32.6933029Z * [new branch] gh/janeyx99/329/orig -> origin/gh/janeyx99/329/orig 2025-12-04T09:23:32.6933163Z * [new branch] gh/janeyx99/330/base -> origin/gh/janeyx99/330/base 2025-12-04T09:23:32.6933370Z * [new branch] gh/janeyx99/330/head -> origin/gh/janeyx99/330/head 2025-12-04T09:23:32.6933515Z * [new branch] gh/janeyx99/330/orig -> origin/gh/janeyx99/330/orig 2025-12-04T09:23:32.6933646Z * [new branch] gh/janeyx99/331/base -> origin/gh/janeyx99/331/base 2025-12-04T09:23:32.6933784Z * [new branch] gh/janeyx99/331/head -> origin/gh/janeyx99/331/head 2025-12-04T09:23:32.6933918Z * [new branch] gh/janeyx99/331/orig -> origin/gh/janeyx99/331/orig 2025-12-04T09:23:32.6934055Z * [new branch] gh/janeyx99/332/base -> origin/gh/janeyx99/332/base 2025-12-04T09:23:32.6934205Z * [new branch] gh/janeyx99/332/head -> origin/gh/janeyx99/332/head 2025-12-04T09:23:32.6934342Z * [new branch] gh/janeyx99/332/orig -> origin/gh/janeyx99/332/orig 2025-12-04T09:23:32.6934610Z * [new branch] gh/janeyx99/333/base -> origin/gh/janeyx99/333/base 2025-12-04T09:23:32.6934746Z * [new branch] gh/janeyx99/333/head -> origin/gh/janeyx99/333/head 2025-12-04T09:23:32.6934886Z * [new branch] gh/janeyx99/333/orig -> origin/gh/janeyx99/333/orig 2025-12-04T09:23:32.6935034Z * [new branch] gh/janeyx99/88/base -> origin/gh/janeyx99/88/base 2025-12-04T09:23:32.6935170Z * [new branch] gh/janeyx99/88/head -> origin/gh/janeyx99/88/head 2025-12-04T09:23:32.6935320Z * [new branch] gh/janeyx99/88/orig -> origin/gh/janeyx99/88/orig 2025-12-04T09:23:32.6941731Z * [new branch] gh/jansel/360/base -> origin/gh/jansel/360/base 2025-12-04T09:23:32.6941903Z * [new branch] gh/jansel/360/head -> origin/gh/jansel/360/head 2025-12-04T09:23:32.6942398Z * [new branch] gh/jansel/451/base -> origin/gh/jansel/451/base 2025-12-04T09:23:32.6942587Z * [new branch] gh/jansel/451/head -> origin/gh/jansel/451/head 2025-12-04T09:23:32.6942722Z * [new branch] gh/jansel/451/orig -> origin/gh/jansel/451/orig 2025-12-04T09:23:32.6942862Z * [new branch] gh/jansel/462/base -> origin/gh/jansel/462/base 2025-12-04T09:23:32.6942993Z * [new branch] gh/jansel/462/head -> origin/gh/jansel/462/head 2025-12-04T09:23:32.6946657Z * [new branch] gh/jansel/462/orig -> origin/gh/jansel/462/orig 2025-12-04T09:23:32.6946844Z * [new branch] gh/jansel/533/base -> origin/gh/jansel/533/base 2025-12-04T09:23:32.6946999Z * [new branch] gh/jansel/533/head -> origin/gh/jansel/533/head 2025-12-04T09:23:32.6947135Z * [new branch] gh/jansel/533/orig -> origin/gh/jansel/533/orig 2025-12-04T09:23:32.6947278Z * [new branch] gh/jansel/552/base -> origin/gh/jansel/552/base 2025-12-04T09:23:32.6947425Z * [new branch] gh/jansel/552/head -> origin/gh/jansel/552/head 2025-12-04T09:23:32.6947552Z * [new branch] gh/jansel/552/orig -> origin/gh/jansel/552/orig 2025-12-04T09:23:32.6948851Z * [new branch] gh/jansel/553/base -> origin/gh/jansel/553/base 2025-12-04T09:23:32.6949195Z * [new branch] gh/jansel/553/head -> origin/gh/jansel/553/head 2025-12-04T09:23:32.6949357Z * [new branch] gh/jansel/553/orig -> origin/gh/jansel/553/orig 2025-12-04T09:23:32.6949487Z * [new branch] gh/jansel/554/base -> origin/gh/jansel/554/base 2025-12-04T09:23:32.6950002Z * [new branch] gh/jansel/554/head -> origin/gh/jansel/554/head 2025-12-04T09:23:32.6950324Z * [new branch] gh/jansel/554/orig -> origin/gh/jansel/554/orig 2025-12-04T09:23:32.6950471Z * [new branch] gh/jansel/555/base -> origin/gh/jansel/555/base 2025-12-04T09:23:32.6950656Z * [new branch] gh/jansel/555/head -> origin/gh/jansel/555/head 2025-12-04T09:23:32.6951167Z * [new branch] gh/jansel/555/orig -> origin/gh/jansel/555/orig 2025-12-04T09:23:32.6952996Z * [new branch] gh/jansel/556/base -> origin/gh/jansel/556/base 2025-12-04T09:23:32.6953259Z * [new branch] gh/jansel/556/head -> origin/gh/jansel/556/head 2025-12-04T09:23:32.6960166Z * [new branch] gh/jansel/556/orig -> origin/gh/jansel/556/orig 2025-12-04T09:23:32.6960337Z * [new branch] gh/jansel/557/base -> origin/gh/jansel/557/base 2025-12-04T09:23:32.6960485Z * [new branch] gh/jansel/557/head -> origin/gh/jansel/557/head 2025-12-04T09:23:32.6960610Z * [new branch] gh/jansel/557/orig -> origin/gh/jansel/557/orig 2025-12-04T09:23:32.6960732Z * [new branch] gh/jansel/558/base -> origin/gh/jansel/558/base 2025-12-04T09:23:32.6960871Z * [new branch] gh/jansel/558/head -> origin/gh/jansel/558/head 2025-12-04T09:23:32.6961000Z * [new branch] gh/jansel/558/orig -> origin/gh/jansel/558/orig 2025-12-04T09:23:32.6966618Z * [new branch] gh/jansel/559/base -> origin/gh/jansel/559/base 2025-12-04T09:23:32.6970714Z * [new branch] gh/jansel/559/head -> origin/gh/jansel/559/head 2025-12-04T09:23:32.6974629Z * [new branch] gh/jansel/559/orig -> origin/gh/jansel/559/orig 2025-12-04T09:23:32.6976518Z * [new branch] gh/jansel/560/base -> origin/gh/jansel/560/base 2025-12-04T09:23:32.6976689Z * [new branch] gh/jansel/560/head -> origin/gh/jansel/560/head 2025-12-04T09:23:32.6976953Z * [new branch] gh/jansel/560/orig -> origin/gh/jansel/560/orig 2025-12-04T09:23:32.6981795Z * [new branch] gh/jansel/561/base -> origin/gh/jansel/561/base 2025-12-04T09:23:32.6985370Z * [new branch] gh/jansel/561/head -> origin/gh/jansel/561/head 2025-12-04T09:23:32.6987298Z * [new branch] gh/jansel/561/orig -> origin/gh/jansel/561/orig 2025-12-04T09:23:32.6987436Z * [new branch] gh/jansel/562/base -> origin/gh/jansel/562/base 2025-12-04T09:23:32.6987572Z * [new branch] gh/jansel/562/head -> origin/gh/jansel/562/head 2025-12-04T09:23:32.6987697Z * [new branch] gh/jansel/562/orig -> origin/gh/jansel/562/orig 2025-12-04T09:23:32.6987822Z * [new branch] gh/jansel/563/base -> origin/gh/jansel/563/base 2025-12-04T09:23:32.6987966Z * [new branch] gh/jansel/563/head -> origin/gh/jansel/563/head 2025-12-04T09:23:32.6988089Z * [new branch] gh/jansel/563/orig -> origin/gh/jansel/563/orig 2025-12-04T09:23:32.6988219Z * [new branch] gh/jansel/564/base -> origin/gh/jansel/564/base 2025-12-04T09:23:32.6988348Z * [new branch] gh/jansel/564/head -> origin/gh/jansel/564/head 2025-12-04T09:23:32.6988472Z * [new branch] gh/jansel/564/orig -> origin/gh/jansel/564/orig 2025-12-04T09:23:32.6988599Z * [new branch] gh/jansel/565/base -> origin/gh/jansel/565/base 2025-12-04T09:23:32.6988722Z * [new branch] gh/jansel/565/head -> origin/gh/jansel/565/head 2025-12-04T09:23:32.6988867Z * [new branch] gh/jansel/565/orig -> origin/gh/jansel/565/orig 2025-12-04T09:23:32.6989224Z * [new branch] gh/jansel/566/base -> origin/gh/jansel/566/base 2025-12-04T09:23:32.6989348Z * [new branch] gh/jansel/566/head -> origin/gh/jansel/566/head 2025-12-04T09:23:32.6989481Z * [new branch] gh/jansel/566/orig -> origin/gh/jansel/566/orig 2025-12-04T09:23:32.6989605Z * [new branch] gh/jansel/567/base -> origin/gh/jansel/567/base 2025-12-04T09:23:32.6989781Z * [new branch] gh/jansel/567/head -> origin/gh/jansel/567/head 2025-12-04T09:23:32.6989905Z * [new branch] gh/jansel/567/orig -> origin/gh/jansel/567/orig 2025-12-04T09:23:32.6990027Z * [new branch] gh/jansel/568/base -> origin/gh/jansel/568/base 2025-12-04T09:23:32.6990156Z * [new branch] gh/jansel/568/head -> origin/gh/jansel/568/head 2025-12-04T09:23:32.6990281Z * [new branch] gh/jansel/568/orig -> origin/gh/jansel/568/orig 2025-12-04T09:23:32.6990404Z * [new branch] gh/jansel/569/base -> origin/gh/jansel/569/base 2025-12-04T09:23:32.6990533Z * [new branch] gh/jansel/569/head -> origin/gh/jansel/569/head 2025-12-04T09:23:32.6990655Z * [new branch] gh/jansel/569/orig -> origin/gh/jansel/569/orig 2025-12-04T09:23:32.6990786Z * [new branch] gh/jansel/570/base -> origin/gh/jansel/570/base 2025-12-04T09:23:32.6990909Z * [new branch] gh/jansel/570/head -> origin/gh/jansel/570/head 2025-12-04T09:23:32.6991031Z * [new branch] gh/jansel/570/orig -> origin/gh/jansel/570/orig 2025-12-04T09:23:32.6991160Z * [new branch] gh/jansel/571/base -> origin/gh/jansel/571/base 2025-12-04T09:23:32.6991284Z * [new branch] gh/jansel/571/head -> origin/gh/jansel/571/head 2025-12-04T09:23:32.6991414Z * [new branch] gh/jansel/571/orig -> origin/gh/jansel/571/orig 2025-12-04T09:23:32.6991543Z * [new branch] gh/jansel/572/base -> origin/gh/jansel/572/base 2025-12-04T09:23:32.6991670Z * [new branch] gh/jansel/572/head -> origin/gh/jansel/572/head 2025-12-04T09:23:32.6991802Z * [new branch] gh/jansel/572/orig -> origin/gh/jansel/572/orig 2025-12-04T09:23:32.6991928Z * [new branch] gh/jansel/573/base -> origin/gh/jansel/573/base 2025-12-04T09:23:32.6992053Z * [new branch] gh/jansel/573/head -> origin/gh/jansel/573/head 2025-12-04T09:23:32.6992192Z * [new branch] gh/jansel/573/orig -> origin/gh/jansel/573/orig 2025-12-04T09:23:32.6992318Z * [new branch] gh/jansel/574/base -> origin/gh/jansel/574/base 2025-12-04T09:23:32.6992449Z * [new branch] gh/jansel/574/head -> origin/gh/jansel/574/head 2025-12-04T09:23:32.6994815Z * [new branch] gh/jansel/574/orig -> origin/gh/jansel/574/orig 2025-12-04T09:23:32.6994959Z * [new branch] gh/jansel/575/base -> origin/gh/jansel/575/base 2025-12-04T09:23:32.7002004Z * [new branch] gh/jansel/575/head -> origin/gh/jansel/575/head 2025-12-04T09:23:32.7002169Z * [new branch] gh/jansel/575/orig -> origin/gh/jansel/575/orig 2025-12-04T09:23:32.7006321Z * [new branch] gh/jansel/576/base -> origin/gh/jansel/576/base 2025-12-04T09:23:32.7010458Z * [new branch] gh/jansel/576/head -> origin/gh/jansel/576/head 2025-12-04T09:23:32.7015550Z * [new branch] gh/jansel/576/orig -> origin/gh/jansel/576/orig 2025-12-04T09:23:32.7017946Z * [new branch] gh/jbschlosser/247/base -> origin/gh/jbschlosser/247/base 2025-12-04T09:23:32.7021314Z * [new branch] gh/jbschlosser/247/head -> origin/gh/jbschlosser/247/head 2025-12-04T09:23:32.7021496Z * [new branch] gh/jbschlosser/247/orig -> origin/gh/jbschlosser/247/orig 2025-12-04T09:23:32.7021871Z * [new branch] gh/jbschlosser/250/base -> origin/gh/jbschlosser/250/base 2025-12-04T09:23:32.7022023Z * [new branch] gh/jbschlosser/250/head -> origin/gh/jbschlosser/250/head 2025-12-04T09:23:32.7022165Z * [new branch] gh/jbschlosser/250/orig -> origin/gh/jbschlosser/250/orig 2025-12-04T09:23:32.7022374Z * [new branch] gh/jerryzh168/1/base -> origin/gh/jerryzh168/1/base 2025-12-04T09:23:32.7022528Z * [new branch] gh/jerryzh168/1/head -> origin/gh/jerryzh168/1/head 2025-12-04T09:23:32.7022661Z * [new branch] gh/jerryzh168/1/orig -> origin/gh/jerryzh168/1/orig 2025-12-04T09:23:32.7022803Z * [new branch] gh/jiayisunx/59/base -> origin/gh/jiayisunx/59/base 2025-12-04T09:23:32.7022938Z * [new branch] gh/jiayisunx/59/head -> origin/gh/jiayisunx/59/head 2025-12-04T09:23:32.7023066Z * [new branch] gh/jiayisunx/59/orig -> origin/gh/jiayisunx/59/orig 2025-12-04T09:23:32.7023211Z * [new branch] gh/jiayisunx/61/base -> origin/gh/jiayisunx/61/base 2025-12-04T09:23:32.7023342Z * [new branch] gh/jiayisunx/61/head -> origin/gh/jiayisunx/61/head 2025-12-04T09:23:32.7023482Z * [new branch] gh/jiayisunx/61/orig -> origin/gh/jiayisunx/61/orig 2025-12-04T09:23:32.7023617Z * [new branch] gh/jiayisunx/68/base -> origin/gh/jiayisunx/68/base 2025-12-04T09:23:32.7023748Z * [new branch] gh/jiayisunx/68/head -> origin/gh/jiayisunx/68/head 2025-12-04T09:23:32.7023887Z * [new branch] gh/jiayisunx/68/orig -> origin/gh/jiayisunx/68/orig 2025-12-04T09:23:32.7024016Z * [new branch] gh/jiayisunx/77/base -> origin/gh/jiayisunx/77/base 2025-12-04T09:23:32.7024147Z * [new branch] gh/jiayisunx/77/head -> origin/gh/jiayisunx/77/head 2025-12-04T09:23:32.7024290Z * [new branch] gh/jiayisunx/77/orig -> origin/gh/jiayisunx/77/orig 2025-12-04T09:23:32.7024421Z * [new branch] gh/jiayisunx/78/base -> origin/gh/jiayisunx/78/base 2025-12-04T09:23:32.7024569Z * [new branch] gh/jiayisunx/78/head -> origin/gh/jiayisunx/78/head 2025-12-04T09:23:32.7024699Z * [new branch] gh/jiayisunx/78/orig -> origin/gh/jiayisunx/78/orig 2025-12-04T09:23:32.7024825Z * [new branch] gh/jiayisunx/79/base -> origin/gh/jiayisunx/79/base 2025-12-04T09:23:32.7024960Z * [new branch] gh/jiayisunx/79/head -> origin/gh/jiayisunx/79/head 2025-12-04T09:23:32.7025112Z * [new branch] gh/jiayisunx/79/orig -> origin/gh/jiayisunx/79/orig 2025-12-04T09:23:32.7025250Z * [new branch] gh/jiayisunx/82/base -> origin/gh/jiayisunx/82/base 2025-12-04T09:23:32.7025380Z * [new branch] gh/jiayisunx/82/head -> origin/gh/jiayisunx/82/head 2025-12-04T09:23:32.7025510Z * [new branch] gh/jiayisunx/82/orig -> origin/gh/jiayisunx/82/orig 2025-12-04T09:23:32.7025659Z * [new branch] gh/jiayisunx/83/base -> origin/gh/jiayisunx/83/base 2025-12-04T09:23:32.7025788Z * [new branch] gh/jiayisunx/83/head -> origin/gh/jiayisunx/83/head 2025-12-04T09:23:32.7025927Z * [new branch] gh/jiayisunx/83/orig -> origin/gh/jiayisunx/83/orig 2025-12-04T09:23:32.7026056Z * [new branch] gh/jiayisunx/84/base -> origin/gh/jiayisunx/84/base 2025-12-04T09:23:32.7026185Z * [new branch] gh/jiayisunx/84/head -> origin/gh/jiayisunx/84/head 2025-12-04T09:23:32.7026321Z * [new branch] gh/jiayisunx/84/orig -> origin/gh/jiayisunx/84/orig 2025-12-04T09:23:32.7029330Z * [new branch] gh/jiayisunx/85/base -> origin/gh/jiayisunx/85/base 2025-12-04T09:23:32.7029903Z * [new branch] gh/jiayisunx/85/head -> origin/gh/jiayisunx/85/head 2025-12-04T09:23:32.7030269Z * [new branch] gh/jiayisunx/85/orig -> origin/gh/jiayisunx/85/orig 2025-12-04T09:23:32.7030418Z * [new branch] gh/jiayisunx/86/base -> origin/gh/jiayisunx/86/base 2025-12-04T09:23:32.7030556Z * [new branch] gh/jiayisunx/86/head -> origin/gh/jiayisunx/86/head 2025-12-04T09:23:32.7030830Z * [new branch] gh/jiayisunx/86/orig -> origin/gh/jiayisunx/86/orig 2025-12-04T09:23:32.7030970Z * [new branch] gh/jiayisunx/87/base -> origin/gh/jiayisunx/87/base 2025-12-04T09:23:32.7031109Z * [new branch] gh/jiayisunx/87/head -> origin/gh/jiayisunx/87/head 2025-12-04T09:23:32.7031249Z * [new branch] gh/jiayisunx/87/orig -> origin/gh/jiayisunx/87/orig 2025-12-04T09:23:32.7031383Z * [new branch] gh/jiayisunx/88/base -> origin/gh/jiayisunx/88/base 2025-12-04T09:23:32.7031543Z * [new branch] gh/jiayisunx/88/head -> origin/gh/jiayisunx/88/head 2025-12-04T09:23:32.7034012Z * [new branch] gh/jiayisunx/88/orig -> origin/gh/jiayisunx/88/orig 2025-12-04T09:23:32.7034276Z * [new branch] gh/jiayisunx/89/base -> origin/gh/jiayisunx/89/base 2025-12-04T09:23:32.7034477Z * [new branch] gh/jiayisunx/89/head -> origin/gh/jiayisunx/89/head 2025-12-04T09:23:32.7034635Z * [new branch] gh/jiayisunx/89/orig -> origin/gh/jiayisunx/89/orig 2025-12-04T09:23:32.7034870Z * [new branch] gh/jiayisunx/90/base -> origin/gh/jiayisunx/90/base 2025-12-04T09:23:32.7040487Z * [new branch] gh/jiayisunx/90/head -> origin/gh/jiayisunx/90/head 2025-12-04T09:23:32.7045904Z * [new branch] gh/jiayisunx/90/orig -> origin/gh/jiayisunx/90/orig 2025-12-04T09:23:32.7048279Z * [new branch] gh/jjwu@meta.com/1/base -> origin/gh/jjwu@meta.com/1/base 2025-12-04T09:23:32.7048490Z * [new branch] gh/jjwu@meta.com/1/head -> origin/gh/jjwu@meta.com/1/head 2025-12-04T09:23:32.7048655Z * [new branch] gh/jturney/1/base -> origin/gh/jturney/1/base 2025-12-04T09:23:32.7048793Z * [new branch] gh/jturney/1/head -> origin/gh/jturney/1/head 2025-12-04T09:23:32.7048946Z * [new branch] gh/jturney/1/orig -> origin/gh/jturney/1/orig 2025-12-04T09:23:32.7049082Z * [new branch] gh/jturney/2/base -> origin/gh/jturney/2/base 2025-12-04T09:23:32.7049216Z * [new branch] gh/jturney/2/head -> origin/gh/jturney/2/head 2025-12-04T09:23:32.7049361Z * [new branch] gh/jturney/2/orig -> origin/gh/jturney/2/orig 2025-12-04T09:23:32.7049518Z * [new branch] gh/karthickai/10/base -> origin/gh/karthickai/10/base 2025-12-04T09:23:32.7049673Z * [new branch] gh/karthickai/10/head -> origin/gh/karthickai/10/head 2025-12-04T09:23:32.7049823Z * [new branch] gh/karthickai/10/orig -> origin/gh/karthickai/10/orig 2025-12-04T09:23:32.7049967Z * [new branch] gh/karthickai/11/base -> origin/gh/karthickai/11/base 2025-12-04T09:23:32.7050118Z * [new branch] gh/karthickai/11/head -> origin/gh/karthickai/11/head 2025-12-04T09:23:32.7050265Z * [new branch] gh/karthickai/11/orig -> origin/gh/karthickai/11/orig 2025-12-04T09:23:32.7050415Z * [new branch] gh/karthickai/12/base -> origin/gh/karthickai/12/base 2025-12-04T09:23:32.7054015Z * [new branch] gh/karthickai/12/head -> origin/gh/karthickai/12/head 2025-12-04T09:23:32.7058192Z * [new branch] gh/karthickai/12/orig -> origin/gh/karthickai/12/orig 2025-12-04T09:23:32.7063417Z * [new branch] gh/karthickai/13/base -> origin/gh/karthickai/13/base 2025-12-04T09:23:32.7067669Z * [new branch] gh/karthickai/13/head -> origin/gh/karthickai/13/head 2025-12-04T09:23:32.7068045Z * [new branch] gh/karthickai/13/orig -> origin/gh/karthickai/13/orig 2025-12-04T09:23:32.7068192Z * [new branch] gh/karthickai/14/base -> origin/gh/karthickai/14/base 2025-12-04T09:23:32.7068329Z * [new branch] gh/karthickai/14/head -> origin/gh/karthickai/14/head 2025-12-04T09:23:32.7068532Z * [new branch] gh/karthickai/14/orig -> origin/gh/karthickai/14/orig 2025-12-04T09:23:32.7068672Z * [new branch] gh/karthickai/15/base -> origin/gh/karthickai/15/base 2025-12-04T09:23:32.7068817Z * [new branch] gh/karthickai/15/head -> origin/gh/karthickai/15/head 2025-12-04T09:23:32.7068954Z * [new branch] gh/karthickai/15/orig -> origin/gh/karthickai/15/orig 2025-12-04T09:23:32.7069089Z * [new branch] gh/karthickai/16/base -> origin/gh/karthickai/16/base 2025-12-04T09:23:32.7069231Z * [new branch] gh/karthickai/16/head -> origin/gh/karthickai/16/head 2025-12-04T09:23:32.7069369Z * [new branch] gh/karthickai/16/orig -> origin/gh/karthickai/16/orig 2025-12-04T09:23:32.7069510Z * [new branch] gh/karthickai/17/base -> origin/gh/karthickai/17/base 2025-12-04T09:23:32.7069647Z * [new branch] gh/karthickai/17/head -> origin/gh/karthickai/17/head 2025-12-04T09:23:32.7069784Z * [new branch] gh/karthickai/17/orig -> origin/gh/karthickai/17/orig 2025-12-04T09:23:32.7069929Z * [new branch] gh/karthickai/18/base -> origin/gh/karthickai/18/base 2025-12-04T09:23:32.7070066Z * [new branch] gh/karthickai/18/head -> origin/gh/karthickai/18/head 2025-12-04T09:23:32.7070210Z * [new branch] gh/karthickai/18/orig -> origin/gh/karthickai/18/orig 2025-12-04T09:23:32.7070347Z * [new branch] gh/karthickai/19/base -> origin/gh/karthickai/19/base 2025-12-04T09:23:32.7070487Z * [new branch] gh/karthickai/19/head -> origin/gh/karthickai/19/head 2025-12-04T09:23:32.7070631Z * [new branch] gh/karthickai/19/orig -> origin/gh/karthickai/19/orig 2025-12-04T09:23:32.7070770Z * [new branch] gh/karthickai/20/base -> origin/gh/karthickai/20/base 2025-12-04T09:23:32.7070923Z * [new branch] gh/karthickai/20/head -> origin/gh/karthickai/20/head 2025-12-04T09:23:32.7071098Z * [new branch] gh/karthickai/20/orig -> origin/gh/karthickai/20/orig 2025-12-04T09:23:32.7072462Z * [new branch] gh/karthickai/21/base -> origin/gh/karthickai/21/base 2025-12-04T09:23:32.7073015Z * [new branch] gh/karthickai/21/head -> origin/gh/karthickai/21/head 2025-12-04T09:23:32.7079000Z * [new branch] gh/karthickai/21/orig -> origin/gh/karthickai/21/orig 2025-12-04T09:23:32.7080017Z * [new branch] gh/karthickai/22/base -> origin/gh/karthickai/22/base 2025-12-04T09:23:32.7080190Z * [new branch] gh/karthickai/22/head -> origin/gh/karthickai/22/head 2025-12-04T09:23:32.7080326Z * [new branch] gh/karthickai/22/orig -> origin/gh/karthickai/22/orig 2025-12-04T09:23:32.7080478Z * [new branch] gh/karthickai/23/base -> origin/gh/karthickai/23/base 2025-12-04T09:23:32.7080613Z * [new branch] gh/karthickai/23/head -> origin/gh/karthickai/23/head 2025-12-04T09:23:32.7080753Z * [new branch] gh/karthickai/23/orig -> origin/gh/karthickai/23/orig 2025-12-04T09:23:32.7080883Z * [new branch] gh/karthickai/24/base -> origin/gh/karthickai/24/base 2025-12-04T09:23:32.7081015Z * [new branch] gh/karthickai/24/head -> origin/gh/karthickai/24/head 2025-12-04T09:23:32.7081177Z * [new branch] gh/karthickai/24/orig -> origin/gh/karthickai/24/orig 2025-12-04T09:23:32.7085491Z * [new branch] gh/karthickai/25/base -> origin/gh/karthickai/25/base 2025-12-04T09:23:32.7086005Z * [new branch] gh/karthickai/25/head -> origin/gh/karthickai/25/head 2025-12-04T09:23:32.7086161Z * [new branch] gh/karthickai/25/orig -> origin/gh/karthickai/25/orig 2025-12-04T09:23:32.7086298Z * [new branch] gh/karthickai/26/base -> origin/gh/karthickai/26/base 2025-12-04T09:23:32.7086616Z * [new branch] gh/karthickai/26/head -> origin/gh/karthickai/26/head 2025-12-04T09:23:32.7087119Z * [new branch] gh/karthickai/26/orig -> origin/gh/karthickai/26/orig 2025-12-04T09:23:32.7089117Z * [new branch] gh/karthickai/6/base -> origin/gh/karthickai/6/base 2025-12-04T09:23:32.7089426Z * [new branch] gh/karthickai/6/head -> origin/gh/karthickai/6/head 2025-12-04T09:23:32.7091475Z * [new branch] gh/karthickai/6/orig -> origin/gh/karthickai/6/orig 2025-12-04T09:23:32.7091946Z * [new branch] gh/krocki/1/base -> origin/gh/krocki/1/base 2025-12-04T09:23:32.7092081Z * [new branch] gh/krocki/1/head -> origin/gh/krocki/1/head 2025-12-04T09:23:32.7094468Z * [new branch] gh/krocki/1/orig -> origin/gh/krocki/1/orig 2025-12-04T09:23:32.7095003Z * [new branch] gh/krocki/2/base -> origin/gh/krocki/2/base 2025-12-04T09:23:32.7095168Z * [new branch] gh/krocki/2/head -> origin/gh/krocki/2/head 2025-12-04T09:23:32.7095311Z * [new branch] gh/krocki/2/orig -> origin/gh/krocki/2/orig 2025-12-04T09:23:32.7096847Z * [new branch] gh/kurtamohler/60/base -> origin/gh/kurtamohler/60/base 2025-12-04T09:23:32.7097187Z * [new branch] gh/kurtamohler/60/head -> origin/gh/kurtamohler/60/head 2025-12-04T09:23:32.7097689Z * [new branch] gh/kurtamohler/60/orig -> origin/gh/kurtamohler/60/orig 2025-12-04T09:23:32.7099144Z * [new branch] gh/kurtamohler/61/base -> origin/gh/kurtamohler/61/base 2025-12-04T09:23:32.7099357Z * [new branch] gh/kurtamohler/61/head -> origin/gh/kurtamohler/61/head 2025-12-04T09:23:32.7102170Z * [new branch] gh/kurtamohler/61/orig -> origin/gh/kurtamohler/61/orig 2025-12-04T09:23:32.7102510Z * [new branch] gh/kurtamohler/62/base -> origin/gh/kurtamohler/62/base 2025-12-04T09:23:32.7102687Z * [new branch] gh/kurtamohler/62/head -> origin/gh/kurtamohler/62/head 2025-12-04T09:23:32.7102844Z * [new branch] gh/kurtamohler/62/orig -> origin/gh/kurtamohler/62/orig 2025-12-04T09:23:32.7103007Z * [new branch] gh/kurtamohler/63/base -> origin/gh/kurtamohler/63/base 2025-12-04T09:23:32.7104145Z * [new branch] gh/kurtamohler/63/head -> origin/gh/kurtamohler/63/head 2025-12-04T09:23:32.7104338Z * [new branch] gh/kurtamohler/63/orig -> origin/gh/kurtamohler/63/orig 2025-12-04T09:23:32.7107117Z * [new branch] gh/kurtamohler/64/base -> origin/gh/kurtamohler/64/base 2025-12-04T09:23:32.7107291Z * [new branch] gh/kurtamohler/64/head -> origin/gh/kurtamohler/64/head 2025-12-04T09:23:32.7109564Z * [new branch] gh/kurtamohler/64/orig -> origin/gh/kurtamohler/64/orig 2025-12-04T09:23:32.7109886Z * [new branch] gh/kurtamohler/65/base -> origin/gh/kurtamohler/65/base 2025-12-04T09:23:32.7110035Z * [new branch] gh/kurtamohler/65/head -> origin/gh/kurtamohler/65/head 2025-12-04T09:23:32.7110186Z * [new branch] gh/kurtamohler/65/orig -> origin/gh/kurtamohler/65/orig 2025-12-04T09:23:32.7110381Z * [new branch] gh/kurtamohler/66/base -> origin/gh/kurtamohler/66/base 2025-12-04T09:23:32.7110805Z * [new branch] gh/kurtamohler/66/head -> origin/gh/kurtamohler/66/head 2025-12-04T09:23:32.7111399Z * [new branch] gh/kurtamohler/66/orig -> origin/gh/kurtamohler/66/orig 2025-12-04T09:23:32.7113231Z * [new branch] gh/kurtamohler/67/base -> origin/gh/kurtamohler/67/base 2025-12-04T09:23:32.7113428Z * [new branch] gh/kurtamohler/67/head -> origin/gh/kurtamohler/67/head 2025-12-04T09:23:32.7120340Z * [new branch] gh/kurtamohler/67/orig -> origin/gh/kurtamohler/67/orig 2025-12-04T09:23:32.7122896Z * [new branch] gh/kwen2501/130/base -> origin/gh/kwen2501/130/base 2025-12-04T09:23:32.7123189Z * [new branch] gh/kwen2501/130/head -> origin/gh/kwen2501/130/head 2025-12-04T09:23:32.7126112Z * [new branch] gh/kwen2501/130/orig -> origin/gh/kwen2501/130/orig 2025-12-04T09:23:32.7126288Z * [new branch] gh/kwen2501/170/base -> origin/gh/kwen2501/170/base 2025-12-04T09:23:32.7126425Z * [new branch] gh/kwen2501/170/head -> origin/gh/kwen2501/170/head 2025-12-04T09:23:32.7126583Z * [new branch] gh/kwen2501/187/base -> origin/gh/kwen2501/187/base 2025-12-04T09:23:32.7126716Z * [new branch] gh/kwen2501/187/head -> origin/gh/kwen2501/187/head 2025-12-04T09:23:32.7126857Z * [new branch] gh/kwen2501/187/orig -> origin/gh/kwen2501/187/orig 2025-12-04T09:23:32.7126997Z * [new branch] gh/kwen2501/188/base -> origin/gh/kwen2501/188/base 2025-12-04T09:23:32.7127124Z * [new branch] gh/kwen2501/188/head -> origin/gh/kwen2501/188/head 2025-12-04T09:23:32.7127265Z * [new branch] gh/kwen2501/188/orig -> origin/gh/kwen2501/188/orig 2025-12-04T09:23:32.7127393Z * [new branch] gh/kwen2501/211/base -> origin/gh/kwen2501/211/base 2025-12-04T09:23:32.7127531Z * [new branch] gh/kwen2501/211/head -> origin/gh/kwen2501/211/head 2025-12-04T09:23:32.7127660Z * [new branch] gh/kwen2501/224/base -> origin/gh/kwen2501/224/base 2025-12-04T09:23:32.7131654Z * [new branch] gh/kwen2501/224/head -> origin/gh/kwen2501/224/head 2025-12-04T09:23:32.7131867Z * [new branch] gh/kwen2501/224/orig -> origin/gh/kwen2501/224/orig 2025-12-04T09:23:32.7132019Z * [new branch] gh/kwen2501/228/base -> origin/gh/kwen2501/228/base 2025-12-04T09:23:32.7132163Z * [new branch] gh/kwen2501/228/head -> origin/gh/kwen2501/228/head 2025-12-04T09:23:32.7132292Z * [new branch] gh/kwen2501/228/orig -> origin/gh/kwen2501/228/orig 2025-12-04T09:23:32.7132422Z * [new branch] gh/kwen2501/234/base -> origin/gh/kwen2501/234/base 2025-12-04T09:23:32.7137948Z * [new branch] gh/kwen2501/234/head -> origin/gh/kwen2501/234/head 2025-12-04T09:23:32.7142534Z * [new branch] gh/kwen2501/234/orig -> origin/gh/kwen2501/234/orig 2025-12-04T09:23:32.7144451Z * [new branch] gh/kwen2501/235/base -> origin/gh/kwen2501/235/base 2025-12-04T09:23:32.7144722Z * [new branch] gh/kwen2501/235/head -> origin/gh/kwen2501/235/head 2025-12-04T09:23:32.7148406Z * [new branch] gh/kwen2501/235/orig -> origin/gh/kwen2501/235/orig 2025-12-04T09:23:32.7148602Z * [new branch] gh/kwen2501/236/base -> origin/gh/kwen2501/236/base 2025-12-04T09:23:32.7148753Z * [new branch] gh/kwen2501/236/head -> origin/gh/kwen2501/236/head 2025-12-04T09:23:32.7148884Z * [new branch] gh/kwen2501/236/orig -> origin/gh/kwen2501/236/orig 2025-12-04T09:23:32.7149028Z * [new branch] gh/kwen2501/237/base -> origin/gh/kwen2501/237/base 2025-12-04T09:23:32.7149166Z * [new branch] gh/kwen2501/237/head -> origin/gh/kwen2501/237/head 2025-12-04T09:23:32.7149312Z * [new branch] gh/kwen2501/237/orig -> origin/gh/kwen2501/237/orig 2025-12-04T09:23:32.7149648Z * [new branch] gh/kwen2501/238/base -> origin/gh/kwen2501/238/base 2025-12-04T09:23:32.7149789Z * [new branch] gh/kwen2501/238/head -> origin/gh/kwen2501/238/head 2025-12-04T09:23:32.7149930Z * [new branch] gh/kwen2501/238/orig -> origin/gh/kwen2501/238/orig 2025-12-04T09:23:32.7150114Z * [new branch] gh/kwen2501/240/base -> origin/gh/kwen2501/240/base 2025-12-04T09:23:32.7150258Z * [new branch] gh/kwen2501/240/head -> origin/gh/kwen2501/240/head 2025-12-04T09:23:32.7150395Z * [new branch] gh/kwen2501/240/orig -> origin/gh/kwen2501/240/orig 2025-12-04T09:23:32.7150527Z * [new branch] gh/kwen2501/241/base -> origin/gh/kwen2501/241/base 2025-12-04T09:23:32.7150670Z * [new branch] gh/kwen2501/241/head -> origin/gh/kwen2501/241/head 2025-12-04T09:23:32.7150801Z * [new branch] gh/kwen2501/241/orig -> origin/gh/kwen2501/241/orig 2025-12-04T09:23:32.7150944Z * [new branch] gh/kwen2501/247/base -> origin/gh/kwen2501/247/base 2025-12-04T09:23:32.7151073Z * [new branch] gh/kwen2501/247/head -> origin/gh/kwen2501/247/head 2025-12-04T09:23:32.7151237Z * [new branch] gh/kwen2501/247/orig -> origin/gh/kwen2501/247/orig 2025-12-04T09:23:32.7151378Z * [new branch] gh/kwen2501/252/base -> origin/gh/kwen2501/252/base 2025-12-04T09:23:32.7151506Z * [new branch] gh/kwen2501/252/head -> origin/gh/kwen2501/252/head 2025-12-04T09:23:32.7151683Z * [new branch] gh/kwen2501/252/orig -> origin/gh/kwen2501/252/orig 2025-12-04T09:23:32.7151957Z * [new branch] gh/kwen2501/259/base -> origin/gh/kwen2501/259/base 2025-12-04T09:23:32.7152116Z * [new branch] gh/kwen2501/259/head -> origin/gh/kwen2501/259/head 2025-12-04T09:23:32.7152267Z * [new branch] gh/kwen2501/259/orig -> origin/gh/kwen2501/259/orig 2025-12-04T09:23:32.7157736Z * [new branch] gh/kwen2501/260/base -> origin/gh/kwen2501/260/base 2025-12-04T09:23:32.7162509Z * [new branch] gh/kwen2501/260/head -> origin/gh/kwen2501/260/head 2025-12-04T09:23:32.7164736Z * [new branch] gh/kwen2501/260/orig -> origin/gh/kwen2501/260/orig 2025-12-04T09:23:32.7165022Z * [new branch] gh/kwen2501/268/base -> origin/gh/kwen2501/268/base 2025-12-04T09:23:32.7172082Z * [new branch] gh/kwen2501/268/head -> origin/gh/kwen2501/268/head 2025-12-04T09:23:32.7176253Z * [new branch] gh/kwen2501/268/orig -> origin/gh/kwen2501/268/orig 2025-12-04T09:23:32.7180442Z * [new branch] gh/kwen2501/269/base -> origin/gh/kwen2501/269/base 2025-12-04T09:23:32.7184700Z * [new branch] gh/kwen2501/269/head -> origin/gh/kwen2501/269/head 2025-12-04T09:23:32.7184874Z * [new branch] gh/kwen2501/269/orig -> origin/gh/kwen2501/269/orig 2025-12-04T09:23:32.7185059Z * [new branch] gh/kwen2501/270/base -> origin/gh/kwen2501/270/base 2025-12-04T09:23:32.7185193Z * [new branch] gh/kwen2501/270/head -> origin/gh/kwen2501/270/head 2025-12-04T09:23:32.7185323Z * [new branch] gh/kwen2501/270/orig -> origin/gh/kwen2501/270/orig 2025-12-04T09:23:32.7185479Z * [new branch] gh/kwen2501/271/base -> origin/gh/kwen2501/271/base 2025-12-04T09:23:32.7185613Z * [new branch] gh/kwen2501/271/head -> origin/gh/kwen2501/271/head 2025-12-04T09:23:32.7185749Z * [new branch] gh/kwen2501/271/orig -> origin/gh/kwen2501/271/orig 2025-12-04T09:23:32.7185875Z * [new branch] gh/kwen2501/274/base -> origin/gh/kwen2501/274/base 2025-12-04T09:23:32.7186003Z * [new branch] gh/kwen2501/274/head -> origin/gh/kwen2501/274/head 2025-12-04T09:23:32.7186294Z * [new branch] gh/kwen2501/274/orig -> origin/gh/kwen2501/274/orig 2025-12-04T09:23:32.7186427Z * [new branch] gh/kwen2501/275/base -> origin/gh/kwen2501/275/base 2025-12-04T09:23:32.7186557Z * [new branch] gh/kwen2501/275/head -> origin/gh/kwen2501/275/head 2025-12-04T09:23:32.7186758Z * [new branch] gh/kwen2501/275/orig -> origin/gh/kwen2501/275/orig 2025-12-04T09:23:32.7186892Z * [new branch] gh/kwen2501/276/base -> origin/gh/kwen2501/276/base 2025-12-04T09:23:32.7187030Z * [new branch] gh/kwen2501/276/head -> origin/gh/kwen2501/276/head 2025-12-04T09:23:32.7187158Z * [new branch] gh/kwen2501/276/orig -> origin/gh/kwen2501/276/orig 2025-12-04T09:23:32.7187289Z * [new branch] gh/kwen2501/277/base -> origin/gh/kwen2501/277/base 2025-12-04T09:23:32.7187429Z * [new branch] gh/kwen2501/277/head -> origin/gh/kwen2501/277/head 2025-12-04T09:23:32.7187561Z * [new branch] gh/kwen2501/277/orig -> origin/gh/kwen2501/277/orig 2025-12-04T09:23:32.7187698Z * [new branch] gh/kwen2501/278/base -> origin/gh/kwen2501/278/base 2025-12-04T09:23:32.7187829Z * [new branch] gh/kwen2501/278/head -> origin/gh/kwen2501/278/head 2025-12-04T09:23:32.7187961Z * [new branch] gh/kwen2501/278/orig -> origin/gh/kwen2501/278/orig 2025-12-04T09:23:32.7188106Z * [new branch] gh/kwen2501/279/base -> origin/gh/kwen2501/279/base 2025-12-04T09:23:32.7188239Z * [new branch] gh/kwen2501/279/head -> origin/gh/kwen2501/279/head 2025-12-04T09:23:32.7188380Z * [new branch] gh/kwen2501/279/orig -> origin/gh/kwen2501/279/orig 2025-12-04T09:23:32.7188511Z * [new branch] gh/kwen2501/280/base -> origin/gh/kwen2501/280/base 2025-12-04T09:23:32.7188642Z * [new branch] gh/kwen2501/280/head -> origin/gh/kwen2501/280/head 2025-12-04T09:23:32.7188781Z * [new branch] gh/kwen2501/280/orig -> origin/gh/kwen2501/280/orig 2025-12-04T09:23:32.7188909Z * [new branch] gh/kwen2501/281/base -> origin/gh/kwen2501/281/base 2025-12-04T09:23:32.7224648Z * [new branch] gh/kwen2501/281/head -> origin/gh/kwen2501/281/head 2025-12-04T09:23:32.7224934Z * [new branch] gh/kwen2501/281/orig -> origin/gh/kwen2501/281/orig 2025-12-04T09:23:32.7225085Z * [new branch] gh/kwen2501/282/base -> origin/gh/kwen2501/282/base 2025-12-04T09:23:32.7225267Z * [new branch] gh/kwen2501/282/head -> origin/gh/kwen2501/282/head 2025-12-04T09:23:32.7225398Z * [new branch] gh/kwen2501/282/orig -> origin/gh/kwen2501/282/orig 2025-12-04T09:23:32.7225538Z * [new branch] gh/kwen2501/283/base -> origin/gh/kwen2501/283/base 2025-12-04T09:23:32.7225669Z * [new branch] gh/kwen2501/283/head -> origin/gh/kwen2501/283/head 2025-12-04T09:23:32.7225800Z * [new branch] gh/kwen2501/283/orig -> origin/gh/kwen2501/283/orig 2025-12-04T09:23:32.7225953Z * [new branch] gh/kwen2501/284/base -> origin/gh/kwen2501/284/base 2025-12-04T09:23:32.7226083Z * [new branch] gh/kwen2501/284/head -> origin/gh/kwen2501/284/head 2025-12-04T09:23:32.7226213Z * [new branch] gh/kwen2501/284/orig -> origin/gh/kwen2501/284/orig 2025-12-04T09:23:32.7226352Z * [new branch] gh/kwen2501/285/base -> origin/gh/kwen2501/285/base 2025-12-04T09:23:32.7226480Z * [new branch] gh/kwen2501/285/head -> origin/gh/kwen2501/285/head 2025-12-04T09:23:32.7226617Z * [new branch] gh/kwen2501/285/orig -> origin/gh/kwen2501/285/orig 2025-12-04T09:23:32.7226740Z * [new branch] gh/kwen2501/286/base -> origin/gh/kwen2501/286/base 2025-12-04T09:23:32.7227064Z * [new branch] gh/kwen2501/286/head -> origin/gh/kwen2501/286/head 2025-12-04T09:23:32.7227208Z * [new branch] gh/kwen2501/286/orig -> origin/gh/kwen2501/286/orig 2025-12-04T09:23:32.7227332Z * [new branch] gh/kwen2501/287/base -> origin/gh/kwen2501/287/base 2025-12-04T09:23:32.7227537Z * [new branch] gh/kwen2501/287/head -> origin/gh/kwen2501/287/head 2025-12-04T09:23:32.7227667Z * [new branch] gh/kwen2501/287/orig -> origin/gh/kwen2501/287/orig 2025-12-04T09:23:32.7227796Z * [new branch] gh/kwen2501/288/base -> origin/gh/kwen2501/288/base 2025-12-04T09:23:32.7227936Z * [new branch] gh/kwen2501/288/head -> origin/gh/kwen2501/288/head 2025-12-04T09:23:32.7228073Z * [new branch] gh/kwen2501/288/orig -> origin/gh/kwen2501/288/orig 2025-12-04T09:23:32.7228235Z * [new branch] gh/laithsakka/251/base -> origin/gh/laithsakka/251/base 2025-12-04T09:23:32.7228379Z * [new branch] gh/laithsakka/251/head -> origin/gh/laithsakka/251/head 2025-12-04T09:23:32.7228518Z * [new branch] gh/laithsakka/251/orig -> origin/gh/laithsakka/251/orig 2025-12-04T09:23:32.7228665Z * [new branch] gh/laithsakka/276/base -> origin/gh/laithsakka/276/base 2025-12-04T09:23:32.7228808Z * [new branch] gh/laithsakka/276/head -> origin/gh/laithsakka/276/head 2025-12-04T09:23:32.7228946Z * [new branch] gh/laithsakka/276/orig -> origin/gh/laithsakka/276/orig 2025-12-04T09:23:32.7229101Z * [new branch] gh/laithsakka/28/base -> origin/gh/laithsakka/28/base 2025-12-04T09:23:32.7229243Z * [new branch] gh/laithsakka/29/base -> origin/gh/laithsakka/29/base 2025-12-04T09:23:32.7229387Z * [new branch] gh/laithsakka/30/base -> origin/gh/laithsakka/30/base 2025-12-04T09:23:32.7229522Z * [new branch] gh/laithsakka/30/head -> origin/gh/laithsakka/30/head 2025-12-04T09:23:32.7229660Z * [new branch] gh/laithsakka/31/base -> origin/gh/laithsakka/31/base 2025-12-04T09:23:32.7229805Z * [new branch] gh/laithsakka/31/head -> origin/gh/laithsakka/31/head 2025-12-04T09:23:32.7229946Z * [new branch] gh/laithsakka/313/base -> origin/gh/laithsakka/313/base 2025-12-04T09:23:32.7230097Z * [new branch] gh/laithsakka/313/head -> origin/gh/laithsakka/313/head 2025-12-04T09:23:32.7230236Z * [new branch] gh/laithsakka/313/orig -> origin/gh/laithsakka/313/orig 2025-12-04T09:23:32.7230371Z * [new branch] gh/laithsakka/316/base -> origin/gh/laithsakka/316/base 2025-12-04T09:23:32.7230515Z * [new branch] gh/laithsakka/316/head -> origin/gh/laithsakka/316/head 2025-12-04T09:23:32.7230651Z * [new branch] gh/laithsakka/316/orig -> origin/gh/laithsakka/316/orig 2025-12-04T09:23:32.7230809Z * [new branch] gh/laithsakka/317/base -> origin/gh/laithsakka/317/base 2025-12-04T09:23:32.7230946Z * [new branch] gh/laithsakka/317/head -> origin/gh/laithsakka/317/head 2025-12-04T09:23:32.7231083Z * [new branch] gh/laithsakka/317/orig -> origin/gh/laithsakka/317/orig 2025-12-04T09:23:32.7231232Z * [new branch] gh/laithsakka/319/base -> origin/gh/laithsakka/319/base 2025-12-04T09:23:32.7231369Z * [new branch] gh/laithsakka/319/head -> origin/gh/laithsakka/319/head 2025-12-04T09:23:32.7231513Z * [new branch] gh/laithsakka/319/orig -> origin/gh/laithsakka/319/orig 2025-12-04T09:23:32.7231648Z * [new branch] gh/laithsakka/32/base -> origin/gh/laithsakka/32/base 2025-12-04T09:23:32.7231785Z * [new branch] gh/laithsakka/32/head -> origin/gh/laithsakka/32/head 2025-12-04T09:23:32.7231929Z * [new branch] gh/laithsakka/320/base -> origin/gh/laithsakka/320/base 2025-12-04T09:23:32.7232100Z * [new branch] gh/laithsakka/320/head -> origin/gh/laithsakka/320/head 2025-12-04T09:23:32.7232247Z * [new branch] gh/laithsakka/320/orig -> origin/gh/laithsakka/320/orig 2025-12-04T09:23:32.7232393Z * [new branch] gh/laithsakka/321/base -> origin/gh/laithsakka/321/base 2025-12-04T09:23:32.7232572Z * [new branch] gh/laithsakka/321/head -> origin/gh/laithsakka/321/head 2025-12-04T09:23:32.7232810Z * [new branch] gh/laithsakka/321/orig -> origin/gh/laithsakka/321/orig 2025-12-04T09:23:32.7232959Z * [new branch] gh/laithsakka/322/base -> origin/gh/laithsakka/322/base 2025-12-04T09:23:32.7233102Z * [new branch] gh/laithsakka/322/head -> origin/gh/laithsakka/322/head 2025-12-04T09:23:32.7233256Z * [new branch] gh/laithsakka/322/orig -> origin/gh/laithsakka/322/orig 2025-12-04T09:23:32.7233399Z * [new branch] gh/laithsakka/323/base -> origin/gh/laithsakka/323/base 2025-12-04T09:23:32.7233555Z * [new branch] gh/laithsakka/323/head -> origin/gh/laithsakka/323/head 2025-12-04T09:23:32.7233701Z * [new branch] gh/laithsakka/323/orig -> origin/gh/laithsakka/323/orig 2025-12-04T09:23:32.7233850Z * [new branch] gh/laithsakka/324/base -> origin/gh/laithsakka/324/base 2025-12-04T09:23:32.7234007Z * [new branch] gh/laithsakka/324/head -> origin/gh/laithsakka/324/head 2025-12-04T09:23:32.7234151Z * [new branch] gh/laithsakka/324/orig -> origin/gh/laithsakka/324/orig 2025-12-04T09:23:32.7234485Z * [new branch] gh/laithsakka/325/base -> origin/gh/laithsakka/325/base 2025-12-04T09:23:32.7235023Z * [new branch] gh/laithsakka/325/head -> origin/gh/laithsakka/325/head 2025-12-04T09:23:32.7236804Z * [new branch] gh/laithsakka/325/orig -> origin/gh/laithsakka/325/orig 2025-12-04T09:23:32.7237206Z * [new branch] gh/laithsakka/326/base -> origin/gh/laithsakka/326/base 2025-12-04T09:23:32.7240026Z * [new branch] gh/laithsakka/326/head -> origin/gh/laithsakka/326/head 2025-12-04T09:23:32.7240364Z * [new branch] gh/laithsakka/326/orig -> origin/gh/laithsakka/326/orig 2025-12-04T09:23:32.7240565Z * [new branch] gh/laithsakka/327/base -> origin/gh/laithsakka/327/base 2025-12-04T09:23:32.7240814Z * [new branch] gh/laithsakka/327/head -> origin/gh/laithsakka/327/head 2025-12-04T09:23:32.7240971Z * [new branch] gh/laithsakka/327/orig -> origin/gh/laithsakka/327/orig 2025-12-04T09:23:32.7242356Z * [new branch] gh/laithsakka/328/base -> origin/gh/laithsakka/328/base 2025-12-04T09:23:32.7242520Z * [new branch] gh/laithsakka/328/head -> origin/gh/laithsakka/328/head 2025-12-04T09:23:32.7245981Z * [new branch] gh/laithsakka/328/orig -> origin/gh/laithsakka/328/orig 2025-12-04T09:23:32.7246358Z * [new branch] gh/liangel/4/base -> origin/gh/liangel/4/base 2025-12-04T09:23:32.7246565Z * [new branch] gh/liangel/4/head -> origin/gh/liangel/4/head 2025-12-04T09:23:32.7246708Z * [new branch] gh/liangel/4/orig -> origin/gh/liangel/4/orig 2025-12-04T09:23:32.7251151Z * [new branch] gh/lucaskabela/1/base -> origin/gh/lucaskabela/1/base 2025-12-04T09:23:32.7251507Z * [new branch] gh/lucaskabela/1/head -> origin/gh/lucaskabela/1/head 2025-12-04T09:23:32.7251669Z * [new branch] gh/lw/4/base -> origin/gh/lw/4/base 2025-12-04T09:23:32.7251813Z * [new branch] gh/lw/4/head -> origin/gh/lw/4/head 2025-12-04T09:23:32.7251944Z * [new branch] gh/lw/4/orig -> origin/gh/lw/4/orig 2025-12-04T09:23:32.7253695Z * [new branch] gh/lw/5/base -> origin/gh/lw/5/base 2025-12-04T09:23:32.7254075Z * [new branch] gh/lw/5/head -> origin/gh/lw/5/head 2025-12-04T09:23:32.7256616Z * [new branch] gh/lw/5/orig -> origin/gh/lw/5/orig 2025-12-04T09:23:32.7256938Z * [new branch] gh/lw/6/base -> origin/gh/lw/6/base 2025-12-04T09:23:32.7257334Z * [new branch] gh/lw/6/head -> origin/gh/lw/6/head 2025-12-04T09:23:32.7257603Z * [new branch] gh/lw/6/orig -> origin/gh/lw/6/orig 2025-12-04T09:23:32.7258326Z * [new branch] gh/malfet/14/base -> origin/gh/malfet/14/base 2025-12-04T09:23:32.7259402Z * [new branch] gh/malfet/417/base -> origin/gh/malfet/417/base 2025-12-04T09:23:32.7262179Z * [new branch] gh/malfet/417/head -> origin/gh/malfet/417/head 2025-12-04T09:23:32.7266191Z * [new branch] gh/malfet/417/orig -> origin/gh/malfet/417/orig 2025-12-04T09:23:32.7267479Z * [new branch] gh/malfet/506/base -> origin/gh/malfet/506/base 2025-12-04T09:23:32.7267631Z * [new branch] gh/malfet/506/head -> origin/gh/malfet/506/head 2025-12-04T09:23:32.7267776Z * [new branch] gh/malfet/506/orig -> origin/gh/malfet/506/orig 2025-12-04T09:23:32.7267928Z * [new branch] gh/malfet/517/base -> origin/gh/malfet/517/base 2025-12-04T09:23:32.7268056Z * [new branch] gh/malfet/517/head -> origin/gh/malfet/517/head 2025-12-04T09:23:32.7268184Z * [new branch] gh/malfet/528/base -> origin/gh/malfet/528/base 2025-12-04T09:23:32.7268318Z * [new branch] gh/malfet/528/head -> origin/gh/malfet/528/head 2025-12-04T09:23:32.7268447Z * [new branch] gh/malfet/528/orig -> origin/gh/malfet/528/orig 2025-12-04T09:23:32.7268573Z * [new branch] gh/malfet/537/base -> origin/gh/malfet/537/base 2025-12-04T09:23:32.7268855Z * [new branch] gh/malfet/537/head -> origin/gh/malfet/537/head 2025-12-04T09:23:32.7269114Z * [new branch] gh/malfet/537/orig -> origin/gh/malfet/537/orig 2025-12-04T09:23:32.7270738Z * [new branch] gh/malfet/546/base -> origin/gh/malfet/546/base 2025-12-04T09:23:32.7270946Z * [new branch] gh/malfet/546/head -> origin/gh/malfet/546/head 2025-12-04T09:23:32.7271632Z * [new branch] gh/malfet/546/orig -> origin/gh/malfet/546/orig 2025-12-04T09:23:32.7273920Z * [new branch] gh/malfet/565/base -> origin/gh/malfet/565/base 2025-12-04T09:23:32.7274125Z * [new branch] gh/malfet/565/head -> origin/gh/malfet/565/head 2025-12-04T09:23:32.7274268Z * [new branch] gh/malfet/565/orig -> origin/gh/malfet/565/orig 2025-12-04T09:23:32.7278142Z * [new branch] gh/malfet/575/base -> origin/gh/malfet/575/base 2025-12-04T09:23:32.7278345Z * [new branch] gh/malfet/575/head -> origin/gh/malfet/575/head 2025-12-04T09:23:32.7278504Z * [new branch] gh/malfet/575/orig -> origin/gh/malfet/575/orig 2025-12-04T09:23:32.7278664Z * [new branch] gh/malfet/580/base -> origin/gh/malfet/580/base 2025-12-04T09:23:32.7282951Z * [new branch] gh/malfet/580/head -> origin/gh/malfet/580/head 2025-12-04T09:23:32.7283134Z * [new branch] gh/malfet/580/orig -> origin/gh/malfet/580/orig 2025-12-04T09:23:32.7283304Z * [new branch] gh/malfet/581/base -> origin/gh/malfet/581/base 2025-12-04T09:23:32.7283457Z * [new branch] gh/malfet/581/head -> origin/gh/malfet/581/head 2025-12-04T09:23:32.7283614Z * [new branch] gh/malfet/581/orig -> origin/gh/malfet/581/orig 2025-12-04T09:23:32.7283764Z * [new branch] gh/malfet/583/base -> origin/gh/malfet/583/base 2025-12-04T09:23:32.7284110Z * [new branch] gh/malfet/583/head -> origin/gh/malfet/583/head 2025-12-04T09:23:32.7284275Z * [new branch] gh/malfet/583/orig -> origin/gh/malfet/583/orig 2025-12-04T09:23:32.7284872Z * [new branch] gh/malfet/586/base -> origin/gh/malfet/586/base 2025-12-04T09:23:32.7285825Z * [new branch] gh/malfet/586/head -> origin/gh/malfet/586/head 2025-12-04T09:23:32.7286121Z * [new branch] gh/malfet/586/orig -> origin/gh/malfet/586/orig 2025-12-04T09:23:32.7290505Z * [new branch] gh/malfet/587/base -> origin/gh/malfet/587/base 2025-12-04T09:23:32.7290691Z * [new branch] gh/malfet/587/head -> origin/gh/malfet/587/head 2025-12-04T09:23:32.7290833Z * [new branch] gh/malfet/587/orig -> origin/gh/malfet/587/orig 2025-12-04T09:23:32.7290967Z * [new branch] gh/malfet/588/base -> origin/gh/malfet/588/base 2025-12-04T09:23:32.7291124Z * [new branch] gh/malfet/588/head -> origin/gh/malfet/588/head 2025-12-04T09:23:32.7291257Z * [new branch] gh/malfet/588/orig -> origin/gh/malfet/588/orig 2025-12-04T09:23:32.7291708Z * [new branch] gh/malfet/589/base -> origin/gh/malfet/589/base 2025-12-04T09:23:32.7295740Z * [new branch] gh/malfet/589/head -> origin/gh/malfet/589/head 2025-12-04T09:23:32.7296276Z * [new branch] gh/malfet/589/orig -> origin/gh/malfet/589/orig 2025-12-04T09:23:32.7296435Z * [new branch] gh/malfet/590/base -> origin/gh/malfet/590/base 2025-12-04T09:23:32.7296575Z * [new branch] gh/malfet/590/head -> origin/gh/malfet/590/head 2025-12-04T09:23:32.7296702Z * [new branch] gh/malfet/590/orig -> origin/gh/malfet/590/orig 2025-12-04T09:23:32.7296827Z * [new branch] gh/malfet/591/base -> origin/gh/malfet/591/base 2025-12-04T09:23:32.7296979Z * [new branch] gh/malfet/591/head -> origin/gh/malfet/591/head 2025-12-04T09:23:32.7297406Z * [new branch] gh/malfet/591/orig -> origin/gh/malfet/591/orig 2025-12-04T09:23:32.7298658Z * [new branch] gh/malfet/592/base -> origin/gh/malfet/592/base 2025-12-04T09:23:32.7299054Z * [new branch] gh/malfet/592/head -> origin/gh/malfet/592/head 2025-12-04T09:23:32.7299958Z * [new branch] gh/malfet/592/orig -> origin/gh/malfet/592/orig 2025-12-04T09:23:32.7300830Z * [new branch] gh/malfet/593/base -> origin/gh/malfet/593/base 2025-12-04T09:23:32.7301905Z * [new branch] gh/malfet/593/head -> origin/gh/malfet/593/head 2025-12-04T09:23:32.7302973Z * [new branch] gh/malfet/593/orig -> origin/gh/malfet/593/orig 2025-12-04T09:23:32.7304182Z * [new branch] gh/malfet/594/base -> origin/gh/malfet/594/base 2025-12-04T09:23:32.7304510Z * [new branch] gh/malfet/594/head -> origin/gh/malfet/594/head 2025-12-04T09:23:32.7305563Z * [new branch] gh/malfet/594/orig -> origin/gh/malfet/594/orig 2025-12-04T09:23:32.7306460Z * [new branch] gh/malfet/595/base -> origin/gh/malfet/595/base 2025-12-04T09:23:32.7309832Z * [new branch] gh/malfet/595/head -> origin/gh/malfet/595/head 2025-12-04T09:23:32.7310166Z * [new branch] gh/malfet/595/orig -> origin/gh/malfet/595/orig 2025-12-04T09:23:32.7311563Z * [new branch] gh/malfet/596/base -> origin/gh/malfet/596/base 2025-12-04T09:23:32.7311758Z * [new branch] gh/malfet/596/head -> origin/gh/malfet/596/head 2025-12-04T09:23:32.7312871Z * [new branch] gh/malfet/596/orig -> origin/gh/malfet/596/orig 2025-12-04T09:23:32.7317037Z * [new branch] gh/malfet/597/base -> origin/gh/malfet/597/base 2025-12-04T09:23:32.7317168Z * [new branch] gh/malfet/597/head -> origin/gh/malfet/597/head 2025-12-04T09:23:32.7317295Z * [new branch] gh/malfet/597/orig -> origin/gh/malfet/597/orig 2025-12-04T09:23:32.7317432Z * [new branch] gh/malfet/598/base -> origin/gh/malfet/598/base 2025-12-04T09:23:32.7317779Z * [new branch] gh/malfet/598/head -> origin/gh/malfet/598/head 2025-12-04T09:23:32.7317952Z * [new branch] gh/malfet/598/orig -> origin/gh/malfet/598/orig 2025-12-04T09:23:32.7322966Z * [new branch] gh/malfet/599/base -> origin/gh/malfet/599/base 2025-12-04T09:23:32.7323155Z * [new branch] gh/malfet/599/head -> origin/gh/malfet/599/head 2025-12-04T09:23:32.7323291Z * [new branch] gh/malfet/599/orig -> origin/gh/malfet/599/orig 2025-12-04T09:23:32.7323441Z * [new branch] gh/malfet/600/base -> origin/gh/malfet/600/base 2025-12-04T09:23:32.7323583Z * [new branch] gh/malfet/600/head -> origin/gh/malfet/600/head 2025-12-04T09:23:32.7325314Z * [new branch] gh/malfet/600/orig -> origin/gh/malfet/600/orig 2025-12-04T09:23:32.7325535Z * [new branch] gh/malfet/601/base -> origin/gh/malfet/601/base 2025-12-04T09:23:32.7325660Z * [new branch] gh/malfet/601/head -> origin/gh/malfet/601/head 2025-12-04T09:23:32.7325794Z * [new branch] gh/malfet/601/orig -> origin/gh/malfet/601/orig 2025-12-04T09:23:32.7325923Z * [new branch] gh/malfet/602/base -> origin/gh/malfet/602/base 2025-12-04T09:23:32.7326055Z * [new branch] gh/malfet/602/head -> origin/gh/malfet/602/head 2025-12-04T09:23:32.7329237Z * [new branch] gh/malfet/602/orig -> origin/gh/malfet/602/orig 2025-12-04T09:23:32.7329891Z * [new branch] gh/malfet/603/base -> origin/gh/malfet/603/base 2025-12-04T09:23:32.7333095Z * [new branch] gh/malfet/603/head -> origin/gh/malfet/603/head 2025-12-04T09:23:32.7333261Z * [new branch] gh/malfet/603/orig -> origin/gh/malfet/603/orig 2025-12-04T09:23:32.7333410Z * [new branch] gh/malfet/604/base -> origin/gh/malfet/604/base 2025-12-04T09:23:32.7333567Z * [new branch] gh/malfet/604/head -> origin/gh/malfet/604/head 2025-12-04T09:23:32.7333712Z * [new branch] gh/malfet/604/orig -> origin/gh/malfet/604/orig 2025-12-04T09:23:32.7333854Z * [new branch] gh/malfet/605/base -> origin/gh/malfet/605/base 2025-12-04T09:23:32.7334006Z * [new branch] gh/malfet/605/head -> origin/gh/malfet/605/head 2025-12-04T09:23:32.7334151Z * [new branch] gh/malfet/605/orig -> origin/gh/malfet/605/orig 2025-12-04T09:23:32.7334323Z * [new branch] gh/malfet/606/base -> origin/gh/malfet/606/base 2025-12-04T09:23:32.7341664Z * [new branch] gh/malfet/606/head -> origin/gh/malfet/606/head 2025-12-04T09:23:32.7346490Z * [new branch] gh/malfet/606/orig -> origin/gh/malfet/606/orig 2025-12-04T09:23:32.7348413Z * [new branch] gh/malfet/607/base -> origin/gh/malfet/607/base 2025-12-04T09:23:32.7348559Z * [new branch] gh/malfet/607/head -> origin/gh/malfet/607/head 2025-12-04T09:23:32.7348693Z * [new branch] gh/malfet/607/orig -> origin/gh/malfet/607/orig 2025-12-04T09:23:32.7348969Z * [new branch] gh/malfet/608/base -> origin/gh/malfet/608/base 2025-12-04T09:23:32.7349102Z * [new branch] gh/malfet/608/head -> origin/gh/malfet/608/head 2025-12-04T09:23:32.7349249Z * [new branch] gh/malfet/608/orig -> origin/gh/malfet/608/orig 2025-12-04T09:23:32.7349589Z * [new branch] gh/malfet/609/base -> origin/gh/malfet/609/base 2025-12-04T09:23:32.7349722Z * [new branch] gh/malfet/609/head -> origin/gh/malfet/609/head 2025-12-04T09:23:32.7349859Z * [new branch] gh/malfet/609/orig -> origin/gh/malfet/609/orig 2025-12-04T09:23:32.7350059Z * [new branch] gh/malfet/610/base -> origin/gh/malfet/610/base 2025-12-04T09:23:32.7350190Z * [new branch] gh/malfet/610/head -> origin/gh/malfet/610/head 2025-12-04T09:23:32.7350328Z * [new branch] gh/malfet/610/orig -> origin/gh/malfet/610/orig 2025-12-04T09:23:32.7350456Z * [new branch] gh/malfet/611/base -> origin/gh/malfet/611/base 2025-12-04T09:23:32.7350596Z * [new branch] gh/malfet/611/head -> origin/gh/malfet/611/head 2025-12-04T09:23:32.7350727Z * [new branch] gh/malfet/611/orig -> origin/gh/malfet/611/orig 2025-12-04T09:23:32.7350859Z * [new branch] gh/malfet/612/base -> origin/gh/malfet/612/base 2025-12-04T09:23:32.7350999Z * [new branch] gh/malfet/612/head -> origin/gh/malfet/612/head 2025-12-04T09:23:32.7351127Z * [new branch] gh/malfet/612/orig -> origin/gh/malfet/612/orig 2025-12-04T09:23:32.7351264Z * [new branch] gh/malfet/64/base -> origin/gh/malfet/64/base 2025-12-04T09:23:32.7351400Z * [new branch] gh/malfet/64/head -> origin/gh/malfet/64/head 2025-12-04T09:23:32.7353125Z * [new branch] gh/manuelcandales/11/base -> origin/gh/manuelcandales/11/base 2025-12-04T09:23:32.7353298Z * [new branch] gh/manuelcandales/11/head -> origin/gh/manuelcandales/11/head 2025-12-04T09:23:32.7353564Z * [new branch] gh/manuelcandales/11/orig -> origin/gh/manuelcandales/11/orig 2025-12-04T09:23:32.7359751Z * [new branch] gh/markkm/1/base -> origin/gh/markkm/1/base 2025-12-04T09:23:32.7362009Z * [new branch] gh/masnesral/1/base -> origin/gh/masnesral/1/base 2025-12-04T09:23:32.7362280Z * [new branch] gh/masnesral/1/head -> origin/gh/masnesral/1/head 2025-12-04T09:23:32.7368347Z * [new branch] gh/masnesral/1/orig -> origin/gh/masnesral/1/orig 2025-12-04T09:23:32.7370301Z * [new branch] gh/mhorowitz/0/base -> origin/gh/mhorowitz/0/base 2025-12-04T09:23:32.7370605Z * [new branch] gh/mhorowitz/0/head -> origin/gh/mhorowitz/0/head 2025-12-04T09:23:32.7375190Z * [new branch] gh/mhorowitz/1/base -> origin/gh/mhorowitz/1/base 2025-12-04T09:23:32.7375452Z * [new branch] gh/mhorowitz/1/head -> origin/gh/mhorowitz/1/head 2025-12-04T09:23:32.7380540Z * [new branch] gh/mhorowitz/2/base -> origin/gh/mhorowitz/2/base 2025-12-04T09:23:32.7385708Z * [new branch] gh/mhorowitz/2/head -> origin/gh/mhorowitz/2/head 2025-12-04T09:23:32.7387286Z * [new branch] gh/mhorowitz/3/base -> origin/gh/mhorowitz/3/base 2025-12-04T09:23:32.7387452Z * [new branch] gh/mhorowitz/3/head -> origin/gh/mhorowitz/3/head 2025-12-04T09:23:32.7387593Z * [new branch] gh/mhorowitz/4/base -> origin/gh/mhorowitz/4/base 2025-12-04T09:23:32.7387736Z * [new branch] gh/mhorowitz/4/head -> origin/gh/mhorowitz/4/head 2025-12-04T09:23:32.7387881Z * [new branch] gh/mhorowitz/5/base -> origin/gh/mhorowitz/5/base 2025-12-04T09:23:32.7388012Z * [new branch] gh/mhorowitz/5/head -> origin/gh/mhorowitz/5/head 2025-12-04T09:23:32.7388152Z * [new branch] gh/mhorowitz/6/base -> origin/gh/mhorowitz/6/base 2025-12-04T09:23:32.7388284Z * [new branch] gh/mhorowitz/6/head -> origin/gh/mhorowitz/6/head 2025-12-04T09:23:32.7388622Z * [new branch] gh/mikaylagawarecki/234/base -> origin/gh/mikaylagawarecki/234/base 2025-12-04T09:23:32.7388799Z * [new branch] gh/mikaylagawarecki/234/head -> origin/gh/mikaylagawarecki/234/head 2025-12-04T09:23:32.7388984Z * [new branch] gh/mikaylagawarecki/235/base -> origin/gh/mikaylagawarecki/235/base 2025-12-04T09:23:32.7389210Z * [new branch] gh/mikaylagawarecki/235/head -> origin/gh/mikaylagawarecki/235/head 2025-12-04T09:23:32.7389373Z * [new branch] gh/mikaylagawarecki/236/base -> origin/gh/mikaylagawarecki/236/base 2025-12-04T09:23:32.7389532Z * [new branch] gh/mikaylagawarecki/236/head -> origin/gh/mikaylagawarecki/236/head 2025-12-04T09:23:32.7389698Z * [new branch] gh/mikaylagawarecki/237/base -> origin/gh/mikaylagawarecki/237/base 2025-12-04T09:23:32.7389856Z * [new branch] gh/mikaylagawarecki/237/head -> origin/gh/mikaylagawarecki/237/head 2025-12-04T09:23:32.7390026Z * [new branch] gh/mikaylagawarecki/238/base -> origin/gh/mikaylagawarecki/238/base 2025-12-04T09:23:32.7390194Z * [new branch] gh/mikaylagawarecki/238/head -> origin/gh/mikaylagawarecki/238/head 2025-12-04T09:23:32.7390353Z * [new branch] gh/mikaylagawarecki/336/base -> origin/gh/mikaylagawarecki/336/base 2025-12-04T09:23:32.7390520Z * [new branch] gh/mikaylagawarecki/336/head -> origin/gh/mikaylagawarecki/336/head 2025-12-04T09:23:32.7390681Z * [new branch] gh/mikaylagawarecki/336/orig -> origin/gh/mikaylagawarecki/336/orig 2025-12-04T09:23:32.7390836Z * [new branch] gh/mikaylagawarecki/341/base -> origin/gh/mikaylagawarecki/341/base 2025-12-04T09:23:32.7391003Z * [new branch] gh/mikaylagawarecki/341/head -> origin/gh/mikaylagawarecki/341/head 2025-12-04T09:23:32.7391163Z * [new branch] gh/mikaylagawarecki/341/orig -> origin/gh/mikaylagawarecki/341/orig 2025-12-04T09:23:32.7391328Z * [new branch] gh/mikaylagawarecki/342/base -> origin/gh/mikaylagawarecki/342/base 2025-12-04T09:23:32.7391493Z * [new branch] gh/mikaylagawarecki/342/head -> origin/gh/mikaylagawarecki/342/head 2025-12-04T09:23:32.7391651Z * [new branch] gh/mikaylagawarecki/342/orig -> origin/gh/mikaylagawarecki/342/orig 2025-12-04T09:23:32.7391820Z * [new branch] gh/mikaylagawarecki/345/base -> origin/gh/mikaylagawarecki/345/base 2025-12-04T09:23:32.7391981Z * [new branch] gh/mikaylagawarecki/345/head -> origin/gh/mikaylagawarecki/345/head 2025-12-04T09:23:32.7392145Z * [new branch] gh/mikaylagawarecki/345/orig -> origin/gh/mikaylagawarecki/345/orig 2025-12-04T09:23:32.7392304Z * [new branch] gh/mikaylagawarecki/346/base -> origin/gh/mikaylagawarecki/346/base 2025-12-04T09:23:32.7392460Z * [new branch] gh/mikaylagawarecki/346/head -> origin/gh/mikaylagawarecki/346/head 2025-12-04T09:23:32.7392772Z * [new branch] gh/mikaylagawarecki/346/orig -> origin/gh/mikaylagawarecki/346/orig 2025-12-04T09:23:32.7392951Z * [new branch] gh/mikaylagawarecki/347/base -> origin/gh/mikaylagawarecki/347/base 2025-12-04T09:23:32.7393126Z * [new branch] gh/mikaylagawarecki/347/head -> origin/gh/mikaylagawarecki/347/head 2025-12-04T09:23:32.7393307Z * [new branch] gh/mikaylagawarecki/347/orig -> origin/gh/mikaylagawarecki/347/orig 2025-12-04T09:23:32.7393490Z * [new branch] gh/mikaylagawarecki/350/base -> origin/gh/mikaylagawarecki/350/base 2025-12-04T09:23:32.7393681Z * [new branch] gh/mikaylagawarecki/350/head -> origin/gh/mikaylagawarecki/350/head 2025-12-04T09:23:32.7393863Z * [new branch] gh/mikaylagawarecki/350/orig -> origin/gh/mikaylagawarecki/350/orig 2025-12-04T09:23:32.7398886Z * [new branch] gh/mikaylagawarecki/351/base -> origin/gh/mikaylagawarecki/351/base 2025-12-04T09:23:32.7402419Z * [new branch] gh/mikaylagawarecki/351/head -> origin/gh/mikaylagawarecki/351/head 2025-12-04T09:23:32.7406607Z * [new branch] gh/mikaylagawarecki/351/orig -> origin/gh/mikaylagawarecki/351/orig 2025-12-04T09:23:32.7412119Z * [new branch] gh/mikaylagawarecki/352/base -> origin/gh/mikaylagawarecki/352/base 2025-12-04T09:23:32.7416876Z * [new branch] gh/mikaylagawarecki/352/head -> origin/gh/mikaylagawarecki/352/head 2025-12-04T09:23:32.7421904Z * [new branch] gh/mikaylagawarecki/352/orig -> origin/gh/mikaylagawarecki/352/orig 2025-12-04T09:23:32.7426092Z * [new branch] gh/mikaylagawarecki/353/base -> origin/gh/mikaylagawarecki/353/base 2025-12-04T09:23:32.7428609Z * [new branch] gh/mikaylagawarecki/353/head -> origin/gh/mikaylagawarecki/353/head 2025-12-04T09:23:32.7428846Z * [new branch] gh/mikaylagawarecki/353/orig -> origin/gh/mikaylagawarecki/353/orig 2025-12-04T09:23:32.7429011Z * [new branch] gh/mikaylagawarecki/354/base -> origin/gh/mikaylagawarecki/354/base 2025-12-04T09:23:32.7429192Z * [new branch] gh/mikaylagawarecki/354/head -> origin/gh/mikaylagawarecki/354/head 2025-12-04T09:23:32.7429365Z * [new branch] gh/mikaylagawarecki/354/orig -> origin/gh/mikaylagawarecki/354/orig 2025-12-04T09:23:32.7429525Z * [new branch] gh/mikaylagawarecki/356/base -> origin/gh/mikaylagawarecki/356/base 2025-12-04T09:23:32.7429690Z * [new branch] gh/mikaylagawarecki/356/head -> origin/gh/mikaylagawarecki/356/head 2025-12-04T09:23:32.7429860Z * [new branch] gh/mikaylagawarecki/356/orig -> origin/gh/mikaylagawarecki/356/orig 2025-12-04T09:23:32.7430022Z * [new branch] gh/mikaylagawarecki/357/base -> origin/gh/mikaylagawarecki/357/base 2025-12-04T09:23:32.7430196Z * [new branch] gh/mikaylagawarecki/357/head -> origin/gh/mikaylagawarecki/357/head 2025-12-04T09:23:32.7430362Z * [new branch] gh/mikaylagawarecki/357/orig -> origin/gh/mikaylagawarecki/357/orig 2025-12-04T09:23:32.7430526Z * [new branch] gh/mikaylagawarecki/359/base -> origin/gh/mikaylagawarecki/359/base 2025-12-04T09:23:32.7430694Z * [new branch] gh/mikaylagawarecki/359/head -> origin/gh/mikaylagawarecki/359/head 2025-12-04T09:23:32.7430854Z * [new branch] gh/mikaylagawarecki/359/orig -> origin/gh/mikaylagawarecki/359/orig 2025-12-04T09:23:32.7431025Z * [new branch] gh/mikaylagawarecki/360/base -> origin/gh/mikaylagawarecki/360/base 2025-12-04T09:23:32.7431183Z * [new branch] gh/mikaylagawarecki/360/head -> origin/gh/mikaylagawarecki/360/head 2025-12-04T09:23:32.7431341Z * [new branch] gh/mikaylagawarecki/360/orig -> origin/gh/mikaylagawarecki/360/orig 2025-12-04T09:23:32.7431507Z * [new branch] gh/mikaylagawarecki/361/base -> origin/gh/mikaylagawarecki/361/base 2025-12-04T09:23:32.7431666Z * [new branch] gh/mikaylagawarecki/361/head -> origin/gh/mikaylagawarecki/361/head 2025-12-04T09:23:32.7431875Z * [new branch] gh/mikaylagawarecki/361/orig -> origin/gh/mikaylagawarecki/361/orig 2025-12-04T09:23:32.7432052Z * [new branch] gh/mikaylagawarecki/362/base -> origin/gh/mikaylagawarecki/362/base 2025-12-04T09:23:32.7432232Z * [new branch] gh/mikaylagawarecki/362/head -> origin/gh/mikaylagawarecki/362/head 2025-12-04T09:23:32.7432431Z * [new branch] gh/mikaylagawarecki/362/orig -> origin/gh/mikaylagawarecki/362/orig 2025-12-04T09:23:32.7432607Z * [new branch] gh/mikaylagawarecki/363/base -> origin/gh/mikaylagawarecki/363/base 2025-12-04T09:23:32.7432961Z * [new branch] gh/mikaylagawarecki/363/head -> origin/gh/mikaylagawarecki/363/head 2025-12-04T09:23:32.7433150Z * [new branch] gh/mikaylagawarecki/363/orig -> origin/gh/mikaylagawarecki/363/orig 2025-12-04T09:23:32.7433334Z * [new branch] gh/mikaylagawarecki/364/base -> origin/gh/mikaylagawarecki/364/base 2025-12-04T09:23:32.7433769Z * [new branch] gh/mikaylagawarecki/364/head -> origin/gh/mikaylagawarecki/364/head 2025-12-04T09:23:32.7433954Z * [new branch] gh/mikaylagawarecki/364/orig -> origin/gh/mikaylagawarecki/364/orig 2025-12-04T09:23:32.7434135Z * [new branch] gh/mikaylagawarecki/365/base -> origin/gh/mikaylagawarecki/365/base 2025-12-04T09:23:32.7434411Z * [new branch] gh/mikaylagawarecki/365/head -> origin/gh/mikaylagawarecki/365/head 2025-12-04T09:23:32.7434574Z * [new branch] gh/mikaylagawarecki/365/orig -> origin/gh/mikaylagawarecki/365/orig 2025-12-04T09:23:32.7434742Z * [new branch] gh/mikaylagawarecki/366/base -> origin/gh/mikaylagawarecki/366/base 2025-12-04T09:23:32.7434905Z * [new branch] gh/mikaylagawarecki/366/head -> origin/gh/mikaylagawarecki/366/head 2025-12-04T09:23:32.7435066Z * [new branch] gh/mikaylagawarecki/366/orig -> origin/gh/mikaylagawarecki/366/orig 2025-12-04T09:23:32.7435255Z * [new branch] gh/mikaylagawarecki/367/base -> origin/gh/mikaylagawarecki/367/base 2025-12-04T09:23:32.7435423Z * [new branch] gh/mikaylagawarecki/367/head -> origin/gh/mikaylagawarecki/367/head 2025-12-04T09:23:32.7435594Z * [new branch] gh/mikaylagawarecki/367/orig -> origin/gh/mikaylagawarecki/367/orig 2025-12-04T09:23:32.7435761Z * [new branch] gh/mikaylagawarecki/368/base -> origin/gh/mikaylagawarecki/368/base 2025-12-04T09:23:32.7435925Z * [new branch] gh/mikaylagawarecki/368/head -> origin/gh/mikaylagawarecki/368/head 2025-12-04T09:23:32.7436098Z * [new branch] gh/mikaylagawarecki/368/orig -> origin/gh/mikaylagawarecki/368/orig 2025-12-04T09:23:32.7436260Z * [new branch] gh/mikaylagawarecki/369/base -> origin/gh/mikaylagawarecki/369/base 2025-12-04T09:23:32.7436430Z * [new branch] gh/mikaylagawarecki/369/head -> origin/gh/mikaylagawarecki/369/head 2025-12-04T09:23:32.7436774Z * [new branch] gh/mikaylagawarecki/369/orig -> origin/gh/mikaylagawarecki/369/orig 2025-12-04T09:23:32.7436947Z * [new branch] gh/mikaylagawarecki/370/base -> origin/gh/mikaylagawarecki/370/base 2025-12-04T09:23:32.7437126Z * [new branch] gh/mikaylagawarecki/370/head -> origin/gh/mikaylagawarecki/370/head 2025-12-04T09:23:32.7437961Z * [new branch] gh/mikaylagawarecki/370/orig -> origin/gh/mikaylagawarecki/370/orig 2025-12-04T09:23:32.7441814Z * [new branch] gh/mikaylagawarecki/371/base -> origin/gh/mikaylagawarecki/371/base 2025-12-04T09:23:32.7442064Z * [new branch] gh/mikaylagawarecki/371/head -> origin/gh/mikaylagawarecki/371/head 2025-12-04T09:23:32.7442255Z * [new branch] gh/mikaylagawarecki/371/orig -> origin/gh/mikaylagawarecki/371/orig 2025-12-04T09:23:32.7442444Z * [new branch] gh/mikaylagawarecki/372/base -> origin/gh/mikaylagawarecki/372/base 2025-12-04T09:23:32.7442645Z * [new branch] gh/mikaylagawarecki/372/head -> origin/gh/mikaylagawarecki/372/head 2025-12-04T09:23:32.7442999Z * [new branch] gh/mikaylagawarecki/372/orig -> origin/gh/mikaylagawarecki/372/orig 2025-12-04T09:23:32.7445949Z * [new branch] gh/mikaylagawarecki/373/base -> origin/gh/mikaylagawarecki/373/base 2025-12-04T09:23:32.7446298Z * [new branch] gh/mikaylagawarecki/373/head -> origin/gh/mikaylagawarecki/373/head 2025-12-04T09:23:32.7446572Z * [new branch] gh/mikaylagawarecki/373/orig -> origin/gh/mikaylagawarecki/373/orig 2025-12-04T09:23:32.7446748Z * [new branch] gh/mikaylagawarecki/374/base -> origin/gh/mikaylagawarecki/374/base 2025-12-04T09:23:32.7446928Z * [new branch] gh/mikaylagawarecki/374/head -> origin/gh/mikaylagawarecki/374/head 2025-12-04T09:23:32.7447175Z * [new branch] gh/mikaylagawarecki/374/orig -> origin/gh/mikaylagawarecki/374/orig 2025-12-04T09:23:32.7451544Z * [new branch] gh/mikaylagawarecki/375/base -> origin/gh/mikaylagawarecki/375/base 2025-12-04T09:23:32.7452088Z * [new branch] gh/mikaylagawarecki/375/head -> origin/gh/mikaylagawarecki/375/head 2025-12-04T09:23:32.7452308Z * [new branch] gh/mikaylagawarecki/375/orig -> origin/gh/mikaylagawarecki/375/orig 2025-12-04T09:23:32.7452499Z * [new branch] gh/mikaylagawarecki/376/base -> origin/gh/mikaylagawarecki/376/base 2025-12-04T09:23:32.7452739Z * [new branch] gh/mikaylagawarecki/376/head -> origin/gh/mikaylagawarecki/376/head 2025-12-04T09:23:32.7453066Z * [new branch] gh/mikaylagawarecki/376/orig -> origin/gh/mikaylagawarecki/376/orig 2025-12-04T09:23:32.7453621Z * [new branch] gh/mikaylagawarecki/377/base -> origin/gh/mikaylagawarecki/377/base 2025-12-04T09:23:32.7453850Z * [new branch] gh/mikaylagawarecki/377/head -> origin/gh/mikaylagawarecki/377/head 2025-12-04T09:23:32.7454517Z * [new branch] gh/mikaylagawarecki/377/orig -> origin/gh/mikaylagawarecki/377/orig 2025-12-04T09:23:32.7457663Z * [new branch] gh/mikaylagawarecki/378/base -> origin/gh/mikaylagawarecki/378/base 2025-12-04T09:23:32.7458044Z * [new branch] gh/mikaylagawarecki/378/head -> origin/gh/mikaylagawarecki/378/head 2025-12-04T09:23:32.7458246Z * [new branch] gh/mikaylagawarecki/378/orig -> origin/gh/mikaylagawarecki/378/orig 2025-12-04T09:23:32.7458447Z * [new branch] gh/mikaylagawarecki/379/base -> origin/gh/mikaylagawarecki/379/base 2025-12-04T09:23:32.7458766Z * [new branch] gh/mikaylagawarecki/379/head -> origin/gh/mikaylagawarecki/379/head 2025-12-04T09:23:32.7459294Z * [new branch] gh/mikaylagawarecki/379/orig -> origin/gh/mikaylagawarecki/379/orig 2025-12-04T09:23:32.7460339Z * [new branch] gh/mikaylagawarecki/380/base -> origin/gh/mikaylagawarecki/380/base 2025-12-04T09:23:32.7460538Z * [new branch] gh/mikaylagawarecki/380/head -> origin/gh/mikaylagawarecki/380/head 2025-12-04T09:23:32.7462843Z * [new branch] gh/mikaylagawarecki/380/orig -> origin/gh/mikaylagawarecki/380/orig 2025-12-04T09:23:32.7463204Z * [new branch] gh/mikaylagawarecki/381/base -> origin/gh/mikaylagawarecki/381/base 2025-12-04T09:23:32.7463383Z * [new branch] gh/mikaylagawarecki/381/head -> origin/gh/mikaylagawarecki/381/head 2025-12-04T09:23:32.7463578Z * [new branch] gh/mikaylagawarecki/381/orig -> origin/gh/mikaylagawarecki/381/orig 2025-12-04T09:23:32.7465182Z * [new branch] gh/mikaylagawarecki/382/base -> origin/gh/mikaylagawarecki/382/base 2025-12-04T09:23:32.7465532Z * [new branch] gh/mikaylagawarecki/382/head -> origin/gh/mikaylagawarecki/382/head 2025-12-04T09:23:32.7465813Z * [new branch] gh/mikaylagawarecki/382/orig -> origin/gh/mikaylagawarecki/382/orig 2025-12-04T09:23:32.7467385Z * [new branch] gh/mikaylagawarecki/383/base -> origin/gh/mikaylagawarecki/383/base 2025-12-04T09:23:32.7467608Z * [new branch] gh/mikaylagawarecki/383/head -> origin/gh/mikaylagawarecki/383/head 2025-12-04T09:23:32.7468271Z * [new branch] gh/mikaylagawarecki/383/orig -> origin/gh/mikaylagawarecki/383/orig 2025-12-04T09:23:32.7473165Z * [new branch] gh/mikaylagawarecki/384/base -> origin/gh/mikaylagawarecki/384/base 2025-12-04T09:23:32.7473378Z * [new branch] gh/mikaylagawarecki/384/head -> origin/gh/mikaylagawarecki/384/head 2025-12-04T09:23:32.7473543Z * [new branch] gh/mikaylagawarecki/384/orig -> origin/gh/mikaylagawarecki/384/orig 2025-12-04T09:23:32.7473722Z * [new branch] gh/mikaylagawarecki/385/base -> origin/gh/mikaylagawarecki/385/base 2025-12-04T09:23:32.7473896Z * [new branch] gh/mikaylagawarecki/385/head -> origin/gh/mikaylagawarecki/385/head 2025-12-04T09:23:32.7478940Z * [new branch] gh/mikaylagawarecki/385/orig -> origin/gh/mikaylagawarecki/385/orig 2025-12-04T09:23:32.7482838Z * [new branch] gh/mikaylagawarecki/386/base -> origin/gh/mikaylagawarecki/386/base 2025-12-04T09:23:32.7486784Z * [new branch] gh/mikaylagawarecki/386/head -> origin/gh/mikaylagawarecki/386/head 2025-12-04T09:23:32.7489687Z * [new branch] gh/mikaylagawarecki/386/orig -> origin/gh/mikaylagawarecki/386/orig 2025-12-04T09:23:32.7492793Z * [new branch] gh/mikaylagawarecki/387/base -> origin/gh/mikaylagawarecki/387/base 2025-12-04T09:23:32.7492994Z * [new branch] gh/mikaylagawarecki/387/head -> origin/gh/mikaylagawarecki/387/head 2025-12-04T09:23:32.7493169Z * [new branch] gh/mikaylagawarecki/387/orig -> origin/gh/mikaylagawarecki/387/orig 2025-12-04T09:23:32.7493493Z * [new branch] gh/mikaylagawarecki/388/base -> origin/gh/mikaylagawarecki/388/base 2025-12-04T09:23:32.7493678Z * [new branch] gh/mikaylagawarecki/388/head -> origin/gh/mikaylagawarecki/388/head 2025-12-04T09:23:32.7493850Z * [new branch] gh/mikaylagawarecki/388/orig -> origin/gh/mikaylagawarecki/388/orig 2025-12-04T09:23:32.7494084Z * [new branch] gh/mikaylagawarecki/389/base -> origin/gh/mikaylagawarecki/389/base 2025-12-04T09:23:32.7498930Z * [new branch] gh/mikaylagawarecki/389/head -> origin/gh/mikaylagawarecki/389/head 2025-12-04T09:23:32.7503730Z * [new branch] gh/mikaylagawarecki/389/orig -> origin/gh/mikaylagawarecki/389/orig 2025-12-04T09:23:32.7507892Z * [new branch] gh/mikaylagawarecki/390/base -> origin/gh/mikaylagawarecki/390/base 2025-12-04T09:23:32.7508124Z * [new branch] gh/mikaylagawarecki/390/head -> origin/gh/mikaylagawarecki/390/head 2025-12-04T09:23:32.7508317Z * [new branch] gh/mikaylagawarecki/390/orig -> origin/gh/mikaylagawarecki/390/orig 2025-12-04T09:23:32.7508498Z * [new branch] gh/mikaylagawarecki/391/base -> origin/gh/mikaylagawarecki/391/base 2025-12-04T09:23:32.7508676Z * [new branch] gh/mikaylagawarecki/391/head -> origin/gh/mikaylagawarecki/391/head 2025-12-04T09:23:32.7508900Z * [new branch] gh/mikaylagawarecki/391/orig -> origin/gh/mikaylagawarecki/391/orig 2025-12-04T09:23:32.7509080Z * [new branch] gh/mikaylagawarecki/392/base -> origin/gh/mikaylagawarecki/392/base 2025-12-04T09:23:32.7509270Z * [new branch] gh/mikaylagawarecki/392/head -> origin/gh/mikaylagawarecki/392/head 2025-12-04T09:23:32.7509449Z * [new branch] gh/mikaylagawarecki/392/orig -> origin/gh/mikaylagawarecki/392/orig 2025-12-04T09:23:32.7509602Z * [new branch] gh/mlazos/41/base -> origin/gh/mlazos/41/base 2025-12-04T09:23:32.7509750Z * [new branch] gh/mlazos/41/head -> origin/gh/mlazos/41/head 2025-12-04T09:23:32.7509884Z * [new branch] gh/mlazos/41/orig -> origin/gh/mlazos/41/orig 2025-12-04T09:23:32.7510042Z * [new branch] gh/mlazos/42/base -> origin/gh/mlazos/42/base 2025-12-04T09:23:32.7510178Z * [new branch] gh/mlazos/42/head -> origin/gh/mlazos/42/head 2025-12-04T09:23:32.7510321Z * [new branch] gh/mlazos/42/orig -> origin/gh/mlazos/42/orig 2025-12-04T09:23:32.7510466Z * [new branch] gh/mlazos/43/base -> origin/gh/mlazos/43/base 2025-12-04T09:23:32.7510612Z * [new branch] gh/mlazos/43/head -> origin/gh/mlazos/43/head 2025-12-04T09:23:32.7510750Z * [new branch] gh/mlazos/43/orig -> origin/gh/mlazos/43/orig 2025-12-04T09:23:32.7510901Z * [new branch] gh/mlazos/44/base -> origin/gh/mlazos/44/base 2025-12-04T09:23:32.7511045Z * [new branch] gh/mlazos/44/head -> origin/gh/mlazos/44/head 2025-12-04T09:23:32.7511191Z * [new branch] gh/mlazos/44/orig -> origin/gh/mlazos/44/orig 2025-12-04T09:23:32.7511332Z * [new branch] gh/mlazos/47/base -> origin/gh/mlazos/47/base 2025-12-04T09:23:32.7511687Z * [new branch] gh/mlazos/47/head -> origin/gh/mlazos/47/head 2025-12-04T09:23:32.7511846Z * [new branch] gh/mlazos/47/orig -> origin/gh/mlazos/47/orig 2025-12-04T09:23:32.7511994Z * [new branch] gh/mlazos/48/base -> origin/gh/mlazos/48/base 2025-12-04T09:23:32.7512219Z * [new branch] gh/mlazos/48/head -> origin/gh/mlazos/48/head 2025-12-04T09:23:32.7512365Z * [new branch] gh/mlazos/48/orig -> origin/gh/mlazos/48/orig 2025-12-04T09:23:32.7512508Z * [new branch] gh/mlazos/49/base -> origin/gh/mlazos/49/base 2025-12-04T09:23:32.7512794Z * [new branch] gh/mlazos/49/head -> origin/gh/mlazos/49/head 2025-12-04T09:23:32.7512949Z * [new branch] gh/mlazos/49/orig -> origin/gh/mlazos/49/orig 2025-12-04T09:23:32.7513096Z * [new branch] gh/mlazos/50/base -> origin/gh/mlazos/50/base 2025-12-04T09:23:32.7513233Z * [new branch] gh/mlazos/50/head -> origin/gh/mlazos/50/head 2025-12-04T09:23:32.7513362Z * [new branch] gh/mlazos/50/orig -> origin/gh/mlazos/50/orig 2025-12-04T09:23:32.7513509Z * [new branch] gh/mlazos/51/base -> origin/gh/mlazos/51/base 2025-12-04T09:23:32.7513658Z * [new branch] gh/mlazos/51/head -> origin/gh/mlazos/51/head 2025-12-04T09:23:32.7513792Z * [new branch] gh/mlazos/51/orig -> origin/gh/mlazos/51/orig 2025-12-04T09:23:32.7513933Z * [new branch] gh/mlazos/52/base -> origin/gh/mlazos/52/base 2025-12-04T09:23:32.7514083Z * [new branch] gh/mlazos/52/head -> origin/gh/mlazos/52/head 2025-12-04T09:23:32.7514231Z * [new branch] gh/mlazos/52/orig -> origin/gh/mlazos/52/orig 2025-12-04T09:23:32.7514376Z * [new branch] gh/mlazos/53/base -> origin/gh/mlazos/53/base 2025-12-04T09:23:32.7516205Z * [new branch] gh/mlazos/53/head -> origin/gh/mlazos/53/head 2025-12-04T09:23:32.7516371Z * [new branch] gh/mlazos/53/orig -> origin/gh/mlazos/53/orig 2025-12-04T09:23:32.7516502Z * [new branch] gh/mlazos/54/base -> origin/gh/mlazos/54/base 2025-12-04T09:23:32.7516793Z * [new branch] gh/mlazos/54/head -> origin/gh/mlazos/54/head 2025-12-04T09:23:32.7522012Z * [new branch] gh/mlazos/54/orig -> origin/gh/mlazos/54/orig 2025-12-04T09:23:32.7524744Z * [new branch] gh/mlazos/55/base -> origin/gh/mlazos/55/base 2025-12-04T09:23:32.7525003Z * [new branch] gh/mlazos/55/head -> origin/gh/mlazos/55/head 2025-12-04T09:23:32.7530353Z * [new branch] gh/mlazos/55/orig -> origin/gh/mlazos/55/orig 2025-12-04T09:23:32.7530558Z * [new branch] gh/mlazos/56/base -> origin/gh/mlazos/56/base 2025-12-04T09:23:32.7530851Z * [new branch] gh/mlazos/56/head -> origin/gh/mlazos/56/head 2025-12-04T09:23:32.7531109Z * [new branch] gh/mlazos/56/orig -> origin/gh/mlazos/56/orig 2025-12-04T09:23:32.7531265Z * [new branch] gh/mlazos/57/base -> origin/gh/mlazos/57/base 2025-12-04T09:23:32.7531413Z * [new branch] gh/mlazos/57/head -> origin/gh/mlazos/57/head 2025-12-04T09:23:32.7531619Z * [new branch] gh/mlazos/57/orig -> origin/gh/mlazos/57/orig 2025-12-04T09:23:32.7531872Z * [new branch] gh/mlazos/58/base -> origin/gh/mlazos/58/base 2025-12-04T09:23:32.7531999Z * [new branch] gh/mlazos/58/head -> origin/gh/mlazos/58/head 2025-12-04T09:23:32.7532132Z * [new branch] gh/mlazos/58/orig -> origin/gh/mlazos/58/orig 2025-12-04T09:23:32.7532258Z * [new branch] gh/mlazos/59/base -> origin/gh/mlazos/59/base 2025-12-04T09:23:32.7532523Z * [new branch] gh/mlazos/59/head -> origin/gh/mlazos/59/head 2025-12-04T09:23:32.7532661Z * [new branch] gh/mlazos/59/orig -> origin/gh/mlazos/59/orig 2025-12-04T09:23:32.7532786Z * [new branch] gh/mlazos/60/base -> origin/gh/mlazos/60/base 2025-12-04T09:23:32.7532989Z * [new branch] gh/mlazos/60/head -> origin/gh/mlazos/60/head 2025-12-04T09:23:32.7533120Z * [new branch] gh/mlazos/60/orig -> origin/gh/mlazos/60/orig 2025-12-04T09:23:32.7533243Z * [new branch] gh/mlazos/61/base -> origin/gh/mlazos/61/base 2025-12-04T09:23:32.7533374Z * [new branch] gh/mlazos/61/head -> origin/gh/mlazos/61/head 2025-12-04T09:23:32.7533496Z * [new branch] gh/mlazos/61/orig -> origin/gh/mlazos/61/orig 2025-12-04T09:23:32.7534752Z * [new branch] gh/mlazos/62/base -> origin/gh/mlazos/62/base 2025-12-04T09:23:32.7535084Z * [new branch] gh/mlazos/62/head -> origin/gh/mlazos/62/head 2025-12-04T09:23:32.7537360Z * [new branch] gh/mlazos/62/orig -> origin/gh/mlazos/62/orig 2025-12-04T09:23:32.7537676Z * [new branch] gh/mlazos/63/base -> origin/gh/mlazos/63/base 2025-12-04T09:23:32.7537852Z * [new branch] gh/mlazos/63/head -> origin/gh/mlazos/63/head 2025-12-04T09:23:32.7538310Z * [new branch] gh/mlazos/63/orig -> origin/gh/mlazos/63/orig 2025-12-04T09:23:32.7542159Z * [new branch] gh/mlazos/64/base -> origin/gh/mlazos/64/base 2025-12-04T09:23:32.7542483Z * [new branch] gh/mlazos/64/head -> origin/gh/mlazos/64/head 2025-12-04T09:23:32.7542662Z * [new branch] gh/mlazos/64/orig -> origin/gh/mlazos/64/orig 2025-12-04T09:23:32.7542812Z * [new branch] gh/mlazos/65/base -> origin/gh/mlazos/65/base 2025-12-04T09:23:32.7543038Z * [new branch] gh/mlazos/65/head -> origin/gh/mlazos/65/head 2025-12-04T09:23:32.7543185Z * [new branch] gh/mlazos/65/orig -> origin/gh/mlazos/65/orig 2025-12-04T09:23:32.7543772Z * [new branch] gh/mlazos/66/base -> origin/gh/mlazos/66/base 2025-12-04T09:23:32.7546902Z * [new branch] gh/mlazos/66/head -> origin/gh/mlazos/66/head 2025-12-04T09:23:32.7547231Z * [new branch] gh/mlazos/66/orig -> origin/gh/mlazos/66/orig 2025-12-04T09:23:32.7547428Z * [new branch] gh/mlazos/67/base -> origin/gh/mlazos/67/base 2025-12-04T09:23:32.7547623Z * [new branch] gh/mlazos/67/head -> origin/gh/mlazos/67/head 2025-12-04T09:23:32.7547759Z * [new branch] gh/mlazos/67/orig -> origin/gh/mlazos/67/orig 2025-12-04T09:23:32.7548609Z * [new branch] gh/mlazos/68/base -> origin/gh/mlazos/68/base 2025-12-04T09:23:32.7549277Z * [new branch] gh/mlazos/68/head -> origin/gh/mlazos/68/head 2025-12-04T09:23:32.7549849Z * [new branch] gh/mlazos/68/orig -> origin/gh/mlazos/68/orig 2025-12-04T09:23:32.7553393Z * [new branch] gh/mlazos/69/base -> origin/gh/mlazos/69/base 2025-12-04T09:23:32.7553732Z * [new branch] gh/mlazos/69/head -> origin/gh/mlazos/69/head 2025-12-04T09:23:32.7553922Z * [new branch] gh/mlazos/69/orig -> origin/gh/mlazos/69/orig 2025-12-04T09:23:32.7554089Z * [new branch] gh/mlazos/70/base -> origin/gh/mlazos/70/base 2025-12-04T09:23:32.7554246Z * [new branch] gh/mlazos/70/head -> origin/gh/mlazos/70/head 2025-12-04T09:23:32.7554596Z * [new branch] gh/mlazos/70/orig -> origin/gh/mlazos/70/orig 2025-12-04T09:23:32.7556567Z * [new branch] gh/mlazos/71/base -> origin/gh/mlazos/71/base 2025-12-04T09:23:32.7557081Z * [new branch] gh/mlazos/71/head -> origin/gh/mlazos/71/head 2025-12-04T09:23:32.7557420Z * [new branch] gh/mlazos/71/orig -> origin/gh/mlazos/71/orig 2025-12-04T09:23:32.7558849Z * [new branch] gh/mlazos/72/base -> origin/gh/mlazos/72/base 2025-12-04T09:23:32.7559342Z * [new branch] gh/mlazos/72/head -> origin/gh/mlazos/72/head 2025-12-04T09:23:32.7561833Z * [new branch] gh/mlazos/72/orig -> origin/gh/mlazos/72/orig 2025-12-04T09:23:32.7562005Z * [new branch] gh/mlazos/73/base -> origin/gh/mlazos/73/base 2025-12-04T09:23:32.7562539Z * [new branch] gh/mlazos/73/head -> origin/gh/mlazos/73/head 2025-12-04T09:23:32.7562714Z * [new branch] gh/mlazos/73/orig -> origin/gh/mlazos/73/orig 2025-12-04T09:23:32.7567724Z * [new branch] gh/mrmiywj/1/base -> origin/gh/mrmiywj/1/base 2025-12-04T09:23:32.7568047Z * [new branch] gh/mrmiywj/1/head -> origin/gh/mrmiywj/1/head 2025-12-04T09:23:32.7568243Z * [new branch] gh/muchulee8/73/base -> origin/gh/muchulee8/73/base 2025-12-04T09:23:32.7568399Z * [new branch] gh/muchulee8/73/head -> origin/gh/muchulee8/73/head 2025-12-04T09:23:32.7568549Z * [new branch] gh/muchulee8/73/orig -> origin/gh/muchulee8/73/orig 2025-12-04T09:23:32.7568840Z * [new branch] gh/naveenthangudu/1/base -> origin/gh/naveenthangudu/1/base 2025-12-04T09:23:32.7569790Z * [new branch] gh/naveenthangudu/1/head -> origin/gh/naveenthangudu/1/head 2025-12-04T09:23:32.7573050Z * [new branch] gh/naveenthangudu/1/orig -> origin/gh/naveenthangudu/1/orig 2025-12-04T09:23:32.7573422Z * [new branch] gh/naveenthangudu/2/base -> origin/gh/naveenthangudu/2/base 2025-12-04T09:23:32.7573603Z * [new branch] gh/naveenthangudu/2/head -> origin/gh/naveenthangudu/2/head 2025-12-04T09:23:32.7573789Z * [new branch] gh/naveenthangudu/2/orig -> origin/gh/naveenthangudu/2/orig 2025-12-04T09:23:32.7574079Z * [new branch] gh/naveenthangudu/3/base -> origin/gh/naveenthangudu/3/base 2025-12-04T09:23:32.7574565Z * [new branch] gh/naveenthangudu/3/head -> origin/gh/naveenthangudu/3/head 2025-12-04T09:23:32.7575202Z * [new branch] gh/naveenthangudu/3/orig -> origin/gh/naveenthangudu/3/orig 2025-12-04T09:23:32.7578316Z * [new branch] gh/naveenthangudu/4/base -> origin/gh/naveenthangudu/4/base 2025-12-04T09:23:32.7578510Z * [new branch] gh/naveenthangudu/4/head -> origin/gh/naveenthangudu/4/head 2025-12-04T09:23:32.7578664Z * [new branch] gh/naveenthangudu/4/orig -> origin/gh/naveenthangudu/4/orig 2025-12-04T09:23:32.7578818Z * [new branch] gh/naveenthangudu/5/base -> origin/gh/naveenthangudu/5/base 2025-12-04T09:23:32.7579122Z * [new branch] gh/naveenthangudu/5/head -> origin/gh/naveenthangudu/5/head 2025-12-04T09:23:32.7581259Z * [new branch] gh/naveenthangudu/5/orig -> origin/gh/naveenthangudu/5/orig 2025-12-04T09:23:32.7581619Z * [new branch] gh/naveenthangudu/6/base -> origin/gh/naveenthangudu/6/base 2025-12-04T09:23:32.7581824Z * [new branch] gh/naveenthangudu/6/head -> origin/gh/naveenthangudu/6/head 2025-12-04T09:23:32.7582226Z * [new branch] gh/naveenthangudu/6/orig -> origin/gh/naveenthangudu/6/orig 2025-12-04T09:23:32.7583390Z * [new branch] gh/naveenthangudu/7/base -> origin/gh/naveenthangudu/7/base 2025-12-04T09:23:32.7583838Z * [new branch] gh/naveenthangudu/7/head -> origin/gh/naveenthangudu/7/head 2025-12-04T09:23:32.7585985Z * [new branch] gh/naveenthangudu/7/orig -> origin/gh/naveenthangudu/7/orig 2025-12-04T09:23:32.7586181Z * [new branch] gh/naveenthangudu/8/base -> origin/gh/naveenthangudu/8/base 2025-12-04T09:23:32.7586496Z * [new branch] gh/naveenthangudu/8/head -> origin/gh/naveenthangudu/8/head 2025-12-04T09:23:32.7586964Z * [new branch] gh/naveenthangudu/8/orig -> origin/gh/naveenthangudu/8/orig 2025-12-04T09:23:32.7588235Z * [new branch] gh/naveenthangudu/9/base -> origin/gh/naveenthangudu/9/base 2025-12-04T09:23:32.7588728Z * [new branch] gh/naveenthangudu/9/head -> origin/gh/naveenthangudu/9/head 2025-12-04T09:23:32.7589715Z * [new branch] gh/naveenthangudu/9/orig -> origin/gh/naveenthangudu/9/orig 2025-12-04T09:23:32.7590852Z * [new branch] gh/nikitaved/1/base -> origin/gh/nikitaved/1/base 2025-12-04T09:23:32.7591282Z * [new branch] gh/nikitaved/1/head -> origin/gh/nikitaved/1/head 2025-12-04T09:23:32.7592212Z * [new branch] gh/nikitaved/1/orig -> origin/gh/nikitaved/1/orig 2025-12-04T09:23:32.7594248Z * [new branch] gh/nikitaved/10/base -> origin/gh/nikitaved/10/base 2025-12-04T09:23:32.7594621Z * [new branch] gh/nikitaved/10/head -> origin/gh/nikitaved/10/head 2025-12-04T09:23:32.7594805Z * [new branch] gh/nikitaved/10/orig -> origin/gh/nikitaved/10/orig 2025-12-04T09:23:32.7603022Z * [new branch] gh/nikitaved/11/base -> origin/gh/nikitaved/11/base 2025-12-04T09:23:32.7607794Z * [new branch] gh/nikitaved/11/head -> origin/gh/nikitaved/11/head 2025-12-04T09:23:32.7612605Z * [new branch] gh/nikitaved/11/orig -> origin/gh/nikitaved/11/orig 2025-12-04T09:23:32.7614879Z * [new branch] gh/nikitaved/12/base -> origin/gh/nikitaved/12/base 2025-12-04T09:23:32.7619810Z * [new branch] gh/nikitaved/12/head -> origin/gh/nikitaved/12/head 2025-12-04T09:23:32.7620072Z * [new branch] gh/nikitaved/12/orig -> origin/gh/nikitaved/12/orig 2025-12-04T09:23:32.7625934Z * [new branch] gh/nikitaved/13/base -> origin/gh/nikitaved/13/base 2025-12-04T09:23:32.7627733Z * [new branch] gh/nikitaved/13/head -> origin/gh/nikitaved/13/head 2025-12-04T09:23:32.7627876Z * [new branch] gh/nikitaved/13/orig -> origin/gh/nikitaved/13/orig 2025-12-04T09:23:32.7628056Z * [new branch] gh/nikitaved/14/base -> origin/gh/nikitaved/14/base 2025-12-04T09:23:32.7628193Z * [new branch] gh/nikitaved/14/head -> origin/gh/nikitaved/14/head 2025-12-04T09:23:32.7628326Z * [new branch] gh/nikitaved/14/orig -> origin/gh/nikitaved/14/orig 2025-12-04T09:23:32.7628471Z * [new branch] gh/nikitaved/15/base -> origin/gh/nikitaved/15/base 2025-12-04T09:23:32.7628602Z * [new branch] gh/nikitaved/15/head -> origin/gh/nikitaved/15/head 2025-12-04T09:23:32.7628733Z * [new branch] gh/nikitaved/15/orig -> origin/gh/nikitaved/15/orig 2025-12-04T09:23:32.7628896Z * [new branch] gh/nikitaved/16/base -> origin/gh/nikitaved/16/base 2025-12-04T09:23:32.7629031Z * [new branch] gh/nikitaved/16/head -> origin/gh/nikitaved/16/head 2025-12-04T09:23:32.7629171Z * [new branch] gh/nikitaved/16/orig -> origin/gh/nikitaved/16/orig 2025-12-04T09:23:32.7629321Z * [new branch] gh/nikitaved/2/base -> origin/gh/nikitaved/2/base 2025-12-04T09:23:32.7629464Z * [new branch] gh/nikitaved/2/head -> origin/gh/nikitaved/2/head 2025-12-04T09:23:32.7629604Z * [new branch] gh/nikitaved/2/orig -> origin/gh/nikitaved/2/orig 2025-12-04T09:23:32.7629737Z * [new branch] gh/nikitaved/4/base -> origin/gh/nikitaved/4/base 2025-12-04T09:23:32.7629879Z * [new branch] gh/nikitaved/4/head -> origin/gh/nikitaved/4/head 2025-12-04T09:23:32.7630241Z * [new branch] gh/nikitaved/4/orig -> origin/gh/nikitaved/4/orig 2025-12-04T09:23:32.7630371Z * [new branch] gh/nikitaved/5/base -> origin/gh/nikitaved/5/base 2025-12-04T09:23:32.7630509Z * [new branch] gh/nikitaved/5/head -> origin/gh/nikitaved/5/head 2025-12-04T09:23:32.7630706Z * [new branch] gh/nikitaved/5/orig -> origin/gh/nikitaved/5/orig 2025-12-04T09:23:32.7630848Z * [new branch] gh/nikitaved/6/base -> origin/gh/nikitaved/6/base 2025-12-04T09:23:32.7630979Z * [new branch] gh/nikitaved/6/head -> origin/gh/nikitaved/6/head 2025-12-04T09:23:32.7631111Z * [new branch] gh/nikitaved/6/orig -> origin/gh/nikitaved/6/orig 2025-12-04T09:23:32.7631251Z * [new branch] gh/nikitaved/8/base -> origin/gh/nikitaved/8/base 2025-12-04T09:23:32.7631382Z * [new branch] gh/nikitaved/8/head -> origin/gh/nikitaved/8/head 2025-12-04T09:23:32.7631523Z * [new branch] gh/nikitaved/8/orig -> origin/gh/nikitaved/8/orig 2025-12-04T09:23:32.7631655Z * [new branch] gh/nikitaved/9/base -> origin/gh/nikitaved/9/base 2025-12-04T09:23:32.7631788Z * [new branch] gh/nikitaved/9/head -> origin/gh/nikitaved/9/head 2025-12-04T09:23:32.7631933Z * [new branch] gh/nikitaved/9/orig -> origin/gh/nikitaved/9/orig 2025-12-04T09:23:32.7632067Z * [new branch] gh/oulgen/10/base -> origin/gh/oulgen/10/base 2025-12-04T09:23:32.7632200Z * [new branch] gh/oulgen/10/head -> origin/gh/oulgen/10/head 2025-12-04T09:23:32.7632333Z * [new branch] gh/oulgen/10/orig -> origin/gh/oulgen/10/orig 2025-12-04T09:23:32.7632452Z * [new branch] gh/oulgen/11/base -> origin/gh/oulgen/11/base 2025-12-04T09:23:32.7632579Z * [new branch] gh/oulgen/11/head -> origin/gh/oulgen/11/head 2025-12-04T09:23:32.7632832Z * [new branch] gh/oulgen/11/orig -> origin/gh/oulgen/11/orig 2025-12-04T09:23:32.7632963Z * [new branch] gh/oulgen/12/base -> origin/gh/oulgen/12/base 2025-12-04T09:23:32.7633099Z * [new branch] gh/oulgen/12/head -> origin/gh/oulgen/12/head 2025-12-04T09:23:32.7633226Z * [new branch] gh/oulgen/12/orig -> origin/gh/oulgen/12/orig 2025-12-04T09:23:32.7633363Z * [new branch] gh/oulgen/13/base -> origin/gh/oulgen/13/base 2025-12-04T09:23:32.7633486Z * [new branch] gh/oulgen/13/head -> origin/gh/oulgen/13/head 2025-12-04T09:23:32.7633740Z * [new branch] gh/oulgen/13/orig -> origin/gh/oulgen/13/orig 2025-12-04T09:23:32.7638012Z * [new branch] gh/oulgen/14/base -> origin/gh/oulgen/14/base 2025-12-04T09:23:32.7638334Z * [new branch] gh/oulgen/14/head -> origin/gh/oulgen/14/head 2025-12-04T09:23:32.7638527Z * [new branch] gh/oulgen/14/orig -> origin/gh/oulgen/14/orig 2025-12-04T09:23:32.7638674Z * [new branch] gh/oulgen/15/base -> origin/gh/oulgen/15/base 2025-12-04T09:23:32.7638892Z * [new branch] gh/oulgen/15/head -> origin/gh/oulgen/15/head 2025-12-04T09:23:32.7639043Z * [new branch] gh/oulgen/15/orig -> origin/gh/oulgen/15/orig 2025-12-04T09:23:32.7639169Z * [new branch] gh/oulgen/16/base -> origin/gh/oulgen/16/base 2025-12-04T09:23:32.7639370Z * [new branch] gh/oulgen/16/head -> origin/gh/oulgen/16/head 2025-12-04T09:23:32.7644194Z * [new branch] gh/oulgen/16/orig -> origin/gh/oulgen/16/orig 2025-12-04T09:23:32.7649329Z * [new branch] gh/oulgen/17/base -> origin/gh/oulgen/17/base 2025-12-04T09:23:32.7651343Z * [new branch] gh/oulgen/17/head -> origin/gh/oulgen/17/head 2025-12-04T09:23:32.7651775Z * [new branch] gh/oulgen/17/orig -> origin/gh/oulgen/17/orig 2025-12-04T09:23:32.7656996Z * [new branch] gh/oulgen/18/base -> origin/gh/oulgen/18/base 2025-12-04T09:23:32.7659008Z * [new branch] gh/oulgen/18/head -> origin/gh/oulgen/18/head 2025-12-04T09:23:32.7659417Z * [new branch] gh/oulgen/18/orig -> origin/gh/oulgen/18/orig 2025-12-04T09:23:32.7664787Z * [new branch] gh/oulgen/19/base -> origin/gh/oulgen/19/base 2025-12-04T09:23:32.7669116Z * [new branch] gh/oulgen/19/head -> origin/gh/oulgen/19/head 2025-12-04T09:23:32.7669308Z * [new branch] gh/oulgen/19/orig -> origin/gh/oulgen/19/orig 2025-12-04T09:23:32.7669452Z * [new branch] gh/oulgen/20/base -> origin/gh/oulgen/20/base 2025-12-04T09:23:32.7669596Z * [new branch] gh/oulgen/20/head -> origin/gh/oulgen/20/head 2025-12-04T09:23:32.7669755Z * [new branch] gh/oulgen/20/orig -> origin/gh/oulgen/20/orig 2025-12-04T09:23:32.7669903Z * [new branch] gh/oulgen/21/base -> origin/gh/oulgen/21/base 2025-12-04T09:23:32.7670053Z * [new branch] gh/oulgen/21/head -> origin/gh/oulgen/21/head 2025-12-04T09:23:32.7670205Z * [new branch] gh/oulgen/21/orig -> origin/gh/oulgen/21/orig 2025-12-04T09:23:32.7670348Z * [new branch] gh/oulgen/22/base -> origin/gh/oulgen/22/base 2025-12-04T09:23:32.7670478Z * [new branch] gh/oulgen/22/head -> origin/gh/oulgen/22/head 2025-12-04T09:23:32.7670623Z * [new branch] gh/oulgen/22/orig -> origin/gh/oulgen/22/orig 2025-12-04T09:23:32.7670776Z * [new branch] gh/oulgen/23/base -> origin/gh/oulgen/23/base 2025-12-04T09:23:32.7670921Z * [new branch] gh/oulgen/23/head -> origin/gh/oulgen/23/head 2025-12-04T09:23:32.7671057Z * [new branch] gh/oulgen/23/orig -> origin/gh/oulgen/23/orig 2025-12-04T09:23:32.7671209Z * [new branch] gh/oulgen/24/base -> origin/gh/oulgen/24/base 2025-12-04T09:23:32.7671349Z * [new branch] gh/oulgen/24/head -> origin/gh/oulgen/24/head 2025-12-04T09:23:32.7671503Z * [new branch] gh/oulgen/24/orig -> origin/gh/oulgen/24/orig 2025-12-04T09:23:32.7671633Z * [new branch] gh/oulgen/25/base -> origin/gh/oulgen/25/base 2025-12-04T09:23:32.7671779Z * [new branch] gh/oulgen/25/head -> origin/gh/oulgen/25/head 2025-12-04T09:23:32.7671921Z * [new branch] gh/oulgen/25/orig -> origin/gh/oulgen/25/orig 2025-12-04T09:23:32.7672063Z * [new branch] gh/oulgen/26/base -> origin/gh/oulgen/26/base 2025-12-04T09:23:32.7672211Z * [new branch] gh/oulgen/26/head -> origin/gh/oulgen/26/head 2025-12-04T09:23:32.7672343Z * [new branch] gh/oulgen/26/orig -> origin/gh/oulgen/26/orig 2025-12-04T09:23:32.7672490Z * [new branch] gh/oulgen/4/base -> origin/gh/oulgen/4/base 2025-12-04T09:23:32.7672793Z * [new branch] gh/oulgen/4/head -> origin/gh/oulgen/4/head 2025-12-04T09:23:32.7672943Z * [new branch] gh/oulgen/4/orig -> origin/gh/oulgen/4/orig 2025-12-04T09:23:32.7673082Z * [new branch] gh/oulgen/7/base -> origin/gh/oulgen/7/base 2025-12-04T09:23:32.7673212Z * [new branch] gh/oulgen/7/head -> origin/gh/oulgen/7/head 2025-12-04T09:23:32.7673338Z * [new branch] gh/oulgen/7/orig -> origin/gh/oulgen/7/orig 2025-12-04T09:23:32.7673474Z * [new branch] gh/oulgen/8/base -> origin/gh/oulgen/8/base 2025-12-04T09:23:32.7673603Z * [new branch] gh/oulgen/8/head -> origin/gh/oulgen/8/head 2025-12-04T09:23:32.7673889Z * [new branch] gh/oulgen/8/orig -> origin/gh/oulgen/8/orig 2025-12-04T09:23:32.7674027Z * [new branch] gh/oulgen/9/base -> origin/gh/oulgen/9/base 2025-12-04T09:23:32.7674153Z * [new branch] gh/oulgen/9/head -> origin/gh/oulgen/9/head 2025-12-04T09:23:32.7674332Z * [new branch] gh/oulgen/9/orig -> origin/gh/oulgen/9/orig 2025-12-04T09:23:32.7674533Z * [new branch] gh/patvig/mtia-serialization -> origin/gh/patvig/mtia-serialization 2025-12-04T09:23:32.7674666Z * [new branch] gh/pearu/108/base -> origin/gh/pearu/108/base 2025-12-04T09:23:32.7674802Z * [new branch] gh/pearu/108/head -> origin/gh/pearu/108/head 2025-12-04T09:23:32.7674933Z * [new branch] gh/pearu/108/orig -> origin/gh/pearu/108/orig 2025-12-04T09:23:32.7679438Z * [new branch] gh/pearu/109/base -> origin/gh/pearu/109/base 2025-12-04T09:23:32.7681762Z * [new branch] gh/pearu/109/head -> origin/gh/pearu/109/head 2025-12-04T09:23:32.7686698Z * [new branch] gh/pearu/109/orig -> origin/gh/pearu/109/orig 2025-12-04T09:23:32.7691332Z * [new branch] gh/pearu/110/base -> origin/gh/pearu/110/base 2025-12-04T09:23:32.7696015Z * [new branch] gh/pearu/110/head -> origin/gh/pearu/110/head 2025-12-04T09:23:32.7698070Z * [new branch] gh/pearu/110/orig -> origin/gh/pearu/110/orig 2025-12-04T09:23:32.7698263Z * [new branch] gh/pearu/111/base -> origin/gh/pearu/111/base 2025-12-04T09:23:32.7698394Z * [new branch] gh/pearu/111/head -> origin/gh/pearu/111/head 2025-12-04T09:23:32.7698522Z * [new branch] gh/pearu/111/orig -> origin/gh/pearu/111/orig 2025-12-04T09:23:32.7698655Z * [new branch] gh/pearu/112/base -> origin/gh/pearu/112/base 2025-12-04T09:23:32.7698794Z * [new branch] gh/pearu/112/head -> origin/gh/pearu/112/head 2025-12-04T09:23:32.7698929Z * [new branch] gh/pearu/112/orig -> origin/gh/pearu/112/orig 2025-12-04T09:23:32.7699054Z * [new branch] gh/pearu/115/base -> origin/gh/pearu/115/base 2025-12-04T09:23:32.7699194Z * [new branch] gh/pearu/115/head -> origin/gh/pearu/115/head 2025-12-04T09:23:32.7699322Z * [new branch] gh/pearu/115/orig -> origin/gh/pearu/115/orig 2025-12-04T09:23:32.7699453Z * [new branch] gh/pearu/116/base -> origin/gh/pearu/116/base 2025-12-04T09:23:32.7699574Z * [new branch] gh/pearu/116/head -> origin/gh/pearu/116/head 2025-12-04T09:23:32.7699699Z * [new branch] gh/pearu/116/orig -> origin/gh/pearu/116/orig 2025-12-04T09:23:32.7699819Z * [new branch] gh/pearu/117/base -> origin/gh/pearu/117/base 2025-12-04T09:23:32.7699948Z * [new branch] gh/pearu/117/head -> origin/gh/pearu/117/head 2025-12-04T09:23:32.7700066Z * [new branch] gh/pearu/117/orig -> origin/gh/pearu/117/orig 2025-12-04T09:23:32.7700185Z * [new branch] gh/pearu/118/base -> origin/gh/pearu/118/base 2025-12-04T09:23:32.7700314Z * [new branch] gh/pearu/118/head -> origin/gh/pearu/118/head 2025-12-04T09:23:32.7700435Z * [new branch] gh/pearu/118/orig -> origin/gh/pearu/118/orig 2025-12-04T09:23:32.7700561Z * [new branch] gh/pearu/119/base -> origin/gh/pearu/119/base 2025-12-04T09:23:32.7700681Z * [new branch] gh/pearu/119/head -> origin/gh/pearu/119/head 2025-12-04T09:23:32.7700800Z * [new branch] gh/pearu/119/orig -> origin/gh/pearu/119/orig 2025-12-04T09:23:32.7700924Z * [new branch] gh/pearu/139/base -> origin/gh/pearu/139/base 2025-12-04T09:23:32.7701187Z * [new branch] gh/pearu/139/head -> origin/gh/pearu/139/head 2025-12-04T09:23:32.7701307Z * [new branch] gh/pearu/139/orig -> origin/gh/pearu/139/orig 2025-12-04T09:23:32.7701435Z * [new branch] gh/pearu/140/base -> origin/gh/pearu/140/base 2025-12-04T09:23:32.7701616Z * [new branch] gh/pearu/140/head -> origin/gh/pearu/140/head 2025-12-04T09:23:32.7701744Z * [new branch] gh/pearu/140/orig -> origin/gh/pearu/140/orig 2025-12-04T09:23:32.7702120Z * [new branch] gh/pearu/142/base -> origin/gh/pearu/142/base 2025-12-04T09:23:32.7702265Z * [new branch] gh/pearu/142/head -> origin/gh/pearu/142/head 2025-12-04T09:23:32.7702407Z * [new branch] gh/pearu/142/orig -> origin/gh/pearu/142/orig 2025-12-04T09:23:32.7702540Z * [new branch] gh/pearu/143/base -> origin/gh/pearu/143/base 2025-12-04T09:23:32.7702685Z * [new branch] gh/pearu/143/head -> origin/gh/pearu/143/head 2025-12-04T09:23:32.7702814Z * [new branch] gh/pearu/143/orig -> origin/gh/pearu/143/orig 2025-12-04T09:23:32.7702976Z * [new branch] gh/pearu/147/base -> origin/gh/pearu/147/base 2025-12-04T09:23:32.7704319Z * [new branch] gh/pearu/147/head -> origin/gh/pearu/147/head 2025-12-04T09:23:32.7707985Z * [new branch] gh/pearu/147/orig -> origin/gh/pearu/147/orig 2025-12-04T09:23:32.7708157Z * [new branch] gh/pearu/149/base -> origin/gh/pearu/149/base 2025-12-04T09:23:32.7708310Z * [new branch] gh/pearu/149/head -> origin/gh/pearu/149/head 2025-12-04T09:23:32.7714320Z * [new branch] gh/pearu/149/orig -> origin/gh/pearu/149/orig 2025-12-04T09:23:32.7718066Z * [new branch] gh/pearu/150/base -> origin/gh/pearu/150/base 2025-12-04T09:23:32.7722374Z * [new branch] gh/pearu/150/head -> origin/gh/pearu/150/head 2025-12-04T09:23:32.7726569Z * [new branch] gh/pearu/150/orig -> origin/gh/pearu/150/orig 2025-12-04T09:23:32.7730734Z * [new branch] gh/pearu/151/base -> origin/gh/pearu/151/base 2025-12-04T09:23:32.7733738Z * [new branch] gh/pearu/151/head -> origin/gh/pearu/151/head 2025-12-04T09:23:32.7737428Z * [new branch] gh/pearu/151/orig -> origin/gh/pearu/151/orig 2025-12-04T09:23:32.7737598Z * [new branch] gh/pearu/152/base -> origin/gh/pearu/152/base 2025-12-04T09:23:32.7737741Z * [new branch] gh/pearu/152/head -> origin/gh/pearu/152/head 2025-12-04T09:23:32.7737869Z * [new branch] gh/pearu/152/orig -> origin/gh/pearu/152/orig 2025-12-04T09:23:32.7738017Z * [new branch] gh/pearu/153/base -> origin/gh/pearu/153/base 2025-12-04T09:23:32.7738191Z * [new branch] gh/pearu/153/head -> origin/gh/pearu/153/head 2025-12-04T09:23:32.7738331Z * [new branch] gh/pearu/153/orig -> origin/gh/pearu/153/orig 2025-12-04T09:23:32.7738459Z * [new branch] gh/pearu/154/base -> origin/gh/pearu/154/base 2025-12-04T09:23:32.7738591Z * [new branch] gh/pearu/154/head -> origin/gh/pearu/154/head 2025-12-04T09:23:32.7738722Z * [new branch] gh/pearu/154/orig -> origin/gh/pearu/154/orig 2025-12-04T09:23:32.7738845Z * [new branch] gh/pearu/155/base -> origin/gh/pearu/155/base 2025-12-04T09:23:32.7738968Z * [new branch] gh/pearu/155/head -> origin/gh/pearu/155/head 2025-12-04T09:23:32.7739097Z * [new branch] gh/pearu/155/orig -> origin/gh/pearu/155/orig 2025-12-04T09:23:32.7739221Z * [new branch] gh/pearu/156/base -> origin/gh/pearu/156/base 2025-12-04T09:23:32.7739615Z * [new branch] gh/pearu/156/head -> origin/gh/pearu/156/head 2025-12-04T09:23:32.7739750Z * [new branch] gh/pearu/156/orig -> origin/gh/pearu/156/orig 2025-12-04T09:23:32.7739886Z * [new branch] gh/pearu/56/base -> origin/gh/pearu/56/base 2025-12-04T09:23:32.7740096Z * [new branch] gh/pearu/56/head -> origin/gh/pearu/56/head 2025-12-04T09:23:32.7740223Z * [new branch] gh/pearu/56/orig -> origin/gh/pearu/56/orig 2025-12-04T09:23:32.7740356Z * [new branch] gh/pearu/97/base -> origin/gh/pearu/97/base 2025-12-04T09:23:32.7740482Z * [new branch] gh/pearu/97/head -> origin/gh/pearu/97/head 2025-12-04T09:23:32.7740604Z * [new branch] gh/pearu/97/orig -> origin/gh/pearu/97/orig 2025-12-04T09:23:32.7740749Z * [new branch] gh/pianpwk/21/base -> origin/gh/pianpwk/21/base 2025-12-04T09:23:32.7740882Z * [new branch] gh/pianpwk/21/head -> origin/gh/pianpwk/21/head 2025-12-04T09:23:32.7741013Z * [new branch] gh/pianpwk/28/base -> origin/gh/pianpwk/28/base 2025-12-04T09:23:32.7741150Z * [new branch] gh/pianpwk/28/head -> origin/gh/pianpwk/28/head 2025-12-04T09:23:32.7741282Z * [new branch] gh/pianpwk/28/orig -> origin/gh/pianpwk/28/orig 2025-12-04T09:23:32.7741420Z * [new branch] gh/pianpwk/29/base -> origin/gh/pianpwk/29/base 2025-12-04T09:23:32.7741552Z * [new branch] gh/pianpwk/29/head -> origin/gh/pianpwk/29/head 2025-12-04T09:23:32.7741829Z * [new branch] gh/pianpwk/29/orig -> origin/gh/pianpwk/29/orig 2025-12-04T09:23:32.7747886Z * [new branch] gh/pianpwk/30/base -> origin/gh/pianpwk/30/base 2025-12-04T09:23:32.7749787Z * [new branch] gh/pianpwk/30/head -> origin/gh/pianpwk/30/head 2025-12-04T09:23:32.7749967Z * [new branch] gh/pianpwk/30/orig -> origin/gh/pianpwk/30/orig 2025-12-04T09:23:32.7750227Z * [new branch] gh/pianpwk/31/base -> origin/gh/pianpwk/31/base 2025-12-04T09:23:32.7750377Z * [new branch] gh/pianpwk/31/head -> origin/gh/pianpwk/31/head 2025-12-04T09:23:32.7750537Z * [new branch] gh/pianpwk/31/orig -> origin/gh/pianpwk/31/orig 2025-12-04T09:23:32.7750690Z * [new branch] gh/pianpwk/32/base -> origin/gh/pianpwk/32/base 2025-12-04T09:23:32.7750861Z * [new branch] gh/pianpwk/32/head -> origin/gh/pianpwk/32/head 2025-12-04T09:23:32.7751004Z * [new branch] gh/pianpwk/32/orig -> origin/gh/pianpwk/32/orig 2025-12-04T09:23:32.7751143Z * [new branch] gh/pianpwk/33/base -> origin/gh/pianpwk/33/base 2025-12-04T09:23:32.7751319Z * [new branch] gh/pianpwk/33/head -> origin/gh/pianpwk/33/head 2025-12-04T09:23:32.7751466Z * [new branch] gh/pianpwk/33/orig -> origin/gh/pianpwk/33/orig 2025-12-04T09:23:32.7751615Z * [new branch] gh/pianpwk/34/base -> origin/gh/pianpwk/34/base 2025-12-04T09:23:32.7756461Z * [new branch] gh/pianpwk/34/head -> origin/gh/pianpwk/34/head 2025-12-04T09:23:32.7760905Z * [new branch] gh/pianpwk/34/orig -> origin/gh/pianpwk/34/orig 2025-12-04T09:23:32.7763243Z * [new branch] gh/pianpwk/35/base -> origin/gh/pianpwk/35/base 2025-12-04T09:23:32.7763465Z * [new branch] gh/pianpwk/35/head -> origin/gh/pianpwk/35/head 2025-12-04T09:23:32.7763612Z * [new branch] gh/pianpwk/35/orig -> origin/gh/pianpwk/35/orig 2025-12-04T09:23:32.7763872Z * [new branch] gh/rec/141/base -> origin/gh/rec/141/base 2025-12-04T09:23:32.7764015Z * [new branch] gh/rec/141/head -> origin/gh/rec/141/head 2025-12-04T09:23:32.7764355Z * [new branch] gh/rec/153/base -> origin/gh/rec/153/base 2025-12-04T09:23:32.7769803Z * [new branch] gh/rec/153/head -> origin/gh/rec/153/head 2025-12-04T09:23:32.7773828Z * [new branch] gh/rec/153/orig -> origin/gh/rec/153/orig 2025-12-04T09:23:32.7776063Z * [new branch] gh/rec/154/base -> origin/gh/rec/154/base 2025-12-04T09:23:32.7776398Z * [new branch] gh/rec/154/head -> origin/gh/rec/154/head 2025-12-04T09:23:32.7776536Z * [new branch] gh/rec/154/orig -> origin/gh/rec/154/orig 2025-12-04T09:23:32.7776681Z * [new branch] gh/rec/164/base -> origin/gh/rec/164/base 2025-12-04T09:23:32.7776810Z * [new branch] gh/rec/164/head -> origin/gh/rec/164/head 2025-12-04T09:23:32.7777012Z * [new branch] gh/rec/164/orig -> origin/gh/rec/164/orig 2025-12-04T09:23:32.7777135Z * [new branch] gh/rec/166/base -> origin/gh/rec/166/base 2025-12-04T09:23:32.7777248Z * [new branch] gh/rec/166/head -> origin/gh/rec/166/head 2025-12-04T09:23:32.7777368Z * [new branch] gh/rec/166/orig -> origin/gh/rec/166/orig 2025-12-04T09:23:32.7777487Z * [new branch] gh/rec/167/base -> origin/gh/rec/167/base 2025-12-04T09:23:32.7777654Z * [new branch] gh/rec/167/head -> origin/gh/rec/167/head 2025-12-04T09:23:32.7777769Z * [new branch] gh/rec/167/orig -> origin/gh/rec/167/orig 2025-12-04T09:23:32.7777882Z * [new branch] gh/rec/168/base -> origin/gh/rec/168/base 2025-12-04T09:23:32.7778003Z * [new branch] gh/rec/168/head -> origin/gh/rec/168/head 2025-12-04T09:23:32.7778115Z * [new branch] gh/rec/168/orig -> origin/gh/rec/168/orig 2025-12-04T09:23:32.7782141Z * [new branch] gh/rec/169/base -> origin/gh/rec/169/base 2025-12-04T09:23:32.7787140Z * [new branch] gh/rec/169/head -> origin/gh/rec/169/head 2025-12-04T09:23:32.7787302Z * [new branch] gh/rec/169/orig -> origin/gh/rec/169/orig 2025-12-04T09:23:32.7787450Z * [new branch] gh/rec/170/base -> origin/gh/rec/170/base 2025-12-04T09:23:32.7787586Z * [new branch] gh/rec/170/head -> origin/gh/rec/170/head 2025-12-04T09:23:32.7787711Z * [new branch] gh/rec/170/orig -> origin/gh/rec/170/orig 2025-12-04T09:23:32.7787831Z * [new branch] gh/rec/171/base -> origin/gh/rec/171/base 2025-12-04T09:23:32.7787949Z * [new branch] gh/rec/171/head -> origin/gh/rec/171/head 2025-12-04T09:23:32.7788071Z * [new branch] gh/rec/171/orig -> origin/gh/rec/171/orig 2025-12-04T09:23:32.7788194Z * [new branch] gh/rec/172/base -> origin/gh/rec/172/base 2025-12-04T09:23:32.7788324Z * [new branch] gh/rec/172/head -> origin/gh/rec/172/head 2025-12-04T09:23:32.7788441Z * [new branch] gh/rec/172/orig -> origin/gh/rec/172/orig 2025-12-04T09:23:32.7788562Z * [new branch] gh/rec/173/base -> origin/gh/rec/173/base 2025-12-04T09:23:32.7788688Z * [new branch] gh/rec/173/head -> origin/gh/rec/173/head 2025-12-04T09:23:32.7788805Z * [new branch] gh/rec/173/orig -> origin/gh/rec/173/orig 2025-12-04T09:23:32.7788921Z * [new branch] gh/rec/174/base -> origin/gh/rec/174/base 2025-12-04T09:23:32.7789045Z * [new branch] gh/rec/174/head -> origin/gh/rec/174/head 2025-12-04T09:23:32.7789163Z * [new branch] gh/rec/174/orig -> origin/gh/rec/174/orig 2025-12-04T09:23:32.7789437Z * [new branch] gh/rec/175/base -> origin/gh/rec/175/base 2025-12-04T09:23:32.7789554Z * [new branch] gh/rec/175/head -> origin/gh/rec/175/head 2025-12-04T09:23:32.7789671Z * [new branch] gh/rec/175/orig -> origin/gh/rec/175/orig 2025-12-04T09:23:32.7789847Z * [new branch] gh/rec/176/base -> origin/gh/rec/176/base 2025-12-04T09:23:32.7789966Z * [new branch] gh/rec/176/head -> origin/gh/rec/176/head 2025-12-04T09:23:32.7790095Z * [new branch] gh/rec/176/orig -> origin/gh/rec/176/orig 2025-12-04T09:23:32.7790215Z * [new branch] gh/rec/177/base -> origin/gh/rec/177/base 2025-12-04T09:23:32.7790334Z * [new branch] gh/rec/177/head -> origin/gh/rec/177/head 2025-12-04T09:23:32.7790462Z * [new branch] gh/rec/177/orig -> origin/gh/rec/177/orig 2025-12-04T09:23:32.7790641Z * [new branch] gh/robert-hardwick/3/base -> origin/gh/robert-hardwick/3/base 2025-12-04T09:23:32.7790806Z * [new branch] gh/robert-hardwick/3/head -> origin/gh/robert-hardwick/3/head 2025-12-04T09:23:32.7790960Z * [new branch] gh/robert-hardwick/3/orig -> origin/gh/robert-hardwick/3/orig 2025-12-04T09:23:32.7791108Z * [new branch] gh/robert-hardwick/4/base -> origin/gh/robert-hardwick/4/base 2025-12-04T09:23:32.7791270Z * [new branch] gh/robert-hardwick/4/head -> origin/gh/robert-hardwick/4/head 2025-12-04T09:23:32.7793171Z * [new branch] gh/robert-hardwick/4/orig -> origin/gh/robert-hardwick/4/orig 2025-12-04T09:23:32.7793362Z * [new branch] gh/robert-hardwick/5/base -> origin/gh/robert-hardwick/5/base 2025-12-04T09:23:32.7793739Z * [new branch] gh/robert-hardwick/5/head -> origin/gh/robert-hardwick/5/head 2025-12-04T09:23:32.7798754Z * [new branch] gh/robert-hardwick/5/orig -> origin/gh/robert-hardwick/5/orig 2025-12-04T09:23:32.7798974Z * [new branch] gh/robert-hardwick/6/base -> origin/gh/robert-hardwick/6/base 2025-12-04T09:23:32.7799126Z * [new branch] gh/robert-hardwick/6/head -> origin/gh/robert-hardwick/6/head 2025-12-04T09:23:32.7799282Z * [new branch] gh/robert-hardwick/6/orig -> origin/gh/robert-hardwick/6/orig 2025-12-04T09:23:32.7799452Z * [new branch] gh/robert-hardwick/7/base -> origin/gh/robert-hardwick/7/base 2025-12-04T09:23:32.7799601Z * [new branch] gh/robert-hardwick/7/head -> origin/gh/robert-hardwick/7/head 2025-12-04T09:23:32.7799759Z * [new branch] gh/robert-hardwick/7/orig -> origin/gh/robert-hardwick/7/orig 2025-12-04T09:23:32.7800673Z * [new branch] gh/robert-hardwick/8/base -> origin/gh/robert-hardwick/8/base 2025-12-04T09:23:32.7801630Z * [new branch] gh/robert-hardwick/8/head -> origin/gh/robert-hardwick/8/head 2025-12-04T09:23:32.7802099Z * [new branch] gh/robert-hardwick/8/orig -> origin/gh/robert-hardwick/8/orig 2025-12-04T09:23:32.7806496Z * [new branch] gh/robert-hardwick/9/base -> origin/gh/robert-hardwick/9/base 2025-12-04T09:23:32.7810362Z * [new branch] gh/robert-hardwick/9/head -> origin/gh/robert-hardwick/9/head 2025-12-04T09:23:32.7815282Z * [new branch] gh/robert-hardwick/9/orig -> origin/gh/robert-hardwick/9/orig 2025-12-04T09:23:32.7819433Z * [new branch] gh/rtimpe/1/base -> origin/gh/rtimpe/1/base 2025-12-04T09:23:32.7821113Z * [new branch] gh/rtimpe/1/head -> origin/gh/rtimpe/1/head 2025-12-04T09:23:32.7821262Z * [new branch] gh/rtimpe/2/base -> origin/gh/rtimpe/2/base 2025-12-04T09:23:32.7821387Z * [new branch] gh/rtimpe/2/head -> origin/gh/rtimpe/2/head 2025-12-04T09:23:32.7821535Z * [new branch] gh/rtimpe/22/base -> origin/gh/rtimpe/22/base 2025-12-04T09:23:32.7821918Z * [new branch] gh/rtimpe/22/head -> origin/gh/rtimpe/22/head 2025-12-04T09:23:32.7822042Z * [new branch] gh/rtimpe/22/orig -> origin/gh/rtimpe/22/orig 2025-12-04T09:23:32.7822180Z * [new branch] gh/rtimpe/23/base -> origin/gh/rtimpe/23/base 2025-12-04T09:23:32.7822415Z * [new branch] gh/rtimpe/23/head -> origin/gh/rtimpe/23/head 2025-12-04T09:23:32.7822539Z * [new branch] gh/rtimpe/23/orig -> origin/gh/rtimpe/23/orig 2025-12-04T09:23:32.7822674Z * [new branch] gh/rtimpe/24/base -> origin/gh/rtimpe/24/base 2025-12-04T09:23:32.7822797Z * [new branch] gh/rtimpe/24/head -> origin/gh/rtimpe/24/head 2025-12-04T09:23:32.7822922Z * [new branch] gh/rtimpe/24/orig -> origin/gh/rtimpe/24/orig 2025-12-04T09:23:32.7823054Z * [new branch] gh/rtimpe/25/base -> origin/gh/rtimpe/25/base 2025-12-04T09:23:32.7823180Z * [new branch] gh/rtimpe/25/head -> origin/gh/rtimpe/25/head 2025-12-04T09:23:32.7823308Z * [new branch] gh/rtimpe/25/orig -> origin/gh/rtimpe/25/orig 2025-12-04T09:23:32.7823431Z * [new branch] gh/rtimpe/26/base -> origin/gh/rtimpe/26/base 2025-12-04T09:23:32.7823554Z * [new branch] gh/rtimpe/26/head -> origin/gh/rtimpe/26/head 2025-12-04T09:23:32.7823686Z * [new branch] gh/rtimpe/26/orig -> origin/gh/rtimpe/26/orig 2025-12-04T09:23:32.7823813Z * [new branch] gh/rtimpe/27/base -> origin/gh/rtimpe/27/base 2025-12-04T09:23:32.7823947Z * [new branch] gh/rtimpe/27/head -> origin/gh/rtimpe/27/head 2025-12-04T09:23:32.7824368Z * [new branch] gh/rtimpe/27/orig -> origin/gh/rtimpe/27/orig 2025-12-04T09:23:32.7824498Z * [new branch] gh/rtimpe/28/base -> origin/gh/rtimpe/28/base 2025-12-04T09:23:32.7824635Z * [new branch] gh/rtimpe/28/head -> origin/gh/rtimpe/28/head 2025-12-04T09:23:32.7824787Z * [new branch] gh/rtimpe/28/orig -> origin/gh/rtimpe/28/orig 2025-12-04T09:23:32.7828055Z * [new branch] gh/rtimpe/29/base -> origin/gh/rtimpe/29/base 2025-12-04T09:23:32.7828232Z * [new branch] gh/rtimpe/29/head -> origin/gh/rtimpe/29/head 2025-12-04T09:23:32.7828379Z * [new branch] gh/rtimpe/29/orig -> origin/gh/rtimpe/29/orig 2025-12-04T09:23:32.7828538Z * [new branch] gh/rtimpe/3/base -> origin/gh/rtimpe/3/base 2025-12-04T09:23:32.7831082Z * [new branch] gh/rtimpe/3/head -> origin/gh/rtimpe/3/head 2025-12-04T09:23:32.7831252Z * [new branch] gh/rtimpe/30/base -> origin/gh/rtimpe/30/base 2025-12-04T09:23:32.7831398Z * [new branch] gh/rtimpe/30/head -> origin/gh/rtimpe/30/head 2025-12-04T09:23:32.7831563Z * [new branch] gh/rtimpe/30/orig -> origin/gh/rtimpe/30/orig 2025-12-04T09:23:32.7831703Z * [new branch] gh/rtimpe/31/base -> origin/gh/rtimpe/31/base 2025-12-04T09:23:32.7835650Z * [new branch] gh/rtimpe/31/head -> origin/gh/rtimpe/31/head 2025-12-04T09:23:32.7835821Z * [new branch] gh/rtimpe/31/orig -> origin/gh/rtimpe/31/orig 2025-12-04T09:23:32.7835985Z * [new branch] gh/rtimpe/32/base -> origin/gh/rtimpe/32/base 2025-12-04T09:23:32.7836130Z * [new branch] gh/rtimpe/32/head -> origin/gh/rtimpe/32/head 2025-12-04T09:23:32.7836260Z * [new branch] gh/rtimpe/32/orig -> origin/gh/rtimpe/32/orig 2025-12-04T09:23:32.7836405Z * [new branch] gh/rtimpe/33/base -> origin/gh/rtimpe/33/base 2025-12-04T09:23:32.7841911Z * [new branch] gh/rtimpe/33/head -> origin/gh/rtimpe/33/head 2025-12-04T09:23:32.7846490Z * [new branch] gh/rtimpe/33/orig -> origin/gh/rtimpe/33/orig 2025-12-04T09:23:32.7850472Z * [new branch] gh/rtimpe/34/base -> origin/gh/rtimpe/34/base 2025-12-04T09:23:32.7854773Z * [new branch] gh/rtimpe/34/head -> origin/gh/rtimpe/34/head 2025-12-04T09:23:32.7858249Z * [new branch] gh/rtimpe/34/orig -> origin/gh/rtimpe/34/orig 2025-12-04T09:23:32.7862282Z * [new branch] gh/rtimpe/35/base -> origin/gh/rtimpe/35/base 2025-12-04T09:23:32.7865299Z * [new branch] gh/rtimpe/35/head -> origin/gh/rtimpe/35/head 2025-12-04T09:23:32.7865637Z * [new branch] gh/rtimpe/35/orig -> origin/gh/rtimpe/35/orig 2025-12-04T09:23:32.7865820Z * [new branch] gh/rtimpe/4/base -> origin/gh/rtimpe/4/base 2025-12-04T09:23:32.7865959Z * [new branch] gh/rtimpe/4/head -> origin/gh/rtimpe/4/head 2025-12-04T09:23:32.7866129Z * [new branch] gh/ruisizhang123/1/base -> origin/gh/ruisizhang123/1/base 2025-12-04T09:23:32.7866277Z * [new branch] gh/ruisizhang123/1/head -> origin/gh/ruisizhang123/1/head 2025-12-04T09:23:32.7866425Z * [new branch] gh/ruisizhang123/1/orig -> origin/gh/ruisizhang123/1/orig 2025-12-04T09:23:32.7866568Z * [new branch] gh/ruisizhang123/4/base -> origin/gh/ruisizhang123/4/base 2025-12-04T09:23:32.7866718Z * [new branch] gh/ruisizhang123/4/head -> origin/gh/ruisizhang123/4/head 2025-12-04T09:23:32.7866881Z * [new branch] gh/ruisizhang123/4/orig -> origin/gh/ruisizhang123/4/orig 2025-12-04T09:23:32.7867023Z * [new branch] gh/ruisizhang123/5/base -> origin/gh/ruisizhang123/5/base 2025-12-04T09:23:32.7867172Z * [new branch] gh/ruisizhang123/5/head -> origin/gh/ruisizhang123/5/head 2025-12-04T09:23:32.7867319Z * [new branch] gh/ruisizhang123/5/orig -> origin/gh/ruisizhang123/5/orig 2025-12-04T09:23:32.7867467Z * [new branch] gh/ruisizhang123/6/base -> origin/gh/ruisizhang123/6/base 2025-12-04T09:23:32.7867607Z * [new branch] gh/ruisizhang123/6/head -> origin/gh/ruisizhang123/6/head 2025-12-04T09:23:32.7867751Z * [new branch] gh/ruisizhang123/6/orig -> origin/gh/ruisizhang123/6/orig 2025-12-04T09:23:32.7867911Z * [new branch] gh/ruisizhang123/7/base -> origin/gh/ruisizhang123/7/base 2025-12-04T09:23:32.7868051Z * [new branch] gh/ruisizhang123/7/head -> origin/gh/ruisizhang123/7/head 2025-12-04T09:23:32.7868197Z * [new branch] gh/ruisizhang123/7/orig -> origin/gh/ruisizhang123/7/orig 2025-12-04T09:23:32.7868338Z * [new branch] gh/ruisizhang123/8/base -> origin/gh/ruisizhang123/8/base 2025-12-04T09:23:32.7868477Z * [new branch] gh/ruisizhang123/8/head -> origin/gh/ruisizhang123/8/head 2025-12-04T09:23:32.7868634Z * [new branch] gh/ruisizhang123/8/orig -> origin/gh/ruisizhang123/8/orig 2025-12-04T09:23:32.7868776Z * [new branch] gh/ruisizhang123/9/base -> origin/gh/ruisizhang123/9/base 2025-12-04T09:23:32.7868924Z * [new branch] gh/ruisizhang123/9/head -> origin/gh/ruisizhang123/9/head 2025-12-04T09:23:32.7869067Z * [new branch] gh/ruisizhang123/9/orig -> origin/gh/ruisizhang123/9/orig 2025-12-04T09:23:32.7869212Z * [new branch] gh/seemethere/52/base -> origin/gh/seemethere/52/base 2025-12-04T09:23:32.7869359Z * [new branch] gh/seemethere/52/head -> origin/gh/seemethere/52/head 2025-12-04T09:23:32.7869502Z * [new branch] gh/seemethere/52/orig -> origin/gh/seemethere/52/orig 2025-12-04T09:23:32.7869650Z * [new branch] gh/seemethere/53/base -> origin/gh/seemethere/53/base 2025-12-04T09:23:32.7869787Z * [new branch] gh/seemethere/53/head -> origin/gh/seemethere/53/head 2025-12-04T09:23:32.7870100Z * [new branch] gh/seemethere/53/orig -> origin/gh/seemethere/53/orig 2025-12-04T09:23:32.7870245Z * [new branch] gh/seemethere/54/base -> origin/gh/seemethere/54/base 2025-12-04T09:23:32.7870384Z * [new branch] gh/seemethere/54/head -> origin/gh/seemethere/54/head 2025-12-04T09:23:32.7870572Z * [new branch] gh/seemethere/54/orig -> origin/gh/seemethere/54/orig 2025-12-04T09:23:32.7870727Z * [new branch] gh/seemethere/55/base -> origin/gh/seemethere/55/base 2025-12-04T09:23:32.7870870Z * [new branch] gh/seemethere/55/head -> origin/gh/seemethere/55/head 2025-12-04T09:23:32.7871013Z * [new branch] gh/seemethere/55/orig -> origin/gh/seemethere/55/orig 2025-12-04T09:23:32.7871149Z * [new branch] gh/seemethere/59/base -> origin/gh/seemethere/59/base 2025-12-04T09:23:32.7871697Z * [new branch] gh/seemethere/59/head -> origin/gh/seemethere/59/head 2025-12-04T09:23:32.7871900Z * [new branch] gh/seemethere/59/orig -> origin/gh/seemethere/59/orig 2025-12-04T09:23:32.7873142Z * [new branch] gh/seemethere/62/base -> origin/gh/seemethere/62/base 2025-12-04T09:23:32.7873521Z * [new branch] gh/seemethere/62/head -> origin/gh/seemethere/62/head 2025-12-04T09:23:32.7877190Z * [new branch] gh/seemethere/62/orig -> origin/gh/seemethere/62/orig 2025-12-04T09:23:32.7877519Z * [new branch] gh/seemethere/63/base -> origin/gh/seemethere/63/base 2025-12-04T09:23:32.7877700Z * [new branch] gh/seemethere/63/head -> origin/gh/seemethere/63/head 2025-12-04T09:23:32.7877884Z * [new branch] gh/seemethere/63/orig -> origin/gh/seemethere/63/orig 2025-12-04T09:23:32.7878030Z * [new branch] gh/seemethere/71/base -> origin/gh/seemethere/71/base 2025-12-04T09:23:32.7878269Z * [new branch] gh/seemethere/71/head -> origin/gh/seemethere/71/head 2025-12-04T09:23:32.7878762Z * [new branch] gh/seemethere/71/orig -> origin/gh/seemethere/71/orig 2025-12-04T09:23:32.7882608Z * [new branch] gh/seemethere/72/base -> origin/gh/seemethere/72/base 2025-12-04T09:23:32.7882941Z * [new branch] gh/seemethere/72/head -> origin/gh/seemethere/72/head 2025-12-04T09:23:32.7883116Z * [new branch] gh/seemethere/72/orig -> origin/gh/seemethere/72/orig 2025-12-04T09:23:32.7883313Z * [new branch] gh/seemethere/73/base -> origin/gh/seemethere/73/base 2025-12-04T09:23:32.7883502Z * [new branch] gh/seemethere/73/head -> origin/gh/seemethere/73/head 2025-12-04T09:23:32.7883658Z * [new branch] gh/seemethere/73/orig -> origin/gh/seemethere/73/orig 2025-12-04T09:23:32.7884038Z * [new branch] gh/seemethere/74/base -> origin/gh/seemethere/74/base 2025-12-04T09:23:32.7884973Z * [new branch] gh/seemethere/74/head -> origin/gh/seemethere/74/head 2025-12-04T09:23:32.7885371Z * [new branch] gh/seemethere/74/orig -> origin/gh/seemethere/74/orig 2025-12-04T09:23:32.7887252Z * [new branch] gh/seemethere/75/base -> origin/gh/seemethere/75/base 2025-12-04T09:23:32.7887588Z * [new branch] gh/seemethere/75/head -> origin/gh/seemethere/75/head 2025-12-04T09:23:32.7887778Z * [new branch] gh/seemethere/75/orig -> origin/gh/seemethere/75/orig 2025-12-04T09:23:32.7890073Z * [new branch] gh/seemethere/76/base -> origin/gh/seemethere/76/base 2025-12-04T09:23:32.7890408Z * [new branch] gh/seemethere/76/head -> origin/gh/seemethere/76/head 2025-12-04T09:23:32.7890619Z * [new branch] gh/seemethere/76/orig -> origin/gh/seemethere/76/orig 2025-12-04T09:23:32.7891021Z * [new branch] gh/shunting314/145/base -> origin/gh/shunting314/145/base 2025-12-04T09:23:32.7892367Z * [new branch] gh/shunting314/145/head -> origin/gh/shunting314/145/head 2025-12-04T09:23:32.7892628Z * [new branch] gh/shunting314/145/orig -> origin/gh/shunting314/145/orig 2025-12-04T09:23:32.7896075Z * [new branch] gh/shunting314/176/base -> origin/gh/shunting314/176/base 2025-12-04T09:23:32.7896434Z * [new branch] gh/shunting314/176/head -> origin/gh/shunting314/176/head 2025-12-04T09:23:32.7896599Z * [new branch] gh/shunting314/176/orig -> origin/gh/shunting314/176/orig 2025-12-04T09:23:32.7896752Z * [new branch] gh/shunting314/249/base -> origin/gh/shunting314/249/base 2025-12-04T09:23:32.7896967Z * [new branch] gh/shunting314/249/head -> origin/gh/shunting314/249/head 2025-12-04T09:23:32.7898494Z * [new branch] gh/shunting314/249/orig -> origin/gh/shunting314/249/orig 2025-12-04T09:23:32.7898736Z * [new branch] gh/shunting314/253/base -> origin/gh/shunting314/253/base 2025-12-04T09:23:32.7899646Z * [new branch] gh/shunting314/253/head -> origin/gh/shunting314/253/head 2025-12-04T09:23:32.7899813Z * [new branch] gh/shunting314/253/orig -> origin/gh/shunting314/253/orig 2025-12-04T09:23:32.7901111Z * [new branch] gh/shunting314/256/base -> origin/gh/shunting314/256/base 2025-12-04T09:23:32.7901579Z * [new branch] gh/shunting314/256/head -> origin/gh/shunting314/256/head 2025-12-04T09:23:32.7903830Z * [new branch] gh/shunting314/256/orig -> origin/gh/shunting314/256/orig 2025-12-04T09:23:32.7908742Z * [new branch] gh/shunting314/257/base -> origin/gh/shunting314/257/base 2025-12-04T09:23:32.7910059Z * [new branch] gh/shunting314/257/head -> origin/gh/shunting314/257/head 2025-12-04T09:23:32.7910225Z * [new branch] gh/shunting314/257/orig -> origin/gh/shunting314/257/orig 2025-12-04T09:23:32.7910387Z * [new branch] gh/shunting314/258/base -> origin/gh/shunting314/258/base 2025-12-04T09:23:32.7914563Z * [new branch] gh/shunting314/258/head -> origin/gh/shunting314/258/head 2025-12-04T09:23:32.7914841Z * [new branch] gh/shunting314/258/orig -> origin/gh/shunting314/258/orig 2025-12-04T09:23:32.7920905Z * [new branch] gh/shunting314/259/base -> origin/gh/shunting314/259/base 2025-12-04T09:23:32.7923166Z * [new branch] gh/shunting314/259/head -> origin/gh/shunting314/259/head 2025-12-04T09:23:32.7928990Z * [new branch] gh/shunting314/259/orig -> origin/gh/shunting314/259/orig 2025-12-04T09:23:32.7931071Z * [new branch] gh/shunting314/260/base -> origin/gh/shunting314/260/base 2025-12-04T09:23:32.7937169Z * [new branch] gh/shunting314/260/head -> origin/gh/shunting314/260/head 2025-12-04T09:23:32.7940708Z * [new branch] gh/shunting314/260/orig -> origin/gh/shunting314/260/orig 2025-12-04T09:23:32.7940989Z * [new branch] gh/shunting314/261/base -> origin/gh/shunting314/261/base 2025-12-04T09:23:32.7945498Z * [new branch] gh/shunting314/261/head -> origin/gh/shunting314/261/head 2025-12-04T09:23:32.7945715Z * [new branch] gh/shunting314/261/orig -> origin/gh/shunting314/261/orig 2025-12-04T09:23:32.7945858Z * [new branch] gh/shunting314/262/base -> origin/gh/shunting314/262/base 2025-12-04T09:23:32.7946005Z * [new branch] gh/shunting314/262/head -> origin/gh/shunting314/262/head 2025-12-04T09:23:32.7946143Z * [new branch] gh/shunting314/262/orig -> origin/gh/shunting314/262/orig 2025-12-04T09:23:32.7946279Z * [new branch] gh/shunting314/263/base -> origin/gh/shunting314/263/base 2025-12-04T09:23:32.7946665Z * [new branch] gh/shunting314/263/head -> origin/gh/shunting314/263/head 2025-12-04T09:23:32.7946814Z * [new branch] gh/shunting314/263/orig -> origin/gh/shunting314/263/orig 2025-12-04T09:23:32.7946971Z * [new branch] gh/shunting314/264/base -> origin/gh/shunting314/264/base 2025-12-04T09:23:32.7947196Z * [new branch] gh/shunting314/264/head -> origin/gh/shunting314/264/head 2025-12-04T09:23:32.7947341Z * [new branch] gh/shunting314/264/orig -> origin/gh/shunting314/264/orig 2025-12-04T09:23:32.7947486Z * [new branch] gh/shunting314/265/base -> origin/gh/shunting314/265/base 2025-12-04T09:23:32.7947623Z * [new branch] gh/shunting314/265/head -> origin/gh/shunting314/265/head 2025-12-04T09:23:32.7947769Z * [new branch] gh/shunting314/265/orig -> origin/gh/shunting314/265/orig 2025-12-04T09:23:32.7947906Z * [new branch] gh/shunting314/266/base -> origin/gh/shunting314/266/base 2025-12-04T09:23:32.7948047Z * [new branch] gh/shunting314/266/head -> origin/gh/shunting314/266/head 2025-12-04T09:23:32.7948191Z * [new branch] gh/shunting314/266/orig -> origin/gh/shunting314/266/orig 2025-12-04T09:23:32.7948333Z * [new branch] gh/shunting314/267/base -> origin/gh/shunting314/267/base 2025-12-04T09:23:32.7948481Z * [new branch] gh/shunting314/267/head -> origin/gh/shunting314/267/head 2025-12-04T09:23:32.7948620Z * [new branch] gh/shunting314/267/orig -> origin/gh/shunting314/267/orig 2025-12-04T09:23:32.7948765Z * [new branch] gh/shunting314/268/base -> origin/gh/shunting314/268/base 2025-12-04T09:23:32.7948911Z * [new branch] gh/shunting314/268/head -> origin/gh/shunting314/268/head 2025-12-04T09:23:32.7949050Z * [new branch] gh/shunting314/268/orig -> origin/gh/shunting314/268/orig 2025-12-04T09:23:32.7949192Z * [new branch] gh/shunting314/269/base -> origin/gh/shunting314/269/base 2025-12-04T09:23:32.7949336Z * [new branch] gh/shunting314/269/head -> origin/gh/shunting314/269/head 2025-12-04T09:23:32.7949474Z * [new branch] gh/shunting314/269/orig -> origin/gh/shunting314/269/orig 2025-12-04T09:23:32.7949635Z * [new branch] gh/silverguo/1/base -> origin/gh/silverguo/1/base 2025-12-04T09:23:32.7949775Z * [new branch] gh/silverguo/1/head -> origin/gh/silverguo/1/head 2025-12-04T09:23:32.7949910Z * [new branch] gh/silverguo/2/base -> origin/gh/silverguo/2/base 2025-12-04T09:23:32.7950050Z * [new branch] gh/silverguo/2/head -> origin/gh/silverguo/2/head 2025-12-04T09:23:32.7950180Z * [new branch] gh/silverguo/3/base -> origin/gh/silverguo/3/base 2025-12-04T09:23:32.7950317Z * [new branch] gh/silverguo/3/head -> origin/gh/silverguo/3/head 2025-12-04T09:23:32.7950462Z * [new branch] gh/silverguo/4/base -> origin/gh/silverguo/4/base 2025-12-04T09:23:32.7950592Z * [new branch] gh/silverguo/4/head -> origin/gh/silverguo/4/head 2025-12-04T09:23:32.7950743Z * [new branch] gh/slayton58/39/base -> origin/gh/slayton58/39/base 2025-12-04T09:23:32.7950880Z * [new branch] gh/slayton58/39/head -> origin/gh/slayton58/39/head 2025-12-04T09:23:32.7951024Z * [new branch] gh/slayton58/39/orig -> origin/gh/slayton58/39/orig 2025-12-04T09:23:32.7951157Z * [new branch] gh/slayton58/42/base -> origin/gh/slayton58/42/base 2025-12-04T09:23:32.7951286Z * [new branch] gh/slayton58/42/head -> origin/gh/slayton58/42/head 2025-12-04T09:23:32.7951425Z * [new branch] gh/slayton58/42/orig -> origin/gh/slayton58/42/orig 2025-12-04T09:23:32.7951556Z * [new branch] gh/slayton58/43/base -> origin/gh/slayton58/43/base 2025-12-04T09:23:32.7951784Z * [new branch] gh/slayton58/43/head -> origin/gh/slayton58/43/head 2025-12-04T09:23:32.7951914Z * [new branch] gh/slayton58/43/orig -> origin/gh/slayton58/43/orig 2025-12-04T09:23:32.7952470Z * [new branch] gh/slayton58/44/base -> origin/gh/slayton58/44/base 2025-12-04T09:23:32.7953401Z * [new branch] gh/slayton58/44/head -> origin/gh/slayton58/44/head 2025-12-04T09:23:32.7957262Z * [new branch] gh/slayton58/44/orig -> origin/gh/slayton58/44/orig 2025-12-04T09:23:32.7957406Z * [new branch] gh/slayton58/45/base -> origin/gh/slayton58/45/base 2025-12-04T09:23:32.7957540Z * [new branch] gh/slayton58/45/head -> origin/gh/slayton58/45/head 2025-12-04T09:23:32.7957667Z * [new branch] gh/slayton58/45/orig -> origin/gh/slayton58/45/orig 2025-12-04T09:23:32.7957803Z * [new branch] gh/slayton58/46/base -> origin/gh/slayton58/46/base 2025-12-04T09:23:32.7961902Z * [new branch] gh/slayton58/46/head -> origin/gh/slayton58/46/head 2025-12-04T09:23:32.7962090Z * [new branch] gh/slayton58/46/orig -> origin/gh/slayton58/46/orig 2025-12-04T09:23:32.7962259Z * [new branch] gh/slayton58/6/base -> origin/gh/slayton58/6/base 2025-12-04T09:23:32.7962443Z * [new branch] gh/slayton58/6/head -> origin/gh/slayton58/6/head 2025-12-04T09:23:32.7962718Z * [new branch] gh/slayton58/7/base -> origin/gh/slayton58/7/base 2025-12-04T09:23:32.7966448Z * [new branch] gh/slayton58/7/head -> origin/gh/slayton58/7/head 2025-12-04T09:23:32.7966740Z * [new branch] gh/soulitzer/269/base -> origin/gh/soulitzer/269/base 2025-12-04T09:23:32.7966904Z * [new branch] gh/soulitzer/269/head -> origin/gh/soulitzer/269/head 2025-12-04T09:23:32.7967146Z * [new branch] gh/soulitzer/269/orig -> origin/gh/soulitzer/269/orig 2025-12-04T09:23:32.7970733Z * [new branch] gh/soulitzer/276/base -> origin/gh/soulitzer/276/base 2025-12-04T09:23:32.7970904Z * [new branch] gh/soulitzer/276/head -> origin/gh/soulitzer/276/head 2025-12-04T09:23:32.7971072Z * [new branch] gh/soulitzer/276/orig -> origin/gh/soulitzer/276/orig 2025-12-04T09:23:32.7971207Z * [new branch] gh/soulitzer/287/base -> origin/gh/soulitzer/287/base 2025-12-04T09:23:32.7971351Z * [new branch] gh/soulitzer/287/head -> origin/gh/soulitzer/287/head 2025-12-04T09:23:32.7971488Z * [new branch] gh/soulitzer/287/orig -> origin/gh/soulitzer/287/orig 2025-12-04T09:23:32.7971619Z * [new branch] gh/soulitzer/296/base -> origin/gh/soulitzer/296/base 2025-12-04T09:23:32.7973866Z * [new branch] gh/soulitzer/296/head -> origin/gh/soulitzer/296/head 2025-12-04T09:23:32.7974117Z * [new branch] gh/soulitzer/296/orig -> origin/gh/soulitzer/296/orig 2025-12-04T09:23:32.7974273Z * [new branch] gh/soulitzer/299/base -> origin/gh/soulitzer/299/base 2025-12-04T09:23:32.7974493Z * [new branch] gh/soulitzer/299/head -> origin/gh/soulitzer/299/head 2025-12-04T09:23:32.7974697Z * [new branch] gh/soulitzer/299/orig -> origin/gh/soulitzer/299/orig 2025-12-04T09:23:32.7980494Z * [new branch] gh/soulitzer/300/base -> origin/gh/soulitzer/300/base 2025-12-04T09:23:32.7983613Z * [new branch] gh/soulitzer/300/head -> origin/gh/soulitzer/300/head 2025-12-04T09:23:32.7983760Z * [new branch] gh/soulitzer/300/orig -> origin/gh/soulitzer/300/orig 2025-12-04T09:23:32.7983906Z * [new branch] gh/soulitzer/301/base -> origin/gh/soulitzer/301/base 2025-12-04T09:23:32.7984038Z * [new branch] gh/soulitzer/301/head -> origin/gh/soulitzer/301/head 2025-12-04T09:23:32.7984357Z * [new branch] gh/soulitzer/301/orig -> origin/gh/soulitzer/301/orig 2025-12-04T09:23:32.7984499Z * [new branch] gh/soulitzer/313/base -> origin/gh/soulitzer/313/base 2025-12-04T09:23:32.7984634Z * [new branch] gh/soulitzer/313/head -> origin/gh/soulitzer/313/head 2025-12-04T09:23:32.7984841Z * [new branch] gh/soulitzer/313/orig -> origin/gh/soulitzer/313/orig 2025-12-04T09:23:32.7984975Z * [new branch] gh/soulitzer/319/base -> origin/gh/soulitzer/319/base 2025-12-04T09:23:32.7985106Z * [new branch] gh/soulitzer/319/head -> origin/gh/soulitzer/319/head 2025-12-04T09:23:32.7985243Z * [new branch] gh/soulitzer/319/orig -> origin/gh/soulitzer/319/orig 2025-12-04T09:23:32.7988126Z * [new branch] gh/soulitzer/320/base -> origin/gh/soulitzer/320/base 2025-12-04T09:23:32.7988283Z * [new branch] gh/soulitzer/320/head -> origin/gh/soulitzer/320/head 2025-12-04T09:23:32.7988493Z * [new branch] gh/soulitzer/320/orig -> origin/gh/soulitzer/320/orig 2025-12-04T09:23:32.7988628Z * [new branch] gh/soulitzer/336/base -> origin/gh/soulitzer/336/base 2025-12-04T09:23:32.7988772Z * [new branch] gh/soulitzer/336/head -> origin/gh/soulitzer/336/head 2025-12-04T09:23:32.7988910Z * [new branch] gh/soulitzer/336/orig -> origin/gh/soulitzer/336/orig 2025-12-04T09:23:32.7989752Z * [new branch] gh/soulitzer/347/base -> origin/gh/soulitzer/347/base 2025-12-04T09:23:32.7990344Z * [new branch] gh/soulitzer/347/head -> origin/gh/soulitzer/347/head 2025-12-04T09:23:32.7990925Z * [new branch] gh/soulitzer/347/orig -> origin/gh/soulitzer/347/orig 2025-12-04T09:23:32.7992150Z * [new branch] gh/soulitzer/349/base -> origin/gh/soulitzer/349/base 2025-12-04T09:23:32.7992550Z * [new branch] gh/soulitzer/349/head -> origin/gh/soulitzer/349/head 2025-12-04T09:23:32.7996433Z * [new branch] gh/soulitzer/349/orig -> origin/gh/soulitzer/349/orig 2025-12-04T09:23:32.7996676Z * [new branch] gh/soulitzer/350/base -> origin/gh/soulitzer/350/base 2025-12-04T09:23:32.8000173Z * [new branch] gh/soulitzer/350/head -> origin/gh/soulitzer/350/head 2025-12-04T09:23:32.8000347Z * [new branch] gh/soulitzer/350/orig -> origin/gh/soulitzer/350/orig 2025-12-04T09:23:32.8000499Z * [new branch] gh/soulitzer/351/base -> origin/gh/soulitzer/351/base 2025-12-04T09:23:32.8000639Z * [new branch] gh/soulitzer/351/head -> origin/gh/soulitzer/351/head 2025-12-04T09:23:32.8000789Z * [new branch] gh/soulitzer/351/orig -> origin/gh/soulitzer/351/orig 2025-12-04T09:23:32.8005358Z * [new branch] gh/soulitzer/353/base -> origin/gh/soulitzer/353/base 2025-12-04T09:23:32.8009497Z * [new branch] gh/soulitzer/353/head -> origin/gh/soulitzer/353/head 2025-12-04T09:23:32.8012235Z * [new branch] gh/soulitzer/353/orig -> origin/gh/soulitzer/353/orig 2025-12-04T09:23:32.8012379Z * [new branch] gh/soulitzer/358/base -> origin/gh/soulitzer/358/base 2025-12-04T09:23:32.8012938Z * [new branch] gh/soulitzer/358/head -> origin/gh/soulitzer/358/head 2025-12-04T09:23:32.8013112Z * [new branch] gh/soulitzer/358/orig -> origin/gh/soulitzer/358/orig 2025-12-04T09:23:32.8013260Z * [new branch] gh/soulitzer/359/base -> origin/gh/soulitzer/359/base 2025-12-04T09:23:32.8013397Z * [new branch] gh/soulitzer/359/head -> origin/gh/soulitzer/359/head 2025-12-04T09:23:32.8013532Z * [new branch] gh/soulitzer/359/orig -> origin/gh/soulitzer/359/orig 2025-12-04T09:23:32.8013672Z * [new branch] gh/soulitzer/374/base -> origin/gh/soulitzer/374/base 2025-12-04T09:23:32.8014006Z * [new branch] gh/soulitzer/374/head -> origin/gh/soulitzer/374/head 2025-12-04T09:23:32.8014142Z * [new branch] gh/soulitzer/374/orig -> origin/gh/soulitzer/374/orig 2025-12-04T09:23:32.8014269Z * [new branch] gh/soulitzer/375/base -> origin/gh/soulitzer/375/base 2025-12-04T09:23:32.8014454Z * [new branch] gh/soulitzer/375/head -> origin/gh/soulitzer/375/head 2025-12-04T09:23:32.8014595Z * [new branch] gh/soulitzer/375/orig -> origin/gh/soulitzer/375/orig 2025-12-04T09:23:32.8014725Z * [new branch] gh/soulitzer/380/base -> origin/gh/soulitzer/380/base 2025-12-04T09:23:32.8014864Z * [new branch] gh/soulitzer/380/head -> origin/gh/soulitzer/380/head 2025-12-04T09:23:32.8014993Z * [new branch] gh/soulitzer/380/orig -> origin/gh/soulitzer/380/orig 2025-12-04T09:23:32.8015125Z * [new branch] gh/soulitzer/385/base -> origin/gh/soulitzer/385/base 2025-12-04T09:23:32.8020770Z * [new branch] gh/soulitzer/385/head -> origin/gh/soulitzer/385/head 2025-12-04T09:23:32.8026002Z * [new branch] gh/soulitzer/385/orig -> origin/gh/soulitzer/385/orig 2025-12-04T09:23:32.8026313Z * [new branch] gh/soulitzer/386/base -> origin/gh/soulitzer/386/base 2025-12-04T09:23:32.8026561Z * [new branch] gh/soulitzer/386/head -> origin/gh/soulitzer/386/head 2025-12-04T09:23:32.8026716Z * [new branch] gh/soulitzer/386/orig -> origin/gh/soulitzer/386/orig 2025-12-04T09:23:32.8026892Z * [new branch] gh/soulitzer/387/base -> origin/gh/soulitzer/387/base 2025-12-04T09:23:32.8027047Z * [new branch] gh/soulitzer/387/head -> origin/gh/soulitzer/387/head 2025-12-04T09:23:32.8027197Z * [new branch] gh/soulitzer/387/orig -> origin/gh/soulitzer/387/orig 2025-12-04T09:23:32.8027360Z * [new branch] gh/soulitzer/388/base -> origin/gh/soulitzer/388/base 2025-12-04T09:23:32.8027506Z * [new branch] gh/soulitzer/388/head -> origin/gh/soulitzer/388/head 2025-12-04T09:23:32.8028025Z * [new branch] gh/soulitzer/388/orig -> origin/gh/soulitzer/388/orig 2025-12-04T09:23:32.8028234Z * [new branch] gh/soulitzer/389/base -> origin/gh/soulitzer/389/base 2025-12-04T09:23:32.8028377Z * [new branch] gh/soulitzer/389/head -> origin/gh/soulitzer/389/head 2025-12-04T09:23:32.8028517Z * [new branch] gh/soulitzer/389/orig -> origin/gh/soulitzer/389/orig 2025-12-04T09:23:32.8028667Z * [new branch] gh/soulitzer/390/base -> origin/gh/soulitzer/390/base 2025-12-04T09:23:32.8028811Z * [new branch] gh/soulitzer/390/head -> origin/gh/soulitzer/390/head 2025-12-04T09:23:32.8028962Z * [new branch] gh/soulitzer/390/orig -> origin/gh/soulitzer/390/orig 2025-12-04T09:23:32.8029124Z * [new branch] gh/soulitzer/391/base -> origin/gh/soulitzer/391/base 2025-12-04T09:23:32.8029266Z * [new branch] gh/soulitzer/391/head -> origin/gh/soulitzer/391/head 2025-12-04T09:23:32.8029431Z * [new branch] gh/soulitzer/391/orig -> origin/gh/soulitzer/391/orig 2025-12-04T09:23:32.8029571Z * [new branch] gh/soulitzer/392/base -> origin/gh/soulitzer/392/base 2025-12-04T09:23:32.8029712Z * [new branch] gh/soulitzer/392/head -> origin/gh/soulitzer/392/head 2025-12-04T09:23:32.8029851Z * [new branch] gh/soulitzer/392/orig -> origin/gh/soulitzer/392/orig 2025-12-04T09:23:32.8029998Z * [new branch] gh/swolchok/728/next -> origin/gh/swolchok/728/next 2025-12-04T09:23:32.8031135Z * [new branch] gh/swolchok/819/base -> origin/gh/swolchok/819/base 2025-12-04T09:23:32.8031450Z * [new branch] gh/swolchok/819/head -> origin/gh/swolchok/819/head 2025-12-04T09:23:32.8031731Z * [new branch] gh/swolchok/819/orig -> origin/gh/swolchok/819/orig 2025-12-04T09:23:32.8033544Z * [new branch] gh/swolchok/824/base -> origin/gh/swolchok/824/base 2025-12-04T09:23:32.8034017Z * [new branch] gh/swolchok/824/head -> origin/gh/swolchok/824/head 2025-12-04T09:23:32.8041574Z * [new branch] gh/swolchok/824/orig -> origin/gh/swolchok/824/orig 2025-12-04T09:23:32.8047401Z * [new branch] gh/swolchok/829/base -> origin/gh/swolchok/829/base 2025-12-04T09:23:32.8049470Z * [new branch] gh/swolchok/829/head -> origin/gh/swolchok/829/head 2025-12-04T09:23:32.8049733Z * [new branch] gh/swolchok/829/orig -> origin/gh/swolchok/829/orig 2025-12-04T09:23:32.8054987Z * [new branch] gh/swolchok/839/base -> origin/gh/swolchok/839/base 2025-12-04T09:23:32.8057419Z * [new branch] gh/swolchok/839/head -> origin/gh/swolchok/839/head 2025-12-04T09:23:32.8057700Z * [new branch] gh/swolchok/839/orig -> origin/gh/swolchok/839/orig 2025-12-04T09:23:32.8063240Z * [new branch] gh/swolchok/841/base -> origin/gh/swolchok/841/base 2025-12-04T09:23:32.8067221Z * [new branch] gh/swolchok/841/head -> origin/gh/swolchok/841/head 2025-12-04T09:23:32.8068849Z * [new branch] gh/swolchok/841/orig -> origin/gh/swolchok/841/orig 2025-12-04T09:23:32.8069002Z * [new branch] gh/swolchok/842/base -> origin/gh/swolchok/842/base 2025-12-04T09:23:32.8069264Z * [new branch] gh/swolchok/842/head -> origin/gh/swolchok/842/head 2025-12-04T09:23:32.8069409Z * [new branch] gh/swolchok/842/orig -> origin/gh/swolchok/842/orig 2025-12-04T09:23:32.8069536Z * [new branch] gh/swolchok/845/base -> origin/gh/swolchok/845/base 2025-12-04T09:23:32.8069678Z * [new branch] gh/swolchok/845/head -> origin/gh/swolchok/845/head 2025-12-04T09:23:32.8069816Z * [new branch] gh/swolchok/845/orig -> origin/gh/swolchok/845/orig 2025-12-04T09:23:32.8069943Z * [new branch] gh/swolchok/848/base -> origin/gh/swolchok/848/base 2025-12-04T09:23:32.8070098Z * [new branch] gh/swolchok/848/head -> origin/gh/swolchok/848/head 2025-12-04T09:23:32.8070224Z * [new branch] gh/swolchok/848/orig -> origin/gh/swolchok/848/orig 2025-12-04T09:23:32.8070359Z * [new branch] gh/swolchok/856/base -> origin/gh/swolchok/856/base 2025-12-04T09:23:32.8070486Z * [new branch] gh/swolchok/856/head -> origin/gh/swolchok/856/head 2025-12-04T09:23:32.8070612Z * [new branch] gh/swolchok/856/orig -> origin/gh/swolchok/856/orig 2025-12-04T09:23:32.8070745Z * [new branch] gh/swolchok/860/base -> origin/gh/swolchok/860/base 2025-12-04T09:23:32.8070876Z * [new branch] gh/swolchok/860/head -> origin/gh/swolchok/860/head 2025-12-04T09:23:32.8071002Z * [new branch] gh/swolchok/860/orig -> origin/gh/swolchok/860/orig 2025-12-04T09:23:32.8071135Z * [new branch] gh/swolchok/861/base -> origin/gh/swolchok/861/base 2025-12-04T09:23:32.8071263Z * [new branch] gh/swolchok/861/head -> origin/gh/swolchok/861/head 2025-12-04T09:23:32.8071394Z * [new branch] gh/swolchok/861/orig -> origin/gh/swolchok/861/orig 2025-12-04T09:23:32.8071520Z * [new branch] gh/swolchok/862/base -> origin/gh/swolchok/862/base 2025-12-04T09:23:32.8071643Z * [new branch] gh/swolchok/862/head -> origin/gh/swolchok/862/head 2025-12-04T09:23:32.8071776Z * [new branch] gh/swolchok/862/orig -> origin/gh/swolchok/862/orig 2025-12-04T09:23:32.8072036Z * [new branch] gh/swolchok/863/base -> origin/gh/swolchok/863/base 2025-12-04T09:23:32.8072171Z * [new branch] gh/swolchok/863/head -> origin/gh/swolchok/863/head 2025-12-04T09:23:32.8072296Z * [new branch] gh/swolchok/863/orig -> origin/gh/swolchok/863/orig 2025-12-04T09:23:32.8072480Z * [new branch] gh/swolchok/864/base -> origin/gh/swolchok/864/base 2025-12-04T09:23:32.8072620Z * [new branch] gh/swolchok/864/head -> origin/gh/swolchok/864/head 2025-12-04T09:23:32.8072943Z * [new branch] gh/swolchok/864/orig -> origin/gh/swolchok/864/orig 2025-12-04T09:23:32.8073100Z * [new branch] gh/swolchok/865/base -> origin/gh/swolchok/865/base 2025-12-04T09:23:32.8073251Z * [new branch] gh/swolchok/865/head -> origin/gh/swolchok/865/head 2025-12-04T09:23:32.8073397Z * [new branch] gh/swolchok/865/orig -> origin/gh/swolchok/865/orig 2025-12-04T09:23:32.8073554Z * [new branch] gh/swolchok/866/base -> origin/gh/swolchok/866/base 2025-12-04T09:23:32.8073700Z * [new branch] gh/swolchok/866/head -> origin/gh/swolchok/866/head 2025-12-04T09:23:32.8073847Z * [new branch] gh/swolchok/866/orig -> origin/gh/swolchok/866/orig 2025-12-04T09:23:32.8074006Z * [new branch] gh/swolchok/867/base -> origin/gh/swolchok/867/base 2025-12-04T09:23:32.8074157Z * [new branch] gh/swolchok/867/head -> origin/gh/swolchok/867/head 2025-12-04T09:23:32.8074301Z * [new branch] gh/swolchok/867/orig -> origin/gh/swolchok/867/orig 2025-12-04T09:23:32.8074437Z * [new branch] gh/swolchok/868/base -> origin/gh/swolchok/868/base 2025-12-04T09:23:32.8074571Z * [new branch] gh/swolchok/868/head -> origin/gh/swolchok/868/head 2025-12-04T09:23:32.8074714Z * [new branch] gh/swolchok/868/orig -> origin/gh/swolchok/868/orig 2025-12-04T09:23:32.8074851Z * [new branch] gh/swolchok/869/base -> origin/gh/swolchok/869/base 2025-12-04T09:23:32.8074989Z * [new branch] gh/swolchok/869/head -> origin/gh/swolchok/869/head 2025-12-04T09:23:32.8075123Z * [new branch] gh/swolchok/869/orig -> origin/gh/swolchok/869/orig 2025-12-04T09:23:32.8075265Z * [new branch] gh/swolchok/870/base -> origin/gh/swolchok/870/base 2025-12-04T09:23:32.8075405Z * [new branch] gh/swolchok/870/head -> origin/gh/swolchok/870/head 2025-12-04T09:23:32.8075577Z * [new branch] gh/swolchok/870/orig -> origin/gh/swolchok/870/orig 2025-12-04T09:23:32.8076812Z * [new branch] gh/swolchok/871/base -> origin/gh/swolchok/871/base 2025-12-04T09:23:32.8077026Z * [new branch] gh/swolchok/871/head -> origin/gh/swolchok/871/head 2025-12-04T09:23:32.8077516Z * [new branch] gh/swolchok/871/orig -> origin/gh/swolchok/871/orig 2025-12-04T09:23:32.8077703Z * [new branch] gh/teja-rao/4/base -> origin/gh/teja-rao/4/base 2025-12-04T09:23:32.8077840Z * [new branch] gh/teja-rao/4/head -> origin/gh/teja-rao/4/head 2025-12-04T09:23:32.8077971Z * [new branch] gh/teja-rao/4/orig -> origin/gh/teja-rao/4/orig 2025-12-04T09:23:32.8081561Z * [new branch] gh/tianyu-l/2/base -> origin/gh/tianyu-l/2/base 2025-12-04T09:23:32.8081721Z * [new branch] gh/tianyu-l/2/head -> origin/gh/tianyu-l/2/head 2025-12-04T09:23:32.8081868Z * [new branch] gh/tianyu-l/2/orig -> origin/gh/tianyu-l/2/orig 2025-12-04T09:23:32.8085033Z * [new branch] gh/tianyu-l/3/base -> origin/gh/tianyu-l/3/base 2025-12-04T09:23:32.8085192Z * [new branch] gh/tianyu-l/3/orig -> origin/gh/tianyu-l/3/orig 2025-12-04T09:23:32.8085522Z * [new branch] gh/tianyu-l/4/base -> origin/gh/tianyu-l/4/base 2025-12-04T09:23:32.8085665Z * [new branch] gh/tianyu-l/4/head -> origin/gh/tianyu-l/4/head 2025-12-04T09:23:32.8085807Z * [new branch] gh/tianyu-l/4/orig -> origin/gh/tianyu-l/4/orig 2025-12-04T09:23:32.8090644Z * [new branch] gh/tugsbayasgalan/10/base -> origin/gh/tugsbayasgalan/10/base 2025-12-04T09:23:32.8095781Z * [new branch] gh/tugsbayasgalan/10/head -> origin/gh/tugsbayasgalan/10/head 2025-12-04T09:23:32.8099529Z * [new branch] gh/tugsbayasgalan/10/orig -> origin/gh/tugsbayasgalan/10/orig 2025-12-04T09:23:32.8104501Z * [new branch] gh/tugsbayasgalan/13/base -> origin/gh/tugsbayasgalan/13/base 2025-12-04T09:23:32.8104824Z * [new branch] gh/tugsbayasgalan/13/head -> origin/gh/tugsbayasgalan/13/head 2025-12-04T09:23:32.8104998Z * [new branch] gh/tugsbayasgalan/13/orig -> origin/gh/tugsbayasgalan/13/orig 2025-12-04T09:23:32.8105241Z * [new branch] gh/tugsbayasgalan/17/base -> origin/gh/tugsbayasgalan/17/base 2025-12-04T09:23:32.8105465Z * [new branch] gh/tugsbayasgalan/17/head -> origin/gh/tugsbayasgalan/17/head 2025-12-04T09:23:32.8105616Z * [new branch] gh/tugsbayasgalan/17/orig -> origin/gh/tugsbayasgalan/17/orig 2025-12-04T09:23:32.8106111Z * [new branch] gh/tugsbayasgalan/2/base -> origin/gh/tugsbayasgalan/2/base 2025-12-04T09:23:32.8106285Z * [new branch] gh/tugsbayasgalan/2/head -> origin/gh/tugsbayasgalan/2/head 2025-12-04T09:23:32.8106430Z * [new branch] gh/tugsbayasgalan/2/orig -> origin/gh/tugsbayasgalan/2/orig 2025-12-04T09:23:32.8106605Z * [new branch] gh/tugsbayasgalan/28/base -> origin/gh/tugsbayasgalan/28/base 2025-12-04T09:23:32.8106754Z * [new branch] gh/tugsbayasgalan/28/head -> origin/gh/tugsbayasgalan/28/head 2025-12-04T09:23:32.8106906Z * [new branch] gh/tugsbayasgalan/28/orig -> origin/gh/tugsbayasgalan/28/orig 2025-12-04T09:23:32.8107064Z * [new branch] gh/tugsbayasgalan/32/base -> origin/gh/tugsbayasgalan/32/base 2025-12-04T09:23:32.8107209Z * [new branch] gh/tugsbayasgalan/32/head -> origin/gh/tugsbayasgalan/32/head 2025-12-04T09:23:32.8107368Z * [new branch] gh/tugsbayasgalan/32/orig -> origin/gh/tugsbayasgalan/32/orig 2025-12-04T09:23:32.8107514Z * [new branch] gh/tugsbayasgalan/35/base -> origin/gh/tugsbayasgalan/35/base 2025-12-04T09:23:32.8107664Z * [new branch] gh/tugsbayasgalan/35/head -> origin/gh/tugsbayasgalan/35/head 2025-12-04T09:23:32.8107819Z * [new branch] gh/tugsbayasgalan/35/orig -> origin/gh/tugsbayasgalan/35/orig 2025-12-04T09:23:32.8107964Z * [new branch] gh/tugsbayasgalan/36/base -> origin/gh/tugsbayasgalan/36/base 2025-12-04T09:23:32.8108118Z * [new branch] gh/tugsbayasgalan/36/head -> origin/gh/tugsbayasgalan/36/head 2025-12-04T09:23:32.8108271Z * [new branch] gh/tugsbayasgalan/36/orig -> origin/gh/tugsbayasgalan/36/orig 2025-12-04T09:23:32.8108415Z * [new branch] gh/tugsbayasgalan/37/base -> origin/gh/tugsbayasgalan/37/base 2025-12-04T09:23:32.8108568Z * [new branch] gh/tugsbayasgalan/37/head -> origin/gh/tugsbayasgalan/37/head 2025-12-04T09:23:32.8111006Z * [new branch] gh/tugsbayasgalan/37/orig -> origin/gh/tugsbayasgalan/37/orig 2025-12-04T09:23:32.8112247Z * [new branch] gh/tugsbayasgalan/43/base -> origin/gh/tugsbayasgalan/43/base 2025-12-04T09:23:32.8112896Z * [new branch] gh/tugsbayasgalan/43/head -> origin/gh/tugsbayasgalan/43/head 2025-12-04T09:23:32.8117667Z * [new branch] gh/tugsbayasgalan/43/orig -> origin/gh/tugsbayasgalan/43/orig 2025-12-04T09:23:32.8118027Z * [new branch] gh/tugsbayasgalan/48/base -> origin/gh/tugsbayasgalan/48/base 2025-12-04T09:23:32.8118572Z * [new branch] gh/tugsbayasgalan/48/head -> origin/gh/tugsbayasgalan/48/head 2025-12-04T09:23:32.8118838Z * [new branch] gh/tugsbayasgalan/48/orig -> origin/gh/tugsbayasgalan/48/orig 2025-12-04T09:23:32.8119444Z * [new branch] gh/tugsbayasgalan/51/base -> origin/gh/tugsbayasgalan/51/base 2025-12-04T09:23:32.8119882Z * [new branch] gh/tugsbayasgalan/51/head -> origin/gh/tugsbayasgalan/51/head 2025-12-04T09:23:32.8120061Z * [new branch] gh/tugsbayasgalan/51/orig -> origin/gh/tugsbayasgalan/51/orig 2025-12-04T09:23:32.8120228Z * [new branch] gh/tugsbayasgalan/52/base -> origin/gh/tugsbayasgalan/52/base 2025-12-04T09:23:32.8120520Z * [new branch] gh/tugsbayasgalan/52/head -> origin/gh/tugsbayasgalan/52/head 2025-12-04T09:23:32.8120927Z * [new branch] gh/tugsbayasgalan/52/orig -> origin/gh/tugsbayasgalan/52/orig 2025-12-04T09:23:32.8122328Z * [new branch] gh/tugsbayasgalan/53/base -> origin/gh/tugsbayasgalan/53/base 2025-12-04T09:23:32.8122765Z * [new branch] gh/tugsbayasgalan/53/head -> origin/gh/tugsbayasgalan/53/head 2025-12-04T09:23:32.8124834Z * [new branch] gh/tugsbayasgalan/53/orig -> origin/gh/tugsbayasgalan/53/orig 2025-12-04T09:23:32.8125053Z * [new branch] gh/tugsbayasgalan/55/base -> origin/gh/tugsbayasgalan/55/base 2025-12-04T09:23:32.8125445Z * [new branch] gh/tugsbayasgalan/55/head -> origin/gh/tugsbayasgalan/55/head 2025-12-04T09:23:32.8127063Z * [new branch] gh/tugsbayasgalan/55/orig -> origin/gh/tugsbayasgalan/55/orig 2025-12-04T09:23:32.8127277Z * [new branch] gh/tugsbayasgalan/59/base -> origin/gh/tugsbayasgalan/59/base 2025-12-04T09:23:32.8129484Z * [new branch] gh/tugsbayasgalan/59/head -> origin/gh/tugsbayasgalan/59/head 2025-12-04T09:23:32.8129830Z * [new branch] gh/tugsbayasgalan/59/orig -> origin/gh/tugsbayasgalan/59/orig 2025-12-04T09:23:32.8130045Z * [new branch] gh/tugsbayasgalan/6/base -> origin/gh/tugsbayasgalan/6/base 2025-12-04T09:23:32.8130407Z * [new branch] gh/tugsbayasgalan/6/head -> origin/gh/tugsbayasgalan/6/head 2025-12-04T09:23:32.8131743Z * [new branch] gh/tugsbayasgalan/6/orig -> origin/gh/tugsbayasgalan/6/orig 2025-12-04T09:23:32.8132032Z * [new branch] gh/tugsbayasgalan/60/base -> origin/gh/tugsbayasgalan/60/base 2025-12-04T09:23:32.8134209Z * [new branch] gh/tugsbayasgalan/60/head -> origin/gh/tugsbayasgalan/60/head 2025-12-04T09:23:32.8134550Z * [new branch] gh/tugsbayasgalan/60/orig -> origin/gh/tugsbayasgalan/60/orig 2025-12-04T09:23:32.8135930Z * [new branch] gh/tugsbayasgalan/61/base -> origin/gh/tugsbayasgalan/61/base 2025-12-04T09:23:32.8136272Z * [new branch] gh/tugsbayasgalan/61/head -> origin/gh/tugsbayasgalan/61/head 2025-12-04T09:23:32.8136650Z * [new branch] gh/tugsbayasgalan/61/orig -> origin/gh/tugsbayasgalan/61/orig 2025-12-04T09:23:32.8138880Z * [new branch] gh/tugsbayasgalan/63/base -> origin/gh/tugsbayasgalan/63/base 2025-12-04T09:23:32.8139061Z * [new branch] gh/tugsbayasgalan/63/head -> origin/gh/tugsbayasgalan/63/head 2025-12-04T09:23:32.8139373Z * [new branch] gh/tugsbayasgalan/63/orig -> origin/gh/tugsbayasgalan/63/orig 2025-12-04T09:23:32.8140411Z * [new branch] gh/tugsbayasgalan/67/base -> origin/gh/tugsbayasgalan/67/base 2025-12-04T09:23:32.8140872Z * [new branch] gh/tugsbayasgalan/67/head -> origin/gh/tugsbayasgalan/67/head 2025-12-04T09:23:32.8143909Z * [new branch] gh/tugsbayasgalan/67/orig -> origin/gh/tugsbayasgalan/67/orig 2025-12-04T09:23:32.8144091Z * [new branch] gh/tugsbayasgalan/68/base -> origin/gh/tugsbayasgalan/68/base 2025-12-04T09:23:32.8144251Z * [new branch] gh/tugsbayasgalan/68/head -> origin/gh/tugsbayasgalan/68/head 2025-12-04T09:23:32.8144760Z * [new branch] gh/tugsbayasgalan/68/orig -> origin/gh/tugsbayasgalan/68/orig 2025-12-04T09:23:32.8146289Z * [new branch] gh/tugsbayasgalan/7/base -> origin/gh/tugsbayasgalan/7/base 2025-12-04T09:23:32.8146502Z * [new branch] gh/tugsbayasgalan/7/head -> origin/gh/tugsbayasgalan/7/head 2025-12-04T09:23:32.8147380Z * [new branch] gh/tugsbayasgalan/7/orig -> origin/gh/tugsbayasgalan/7/orig 2025-12-04T09:23:32.8148645Z * [new branch] gh/tugsbayasgalan/70/base -> origin/gh/tugsbayasgalan/70/base 2025-12-04T09:23:32.8149297Z * [new branch] gh/tugsbayasgalan/70/head -> origin/gh/tugsbayasgalan/70/head 2025-12-04T09:23:32.8150035Z * [new branch] gh/tugsbayasgalan/70/orig -> origin/gh/tugsbayasgalan/70/orig 2025-12-04T09:23:32.8151651Z * [new branch] gh/tugsbayasgalan/71/base -> origin/gh/tugsbayasgalan/71/base 2025-12-04T09:23:32.8151902Z * [new branch] gh/tugsbayasgalan/71/head -> origin/gh/tugsbayasgalan/71/head 2025-12-04T09:23:32.8154316Z * [new branch] gh/tugsbayasgalan/71/orig -> origin/gh/tugsbayasgalan/71/orig 2025-12-04T09:23:32.8154645Z * [new branch] gh/tugsbayasgalan/72/base -> origin/gh/tugsbayasgalan/72/base 2025-12-04T09:23:32.8154873Z * [new branch] gh/tugsbayasgalan/72/head -> origin/gh/tugsbayasgalan/72/head 2025-12-04T09:23:32.8155300Z * [new branch] gh/tugsbayasgalan/72/orig -> origin/gh/tugsbayasgalan/72/orig 2025-12-04T09:23:32.8156888Z * [new branch] gh/tugsbayasgalan/73/base -> origin/gh/tugsbayasgalan/73/base 2025-12-04T09:23:32.8157071Z * [new branch] gh/tugsbayasgalan/73/head -> origin/gh/tugsbayasgalan/73/head 2025-12-04T09:23:32.8159192Z * [new branch] gh/tugsbayasgalan/73/orig -> origin/gh/tugsbayasgalan/73/orig 2025-12-04T09:23:32.8159530Z * [new branch] gh/tugsbayasgalan/74/base -> origin/gh/tugsbayasgalan/74/base 2025-12-04T09:23:32.8159728Z * [new branch] gh/tugsbayasgalan/74/head -> origin/gh/tugsbayasgalan/74/head 2025-12-04T09:23:32.8160208Z * [new branch] gh/tugsbayasgalan/74/orig -> origin/gh/tugsbayasgalan/74/orig 2025-12-04T09:23:32.8164650Z * [new branch] gh/tugsbayasgalan/75/base -> origin/gh/tugsbayasgalan/75/base 2025-12-04T09:23:32.8164972Z * [new branch] gh/tugsbayasgalan/75/head -> origin/gh/tugsbayasgalan/75/head 2025-12-04T09:23:32.8165147Z * [new branch] gh/tugsbayasgalan/75/orig -> origin/gh/tugsbayasgalan/75/orig 2025-12-04T09:23:32.8165377Z * [new branch] gh/tugsbayasgalan/76/base -> origin/gh/tugsbayasgalan/76/base 2025-12-04T09:23:32.8165609Z * [new branch] gh/tugsbayasgalan/76/head -> origin/gh/tugsbayasgalan/76/head 2025-12-04T09:23:32.8165862Z * [new branch] gh/tugsbayasgalan/76/orig -> origin/gh/tugsbayasgalan/76/orig 2025-12-04T09:23:32.8169930Z * [new branch] gh/tugsbayasgalan/77/base -> origin/gh/tugsbayasgalan/77/base 2025-12-04T09:23:32.8170258Z * [new branch] gh/tugsbayasgalan/77/head -> origin/gh/tugsbayasgalan/77/head 2025-12-04T09:23:32.8170476Z * [new branch] gh/tugsbayasgalan/77/orig -> origin/gh/tugsbayasgalan/77/orig 2025-12-04T09:23:32.8170665Z * [new branch] gh/tugsbayasgalan/78/base -> origin/gh/tugsbayasgalan/78/base 2025-12-04T09:23:32.8170850Z * [new branch] gh/tugsbayasgalan/78/head -> origin/gh/tugsbayasgalan/78/head 2025-12-04T09:23:32.8171023Z * [new branch] gh/tugsbayasgalan/78/orig -> origin/gh/tugsbayasgalan/78/orig 2025-12-04T09:23:32.8172981Z * [new branch] gh/tugsbayasgalan/79/base -> origin/gh/tugsbayasgalan/79/base 2025-12-04T09:23:32.8173369Z * [new branch] gh/tugsbayasgalan/79/head -> origin/gh/tugsbayasgalan/79/head 2025-12-04T09:23:32.8173883Z * [new branch] gh/tugsbayasgalan/79/orig -> origin/gh/tugsbayasgalan/79/orig 2025-12-04T09:23:32.8175253Z * [new branch] gh/tugsbayasgalan/8/base -> origin/gh/tugsbayasgalan/8/base 2025-12-04T09:23:32.8175592Z * [new branch] gh/tugsbayasgalan/8/head -> origin/gh/tugsbayasgalan/8/head 2025-12-04T09:23:32.8176106Z * [new branch] gh/tugsbayasgalan/8/orig -> origin/gh/tugsbayasgalan/8/orig 2025-12-04T09:23:32.8178775Z * [new branch] gh/tugsbayasgalan/80/base -> origin/gh/tugsbayasgalan/80/base 2025-12-04T09:23:32.8179133Z * [new branch] gh/tugsbayasgalan/80/head -> origin/gh/tugsbayasgalan/80/head 2025-12-04T09:23:32.8179348Z * [new branch] gh/tugsbayasgalan/80/orig -> origin/gh/tugsbayasgalan/80/orig 2025-12-04T09:23:32.8179868Z * [new branch] gh/tugsbayasgalan/81/base -> origin/gh/tugsbayasgalan/81/base 2025-12-04T09:23:32.8180381Z * [new branch] gh/tugsbayasgalan/81/head -> origin/gh/tugsbayasgalan/81/head 2025-12-04T09:23:32.8180573Z * [new branch] gh/tugsbayasgalan/81/orig -> origin/gh/tugsbayasgalan/81/orig 2025-12-04T09:23:32.8184765Z * [new branch] gh/tugsbayasgalan/82/base -> origin/gh/tugsbayasgalan/82/base 2025-12-04T09:23:32.8187191Z * [new branch] gh/tugsbayasgalan/82/head -> origin/gh/tugsbayasgalan/82/head 2025-12-04T09:23:32.8187506Z * [new branch] gh/tugsbayasgalan/82/orig -> origin/gh/tugsbayasgalan/82/orig 2025-12-04T09:23:32.8193536Z * [new branch] gh/tugsbayasgalan/83/base -> origin/gh/tugsbayasgalan/83/base 2025-12-04T09:23:32.8193729Z * [new branch] gh/tugsbayasgalan/83/head -> origin/gh/tugsbayasgalan/83/head 2025-12-04T09:23:32.8193888Z * [new branch] gh/tugsbayasgalan/83/orig -> origin/gh/tugsbayasgalan/83/orig 2025-12-04T09:23:32.8194039Z * [new branch] gh/tugsbayasgalan/84/base -> origin/gh/tugsbayasgalan/84/base 2025-12-04T09:23:32.8194204Z * [new branch] gh/tugsbayasgalan/84/head -> origin/gh/tugsbayasgalan/84/head 2025-12-04T09:23:32.8194364Z * [new branch] gh/tugsbayasgalan/84/orig -> origin/gh/tugsbayasgalan/84/orig 2025-12-04T09:23:32.8194516Z * [new branch] gh/tugsbayasgalan/85/base -> origin/gh/tugsbayasgalan/85/base 2025-12-04T09:23:32.8194693Z * [new branch] gh/tugsbayasgalan/85/head -> origin/gh/tugsbayasgalan/85/head 2025-12-04T09:23:32.8194840Z * [new branch] gh/tugsbayasgalan/85/orig -> origin/gh/tugsbayasgalan/85/orig 2025-12-04T09:23:32.8194984Z * [new branch] gh/tugsbayasgalan/86/base -> origin/gh/tugsbayasgalan/86/base 2025-12-04T09:23:32.8195139Z * [new branch] gh/tugsbayasgalan/86/head -> origin/gh/tugsbayasgalan/86/head 2025-12-04T09:23:32.8195284Z * [new branch] gh/tugsbayasgalan/86/orig -> origin/gh/tugsbayasgalan/86/orig 2025-12-04T09:23:32.8197392Z * [new branch] gh/tugsbayasgalan/87/base -> origin/gh/tugsbayasgalan/87/base 2025-12-04T09:23:32.8197722Z * [new branch] gh/tugsbayasgalan/87/head -> origin/gh/tugsbayasgalan/87/head 2025-12-04T09:23:32.8198060Z * [new branch] gh/tugsbayasgalan/87/orig -> origin/gh/tugsbayasgalan/87/orig 2025-12-04T09:23:32.8198229Z * [new branch] gh/tugsbayasgalan/88/base -> origin/gh/tugsbayasgalan/88/base 2025-12-04T09:23:32.8198379Z * [new branch] gh/tugsbayasgalan/88/head -> origin/gh/tugsbayasgalan/88/head 2025-12-04T09:23:32.8198528Z * [new branch] gh/tugsbayasgalan/88/orig -> origin/gh/tugsbayasgalan/88/orig 2025-12-04T09:23:32.8198684Z * [new branch] gh/tugsbayasgalan/89/base -> origin/gh/tugsbayasgalan/89/base 2025-12-04T09:23:32.8200904Z * [new branch] gh/tugsbayasgalan/89/head -> origin/gh/tugsbayasgalan/89/head 2025-12-04T09:23:32.8201264Z * [new branch] gh/tugsbayasgalan/89/orig -> origin/gh/tugsbayasgalan/89/orig 2025-12-04T09:23:32.8201608Z * [new branch] gh/tugsbayasgalan/9/base -> origin/gh/tugsbayasgalan/9/base 2025-12-04T09:23:32.8201760Z * [new branch] gh/tugsbayasgalan/9/head -> origin/gh/tugsbayasgalan/9/head 2025-12-04T09:23:32.8201918Z * [new branch] gh/tugsbayasgalan/9/orig -> origin/gh/tugsbayasgalan/9/orig 2025-12-04T09:23:32.8207109Z * [new branch] gh/tugsbayasgalan/90/base -> origin/gh/tugsbayasgalan/90/base 2025-12-04T09:23:32.8207425Z * [new branch] gh/tugsbayasgalan/90/head -> origin/gh/tugsbayasgalan/90/head 2025-12-04T09:23:32.8207589Z * [new branch] gh/tugsbayasgalan/90/orig -> origin/gh/tugsbayasgalan/90/orig 2025-12-04T09:23:32.8207746Z * [new branch] gh/tugsbayasgalan/91/base -> origin/gh/tugsbayasgalan/91/base 2025-12-04T09:23:32.8207897Z * [new branch] gh/tugsbayasgalan/91/head -> origin/gh/tugsbayasgalan/91/head 2025-12-04T09:23:32.8208062Z * [new branch] gh/tugsbayasgalan/91/orig -> origin/gh/tugsbayasgalan/91/orig 2025-12-04T09:23:32.8209655Z * [new branch] gh/tugsbayasgalan/92/base -> origin/gh/tugsbayasgalan/92/base 2025-12-04T09:23:32.8209842Z * [new branch] gh/tugsbayasgalan/92/head -> origin/gh/tugsbayasgalan/92/head 2025-12-04T09:23:32.8210016Z * [new branch] gh/tugsbayasgalan/92/orig -> origin/gh/tugsbayasgalan/92/orig 2025-12-04T09:23:32.8210164Z * [new branch] gh/tugsbayasgalan/93/base -> origin/gh/tugsbayasgalan/93/base 2025-12-04T09:23:32.8210440Z * [new branch] gh/tugsbayasgalan/93/head -> origin/gh/tugsbayasgalan/93/head 2025-12-04T09:23:32.8210643Z * [new branch] gh/tugsbayasgalan/93/orig -> origin/gh/tugsbayasgalan/93/orig 2025-12-04T09:23:32.8216449Z * [new branch] gh/v0i0/14/base -> origin/gh/v0i0/14/base 2025-12-04T09:23:32.8216770Z * [new branch] gh/v0i0/14/head -> origin/gh/v0i0/14/head 2025-12-04T09:23:32.8216948Z * [new branch] gh/v0i0/14/orig -> origin/gh/v0i0/14/orig 2025-12-04T09:23:32.8217070Z * [new branch] gh/v0i0/15/base -> origin/gh/v0i0/15/base 2025-12-04T09:23:32.8217186Z * [new branch] gh/v0i0/15/head -> origin/gh/v0i0/15/head 2025-12-04T09:23:32.8217445Z * [new branch] gh/v0i0/15/orig -> origin/gh/v0i0/15/orig 2025-12-04T09:23:32.8221732Z * [new branch] gh/v0i0/16/base -> origin/gh/v0i0/16/base 2025-12-04T09:23:32.8222059Z * [new branch] gh/v0i0/16/head -> origin/gh/v0i0/16/head 2025-12-04T09:23:32.8222209Z * [new branch] gh/v0i0/16/orig -> origin/gh/v0i0/16/orig 2025-12-04T09:23:32.8222328Z * [new branch] gh/v0i0/17/base -> origin/gh/v0i0/17/base 2025-12-04T09:23:32.8222455Z * [new branch] gh/v0i0/17/head -> origin/gh/v0i0/17/head 2025-12-04T09:23:32.8222588Z * [new branch] gh/v0i0/17/orig -> origin/gh/v0i0/17/orig 2025-12-04T09:23:32.8222851Z * [new branch] gh/v0i0/18/base -> origin/gh/v0i0/18/base 2025-12-04T09:23:32.8223109Z * [new branch] gh/v0i0/18/head -> origin/gh/v0i0/18/head 2025-12-04T09:23:32.8223254Z * [new branch] gh/v0i0/18/orig -> origin/gh/v0i0/18/orig 2025-12-04T09:23:32.8223374Z * [new branch] gh/v0i0/19/base -> origin/gh/v0i0/19/base 2025-12-04T09:23:32.8223492Z * [new branch] gh/v0i0/19/head -> origin/gh/v0i0/19/head 2025-12-04T09:23:32.8223622Z * [new branch] gh/v0i0/19/orig -> origin/gh/v0i0/19/orig 2025-12-04T09:23:32.8228388Z * [new branch] gh/vishal9-team/1/base -> origin/gh/vishal9-team/1/base 2025-12-04T09:23:32.8228590Z * [new branch] gh/vishal9-team/1/head -> origin/gh/vishal9-team/1/head 2025-12-04T09:23:32.8228980Z * [new branch] gh/vishal9-team/2/base -> origin/gh/vishal9-team/2/base 2025-12-04T09:23:32.8229140Z * [new branch] gh/vishal9-team/2/head -> origin/gh/vishal9-team/2/head 2025-12-04T09:23:32.8229299Z * [new branch] gh/vishal9-team/2/orig -> origin/gh/vishal9-team/2/orig 2025-12-04T09:23:32.8229513Z * [new branch] gh/vishal9-team/3/base -> origin/gh/vishal9-team/3/base 2025-12-04T09:23:32.8229681Z * [new branch] gh/vishal9-team/3/head -> origin/gh/vishal9-team/3/head 2025-12-04T09:23:32.8230015Z * [new branch] gh/vishal9-team/3/orig -> origin/gh/vishal9-team/3/orig 2025-12-04T09:23:32.8230331Z * [new branch] gh/vishal9-team/4/base -> origin/gh/vishal9-team/4/base 2025-12-04T09:23:32.8230763Z * [new branch] gh/vishal9-team/4/head -> origin/gh/vishal9-team/4/head 2025-12-04T09:23:32.8231098Z * [new branch] gh/vishal9-team/4/orig -> origin/gh/vishal9-team/4/orig 2025-12-04T09:23:32.8232504Z * [new branch] gh/vkuzo/1/next -> origin/gh/vkuzo/1/next 2025-12-04T09:23:32.8233096Z * [new branch] gh/vkuzo/2/next -> origin/gh/vkuzo/2/next 2025-12-04T09:23:32.8237844Z * [new branch] gh/vkuzo/3/next -> origin/gh/vkuzo/3/next 2025-12-04T09:23:32.8240211Z * [new branch] gh/wconstab/424/base -> origin/gh/wconstab/424/base 2025-12-04T09:23:32.8240542Z * [new branch] gh/wconstab/424/head -> origin/gh/wconstab/424/head 2025-12-04T09:23:32.8240735Z * [new branch] gh/wconstab/424/orig -> origin/gh/wconstab/424/orig 2025-12-04T09:23:32.8240962Z * [new branch] gh/wconstab/435/base -> origin/gh/wconstab/435/base 2025-12-04T09:23:32.8241141Z * [new branch] gh/wconstab/435/head -> origin/gh/wconstab/435/head 2025-12-04T09:23:32.8241306Z * [new branch] gh/wconstab/435/orig -> origin/gh/wconstab/435/orig 2025-12-04T09:23:32.8241538Z * [new branch] gh/wconstab/444/base -> origin/gh/wconstab/444/base 2025-12-04T09:23:32.8242587Z * [new branch] gh/wconstab/444/head -> origin/gh/wconstab/444/head 2025-12-04T09:23:32.8242888Z * [new branch] gh/wconstab/444/orig -> origin/gh/wconstab/444/orig 2025-12-04T09:23:32.8245610Z * [new branch] gh/wconstab/447/base -> origin/gh/wconstab/447/base 2025-12-04T09:23:32.8245930Z * [new branch] gh/wconstab/447/head -> origin/gh/wconstab/447/head 2025-12-04T09:23:32.8246094Z * [new branch] gh/wconstab/447/orig -> origin/gh/wconstab/447/orig 2025-12-04T09:23:32.8246263Z * [new branch] gh/wconstab/448/base -> origin/gh/wconstab/448/base 2025-12-04T09:23:32.8246996Z * [new branch] gh/wconstab/448/head -> origin/gh/wconstab/448/head 2025-12-04T09:23:32.8250195Z * [new branch] gh/wconstab/448/orig -> origin/gh/wconstab/448/orig 2025-12-04T09:23:32.8250509Z * [new branch] gh/wconstab/449/base -> origin/gh/wconstab/449/base 2025-12-04T09:23:32.8250678Z * [new branch] gh/wconstab/449/head -> origin/gh/wconstab/449/head 2025-12-04T09:23:32.8250874Z * [new branch] gh/wconstab/449/orig -> origin/gh/wconstab/449/orig 2025-12-04T09:23:32.8251001Z * [new branch] gh/wconstab/450/base -> origin/gh/wconstab/450/base 2025-12-04T09:23:32.8251209Z * [new branch] gh/wconstab/450/head -> origin/gh/wconstab/450/head 2025-12-04T09:23:32.8252079Z * [new branch] gh/wconstab/450/orig -> origin/gh/wconstab/450/orig 2025-12-04T09:23:32.8252659Z * [new branch] gh/wconstab/451/base -> origin/gh/wconstab/451/base 2025-12-04T09:23:32.8255568Z * [new branch] gh/wconstab/451/head -> origin/gh/wconstab/451/head 2025-12-04T09:23:32.8255906Z * [new branch] gh/wconstab/451/orig -> origin/gh/wconstab/451/orig 2025-12-04T09:23:32.8256054Z * [new branch] gh/wconstab/452/base -> origin/gh/wconstab/452/base 2025-12-04T09:23:32.8256184Z * [new branch] gh/wconstab/452/head -> origin/gh/wconstab/452/head 2025-12-04T09:23:32.8256399Z * [new branch] gh/wconstab/452/orig -> origin/gh/wconstab/452/orig 2025-12-04T09:23:32.8258370Z * [new branch] gh/wconstab/453/base -> origin/gh/wconstab/453/base 2025-12-04T09:23:32.8258559Z * [new branch] gh/wconstab/453/head -> origin/gh/wconstab/453/head 2025-12-04T09:23:32.8258697Z * [new branch] gh/wconstab/453/orig -> origin/gh/wconstab/453/orig 2025-12-04T09:23:32.8259223Z * [new branch] gh/wconstab/454/base -> origin/gh/wconstab/454/base 2025-12-04T09:23:32.8262087Z * [new branch] gh/wconstab/454/head -> origin/gh/wconstab/454/head 2025-12-04T09:23:32.8262439Z * [new branch] gh/wconstab/454/orig -> origin/gh/wconstab/454/orig 2025-12-04T09:23:32.8262593Z * [new branch] gh/wconstab/455/base -> origin/gh/wconstab/455/base 2025-12-04T09:23:32.8262751Z * [new branch] gh/wconstab/455/head -> origin/gh/wconstab/455/head 2025-12-04T09:23:32.8263118Z * [new branch] gh/wconstab/455/orig -> origin/gh/wconstab/455/orig 2025-12-04T09:23:32.8267610Z * [new branch] gh/wconstab/456/base -> origin/gh/wconstab/456/base 2025-12-04T09:23:32.8267918Z * [new branch] gh/wconstab/456/head -> origin/gh/wconstab/456/head 2025-12-04T09:23:32.8268087Z * [new branch] gh/wconstab/456/orig -> origin/gh/wconstab/456/orig 2025-12-04T09:23:32.8268256Z * [new branch] gh/wconstab/457/base -> origin/gh/wconstab/457/base 2025-12-04T09:23:32.8268402Z * [new branch] gh/wconstab/457/head -> origin/gh/wconstab/457/head 2025-12-04T09:23:32.8268565Z * [new branch] gh/wconstab/457/orig -> origin/gh/wconstab/457/orig 2025-12-04T09:23:32.8269244Z * [new branch] gh/wconstab/458/base -> origin/gh/wconstab/458/base 2025-12-04T09:23:32.8272111Z * [new branch] gh/wconstab/458/head -> origin/gh/wconstab/458/head 2025-12-04T09:23:32.8272299Z * [new branch] gh/wconstab/458/orig -> origin/gh/wconstab/458/orig 2025-12-04T09:23:32.8272461Z * [new branch] gh/wconstab/459/base -> origin/gh/wconstab/459/base 2025-12-04T09:23:32.8272603Z * [new branch] gh/wconstab/459/head -> origin/gh/wconstab/459/head 2025-12-04T09:23:32.8272964Z * [new branch] gh/wconstab/459/orig -> origin/gh/wconstab/459/orig 2025-12-04T09:23:32.8277399Z * [new branch] gh/wconstab/460/base -> origin/gh/wconstab/460/base 2025-12-04T09:23:32.8277587Z * [new branch] gh/wconstab/460/head -> origin/gh/wconstab/460/head 2025-12-04T09:23:32.8277735Z * [new branch] gh/wconstab/460/orig -> origin/gh/wconstab/460/orig 2025-12-04T09:23:32.8277869Z * [new branch] gh/wconstab/461/base -> origin/gh/wconstab/461/base 2025-12-04T09:23:32.8278005Z * [new branch] gh/wconstab/461/head -> origin/gh/wconstab/461/head 2025-12-04T09:23:32.8278183Z * [new branch] gh/wconstab/461/orig -> origin/gh/wconstab/461/orig 2025-12-04T09:23:32.8279535Z * [new branch] gh/wconstab/462/base -> origin/gh/wconstab/462/base 2025-12-04T09:23:32.8279845Z * [new branch] gh/wconstab/462/head -> origin/gh/wconstab/462/head 2025-12-04T09:23:32.8280369Z * [new branch] gh/wconstab/462/orig -> origin/gh/wconstab/462/orig 2025-12-04T09:23:32.8282267Z * [new branch] gh/wconstab/463/base -> origin/gh/wconstab/463/base 2025-12-04T09:23:32.8282751Z * [new branch] gh/wconstab/463/head -> origin/gh/wconstab/463/head 2025-12-04T09:23:32.8282914Z * [new branch] gh/wconstab/463/orig -> origin/gh/wconstab/463/orig 2025-12-04T09:23:32.8285195Z * [new branch] gh/wconstab/464/base -> origin/gh/wconstab/464/base 2025-12-04T09:23:32.8285668Z * [new branch] gh/wconstab/464/head -> origin/gh/wconstab/464/head 2025-12-04T09:23:32.8285927Z * [new branch] gh/wconstab/464/orig -> origin/gh/wconstab/464/orig 2025-12-04T09:23:32.8286177Z * [new branch] gh/wconstab/465/base -> origin/gh/wconstab/465/base 2025-12-04T09:23:32.8289066Z * [new branch] gh/wconstab/465/head -> origin/gh/wconstab/465/head 2025-12-04T09:23:32.8293745Z * [new branch] gh/wconstab/465/orig -> origin/gh/wconstab/465/orig 2025-12-04T09:23:32.8299006Z * [new branch] gh/wconstab/466/base -> origin/gh/wconstab/466/base 2025-12-04T09:23:32.8303796Z * [new branch] gh/wconstab/466/head -> origin/gh/wconstab/466/head 2025-12-04T09:23:32.8306048Z * [new branch] gh/wconstab/466/orig -> origin/gh/wconstab/466/orig 2025-12-04T09:23:32.8306332Z * [new branch] gh/wconstab/467/base -> origin/gh/wconstab/467/base 2025-12-04T09:23:32.8306564Z * [new branch] gh/wconstab/467/head -> origin/gh/wconstab/467/head 2025-12-04T09:23:32.8306707Z * [new branch] gh/wconstab/467/orig -> origin/gh/wconstab/467/orig 2025-12-04T09:23:32.8306860Z * [new branch] gh/wconstab/468/base -> origin/gh/wconstab/468/base 2025-12-04T09:23:32.8307002Z * [new branch] gh/wconstab/468/head -> origin/gh/wconstab/468/head 2025-12-04T09:23:32.8307153Z * [new branch] gh/wconstab/468/orig -> origin/gh/wconstab/468/orig 2025-12-04T09:23:32.8307307Z * [new branch] gh/weifengpy/39/base -> origin/gh/weifengpy/39/base 2025-12-04T09:23:32.8307462Z * [new branch] gh/weifengpy/39/head -> origin/gh/weifengpy/39/head 2025-12-04T09:23:32.8308100Z * [new branch] gh/weifengpy/39/orig -> origin/gh/weifengpy/39/orig 2025-12-04T09:23:32.8308297Z * [new branch] gh/weifengpy/40/base -> origin/gh/weifengpy/40/base 2025-12-04T09:23:32.8308597Z * [new branch] gh/weifengpy/40/head -> origin/gh/weifengpy/40/head 2025-12-04T09:23:32.8308763Z * [new branch] gh/weifengpy/40/orig -> origin/gh/weifengpy/40/orig 2025-12-04T09:23:32.8308914Z * [new branch] gh/weifengpy/41/base -> origin/gh/weifengpy/41/base 2025-12-04T09:23:32.8309055Z * [new branch] gh/weifengpy/41/head -> origin/gh/weifengpy/41/head 2025-12-04T09:23:32.8309201Z * [new branch] gh/weifengpy/41/orig -> origin/gh/weifengpy/41/orig 2025-12-04T09:23:32.8309390Z * [new branch] gh/williamwen42/250/base -> origin/gh/williamwen42/250/base 2025-12-04T09:23:32.8309543Z * [new branch] gh/williamwen42/250/head -> origin/gh/williamwen42/250/head 2025-12-04T09:23:32.8309701Z * [new branch] gh/williamwen42/250/orig -> origin/gh/williamwen42/250/orig 2025-12-04T09:23:32.8309850Z * [new branch] gh/williamwen42/279/base -> origin/gh/williamwen42/279/base 2025-12-04T09:23:32.8310001Z * [new branch] gh/williamwen42/279/head -> origin/gh/williamwen42/279/head 2025-12-04T09:23:32.8310146Z * [new branch] gh/williamwen42/279/orig -> origin/gh/williamwen42/279/orig 2025-12-04T09:23:32.8310292Z * [new branch] gh/williamwen42/282/base -> origin/gh/williamwen42/282/base 2025-12-04T09:23:32.8310474Z * [new branch] gh/williamwen42/282/head -> origin/gh/williamwen42/282/head 2025-12-04T09:23:32.8310619Z * [new branch] gh/williamwen42/282/orig -> origin/gh/williamwen42/282/orig 2025-12-04T09:23:32.8311448Z * [new branch] gh/williamwen42/287/base -> origin/gh/williamwen42/287/base 2025-12-04T09:23:32.8311633Z * [new branch] gh/williamwen42/287/head -> origin/gh/williamwen42/287/head 2025-12-04T09:23:32.8312759Z * [new branch] gh/williamwen42/287/orig -> origin/gh/williamwen42/287/orig 2025-12-04T09:23:32.8313519Z * [new branch] gh/williamwen42/288/base -> origin/gh/williamwen42/288/base 2025-12-04T09:23:32.8314136Z * [new branch] gh/williamwen42/288/head -> origin/gh/williamwen42/288/head 2025-12-04T09:23:32.8317885Z * [new branch] gh/williamwen42/288/orig -> origin/gh/williamwen42/288/orig 2025-12-04T09:23:32.8318241Z * [new branch] gh/williamwen42/296/base -> origin/gh/williamwen42/296/base 2025-12-04T09:23:32.8318414Z * [new branch] gh/williamwen42/296/head -> origin/gh/williamwen42/296/head 2025-12-04T09:23:32.8318576Z * [new branch] gh/williamwen42/296/orig -> origin/gh/williamwen42/296/orig 2025-12-04T09:23:32.8318855Z * [new branch] gh/williamwen42/297/base -> origin/gh/williamwen42/297/base 2025-12-04T09:23:32.8319114Z * [new branch] gh/williamwen42/297/head -> origin/gh/williamwen42/297/head 2025-12-04T09:23:32.8321508Z * [new branch] gh/williamwen42/297/orig -> origin/gh/williamwen42/297/orig 2025-12-04T09:23:32.8321842Z * [new branch] gh/williamwen42/306/base -> origin/gh/williamwen42/306/base 2025-12-04T09:23:32.8322002Z * [new branch] gh/williamwen42/306/head -> origin/gh/williamwen42/306/head 2025-12-04T09:23:32.8322338Z * [new branch] gh/williamwen42/306/orig -> origin/gh/williamwen42/306/orig 2025-12-04T09:23:32.8327197Z * [new branch] gh/williamwen42/309/base -> origin/gh/williamwen42/309/base 2025-12-04T09:23:32.8327393Z * [new branch] gh/williamwen42/309/head -> origin/gh/williamwen42/309/head 2025-12-04T09:23:32.8327563Z * [new branch] gh/williamwen42/309/orig -> origin/gh/williamwen42/309/orig 2025-12-04T09:23:32.8327726Z * [new branch] gh/williamwen42/310/base -> origin/gh/williamwen42/310/base 2025-12-04T09:23:32.8327877Z * [new branch] gh/williamwen42/310/head -> origin/gh/williamwen42/310/head 2025-12-04T09:23:32.8328048Z * [new branch] gh/williamwen42/310/orig -> origin/gh/williamwen42/310/orig 2025-12-04T09:23:32.8329398Z * [new branch] gh/williamwen42/311/base -> origin/gh/williamwen42/311/base 2025-12-04T09:23:32.8329697Z * [new branch] gh/williamwen42/311/head -> origin/gh/williamwen42/311/head 2025-12-04T09:23:32.8329982Z * [new branch] gh/williamwen42/311/orig -> origin/gh/williamwen42/311/orig 2025-12-04T09:23:32.8331904Z * [new branch] gh/williamwen42/319/base -> origin/gh/williamwen42/319/base 2025-12-04T09:23:32.8332243Z * [new branch] gh/williamwen42/319/head -> origin/gh/williamwen42/319/head 2025-12-04T09:23:32.8332417Z * [new branch] gh/williamwen42/319/orig -> origin/gh/williamwen42/319/orig 2025-12-04T09:23:32.8334276Z * [new branch] gh/williamwen42/325/base -> origin/gh/williamwen42/325/base 2025-12-04T09:23:32.8334596Z * [new branch] gh/williamwen42/325/head -> origin/gh/williamwen42/325/head 2025-12-04T09:23:32.8334765Z * [new branch] gh/williamwen42/325/orig -> origin/gh/williamwen42/325/orig 2025-12-04T09:23:32.8336306Z * [new branch] gh/williamwen42/326/base -> origin/gh/williamwen42/326/base 2025-12-04T09:23:32.8336639Z * [new branch] gh/williamwen42/326/head -> origin/gh/williamwen42/326/head 2025-12-04T09:23:32.8336799Z * [new branch] gh/williamwen42/326/orig -> origin/gh/williamwen42/326/orig 2025-12-04T09:23:32.8338685Z * [new branch] gh/williamwen42/327/base -> origin/gh/williamwen42/327/base 2025-12-04T09:23:32.8339182Z * [new branch] gh/williamwen42/327/head -> origin/gh/williamwen42/327/head 2025-12-04T09:23:32.8339339Z * [new branch] gh/williamwen42/327/orig -> origin/gh/williamwen42/327/orig 2025-12-04T09:23:32.8341135Z * [new branch] gh/williamwen42/328/base -> origin/gh/williamwen42/328/base 2025-12-04T09:23:32.8341608Z * [new branch] gh/williamwen42/328/head -> origin/gh/williamwen42/328/head 2025-12-04T09:23:32.8341839Z * [new branch] gh/williamwen42/328/orig -> origin/gh/williamwen42/328/orig 2025-12-04T09:23:32.8343610Z * [new branch] gh/williamwen42/329/base -> origin/gh/williamwen42/329/base 2025-12-04T09:23:32.8343953Z * [new branch] gh/williamwen42/329/head -> origin/gh/williamwen42/329/head 2025-12-04T09:23:32.8344118Z * [new branch] gh/williamwen42/329/orig -> origin/gh/williamwen42/329/orig 2025-12-04T09:23:32.8346605Z * [new branch] gh/williamwen42/330/base -> origin/gh/williamwen42/330/base 2025-12-04T09:23:32.8346787Z * [new branch] gh/williamwen42/330/head -> origin/gh/williamwen42/330/head 2025-12-04T09:23:32.8346940Z * [new branch] gh/williamwen42/330/orig -> origin/gh/williamwen42/330/orig 2025-12-04T09:23:32.8347716Z * [new branch] gh/williamwen42/331/base -> origin/gh/williamwen42/331/base 2025-12-04T09:23:32.8348594Z * [new branch] gh/williamwen42/331/head -> origin/gh/williamwen42/331/head 2025-12-04T09:23:32.8348876Z * [new branch] gh/williamwen42/331/orig -> origin/gh/williamwen42/331/orig 2025-12-04T09:23:32.8350109Z * [new branch] gh/williamwen42/332/base -> origin/gh/williamwen42/332/base 2025-12-04T09:23:32.8350398Z * [new branch] gh/williamwen42/332/head -> origin/gh/williamwen42/332/head 2025-12-04T09:23:32.8351413Z * [new branch] gh/williamwen42/332/orig -> origin/gh/williamwen42/332/orig 2025-12-04T09:23:32.8352804Z * [new branch] gh/williamwen42/333/base -> origin/gh/williamwen42/333/base 2025-12-04T09:23:32.8353938Z * [new branch] gh/williamwen42/333/head -> origin/gh/williamwen42/333/head 2025-12-04T09:23:32.8354356Z * [new branch] gh/williamwen42/333/orig -> origin/gh/williamwen42/333/orig 2025-12-04T09:23:32.8355635Z * [new branch] gh/williamwen42/334/base -> origin/gh/williamwen42/334/base 2025-12-04T09:23:32.8356720Z * [new branch] gh/williamwen42/334/head -> origin/gh/williamwen42/334/head 2025-12-04T09:23:32.8357084Z * [new branch] gh/williamwen42/334/orig -> origin/gh/williamwen42/334/orig 2025-12-04T09:23:32.8364876Z * [new branch] gh/williamwen42/335/base -> origin/gh/williamwen42/335/base 2025-12-04T09:23:32.8365333Z * [new branch] gh/williamwen42/335/head -> origin/gh/williamwen42/335/head 2025-12-04T09:23:32.8365518Z * [new branch] gh/williamwen42/335/orig -> origin/gh/williamwen42/335/orig 2025-12-04T09:23:32.8365697Z * [new branch] gh/williamwen42/336/base -> origin/gh/williamwen42/336/base 2025-12-04T09:23:32.8365856Z * [new branch] gh/williamwen42/336/head -> origin/gh/williamwen42/336/head 2025-12-04T09:23:32.8366040Z * [new branch] gh/williamwen42/336/orig -> origin/gh/williamwen42/336/orig 2025-12-04T09:23:32.8366293Z * [new branch] gh/williamwen42/337/base -> origin/gh/williamwen42/337/base 2025-12-04T09:23:32.8371112Z * [new branch] gh/williamwen42/337/head -> origin/gh/williamwen42/337/head 2025-12-04T09:23:32.8374915Z * [new branch] gh/williamwen42/337/orig -> origin/gh/williamwen42/337/orig 2025-12-04T09:23:32.8379613Z * [new branch] gh/williamwen42/338/base -> origin/gh/williamwen42/338/base 2025-12-04T09:23:32.8381880Z * [new branch] gh/williamwen42/338/head -> origin/gh/williamwen42/338/head 2025-12-04T09:23:32.8382404Z * [new branch] gh/williamwen42/338/orig -> origin/gh/williamwen42/338/orig 2025-12-04T09:23:32.8387478Z * [new branch] gh/williamwen42/339/base -> origin/gh/williamwen42/339/base 2025-12-04T09:23:32.8387843Z * [new branch] gh/williamwen42/339/head -> origin/gh/williamwen42/339/head 2025-12-04T09:23:32.8388006Z * [new branch] gh/williamwen42/339/orig -> origin/gh/williamwen42/339/orig 2025-12-04T09:23:32.8388168Z * [new branch] gh/williamwen42/340/base -> origin/gh/williamwen42/340/base 2025-12-04T09:23:32.8388316Z * [new branch] gh/williamwen42/340/head -> origin/gh/williamwen42/340/head 2025-12-04T09:23:32.8388471Z * [new branch] gh/williamwen42/340/orig -> origin/gh/williamwen42/340/orig 2025-12-04T09:23:32.8388618Z * [new branch] gh/williamwen42/341/base -> origin/gh/williamwen42/341/base 2025-12-04T09:23:32.8388787Z * [new branch] gh/williamwen42/341/head -> origin/gh/williamwen42/341/head 2025-12-04T09:23:32.8388937Z * [new branch] gh/williamwen42/341/orig -> origin/gh/williamwen42/341/orig 2025-12-04T09:23:32.8389080Z * [new branch] gh/williamwen42/342/base -> origin/gh/williamwen42/342/base 2025-12-04T09:23:32.8389235Z * [new branch] gh/williamwen42/342/head -> origin/gh/williamwen42/342/head 2025-12-04T09:23:32.8389376Z * [new branch] gh/williamwen42/342/orig -> origin/gh/williamwen42/342/orig 2025-12-04T09:23:32.8389520Z * [new branch] gh/williamwen42/343/base -> origin/gh/williamwen42/343/base 2025-12-04T09:23:32.8389681Z * [new branch] gh/williamwen42/343/head -> origin/gh/williamwen42/343/head 2025-12-04T09:23:32.8389824Z * [new branch] gh/williamwen42/343/orig -> origin/gh/williamwen42/343/orig 2025-12-04T09:23:32.8389980Z * [new branch] gh/williamwen42/344/base -> origin/gh/williamwen42/344/base 2025-12-04T09:23:32.8390123Z * [new branch] gh/williamwen42/344/head -> origin/gh/williamwen42/344/head 2025-12-04T09:23:32.8390270Z * [new branch] gh/williamwen42/344/orig -> origin/gh/williamwen42/344/orig 2025-12-04T09:23:32.8390433Z * [new branch] gh/williamwen42/345/base -> origin/gh/williamwen42/345/base 2025-12-04T09:23:32.8390584Z * [new branch] gh/williamwen42/345/head -> origin/gh/williamwen42/345/head 2025-12-04T09:23:32.8390751Z * [new branch] gh/williamwen42/345/orig -> origin/gh/williamwen42/345/orig 2025-12-04T09:23:32.8390906Z * [new branch] gh/williamwen42/346/base -> origin/gh/williamwen42/346/base 2025-12-04T09:23:32.8391060Z * [new branch] gh/williamwen42/346/head -> origin/gh/williamwen42/346/head 2025-12-04T09:23:32.8391218Z * [new branch] gh/williamwen42/346/orig -> origin/gh/williamwen42/346/orig 2025-12-04T09:23:32.8391373Z * [new branch] gh/williamwen42/347/base -> origin/gh/williamwen42/347/base 2025-12-04T09:23:32.8391538Z * [new branch] gh/williamwen42/347/head -> origin/gh/williamwen42/347/head 2025-12-04T09:23:32.8391751Z * [new branch] gh/williamwen42/347/orig -> origin/gh/williamwen42/347/orig 2025-12-04T09:23:32.8393413Z * [new branch] gh/williamwen42/348/base -> origin/gh/williamwen42/348/base 2025-12-04T09:23:32.8393616Z * [new branch] gh/williamwen42/348/head -> origin/gh/williamwen42/348/head 2025-12-04T09:23:32.8394162Z * [new branch] gh/williamwen42/348/orig -> origin/gh/williamwen42/348/orig 2025-12-04T09:23:32.8397286Z * [new branch] gh/williamwen42/349/base -> origin/gh/williamwen42/349/base 2025-12-04T09:23:32.8397617Z * [new branch] gh/williamwen42/349/head -> origin/gh/williamwen42/349/head 2025-12-04T09:23:32.8398022Z * [new branch] gh/williamwen42/349/orig -> origin/gh/williamwen42/349/orig 2025-12-04T09:23:32.8398510Z * [new branch] gh/williamwen42/350/base -> origin/gh/williamwen42/350/base 2025-12-04T09:23:32.8398732Z * [new branch] gh/williamwen42/350/head -> origin/gh/williamwen42/350/head 2025-12-04T09:23:32.8399713Z * [new branch] gh/williamwen42/350/orig -> origin/gh/williamwen42/350/orig 2025-12-04T09:23:32.8402793Z * [new branch] gh/williamwen42/351/base -> origin/gh/williamwen42/351/base 2025-12-04T09:23:32.8403126Z * [new branch] gh/williamwen42/351/head -> origin/gh/williamwen42/351/head 2025-12-04T09:23:32.8403310Z * [new branch] gh/williamwen42/351/orig -> origin/gh/williamwen42/351/orig 2025-12-04T09:23:32.8403529Z * [new branch] gh/williamwen42/352/base -> origin/gh/williamwen42/352/base 2025-12-04T09:23:32.8403695Z * [new branch] gh/williamwen42/352/head -> origin/gh/williamwen42/352/head 2025-12-04T09:23:32.8404324Z * [new branch] gh/williamwen42/352/orig -> origin/gh/williamwen42/352/orig 2025-12-04T09:23:32.8408658Z * [new branch] gh/williamwen42/353/base -> origin/gh/williamwen42/353/base 2025-12-04T09:23:32.8408995Z * [new branch] gh/williamwen42/353/head -> origin/gh/williamwen42/353/head 2025-12-04T09:23:32.8409175Z * [new branch] gh/williamwen42/353/orig -> origin/gh/williamwen42/353/orig 2025-12-04T09:23:32.8409323Z * [new branch] gh/williamwen42/354/base -> origin/gh/williamwen42/354/base 2025-12-04T09:23:32.8409462Z * [new branch] gh/williamwen42/354/head -> origin/gh/williamwen42/354/head 2025-12-04T09:23:32.8409721Z * [new branch] gh/williamwen42/354/orig -> origin/gh/williamwen42/354/orig 2025-12-04T09:23:32.8410001Z * [new branch] gh/williamwen42/355/base -> origin/gh/williamwen42/355/base 2025-12-04T09:23:32.8410790Z * [new branch] gh/williamwen42/355/head -> origin/gh/williamwen42/355/head 2025-12-04T09:23:32.8411245Z * [new branch] gh/williamwen42/355/orig -> origin/gh/williamwen42/355/orig 2025-12-04T09:23:32.8415116Z * [new branch] gh/williamwen42/356/base -> origin/gh/williamwen42/356/base 2025-12-04T09:23:32.8415316Z * [new branch] gh/williamwen42/356/head -> origin/gh/williamwen42/356/head 2025-12-04T09:23:32.8415466Z * [new branch] gh/williamwen42/356/orig -> origin/gh/williamwen42/356/orig 2025-12-04T09:23:32.8415613Z * [new branch] gh/williamwen42/357/base -> origin/gh/williamwen42/357/base 2025-12-04T09:23:32.8415753Z * [new branch] gh/williamwen42/357/head -> origin/gh/williamwen42/357/head 2025-12-04T09:23:32.8416128Z * [new branch] gh/williamwen42/357/orig -> origin/gh/williamwen42/357/orig 2025-12-04T09:23:32.8417296Z * [new branch] gh/williamwen42/358/base -> origin/gh/williamwen42/358/base 2025-12-04T09:23:32.8417630Z * [new branch] gh/williamwen42/358/head -> origin/gh/williamwen42/358/head 2025-12-04T09:23:32.8420022Z * [new branch] gh/williamwen42/358/orig -> origin/gh/williamwen42/358/orig 2025-12-04T09:23:32.8420188Z * [new branch] gh/xmfan/169/base -> origin/gh/xmfan/169/base 2025-12-04T09:23:32.8420341Z * [new branch] gh/xmfan/169/head -> origin/gh/xmfan/169/head 2025-12-04T09:23:32.8424373Z * [new branch] gh/xmfan/170/base -> origin/gh/xmfan/170/base 2025-12-04T09:23:32.8424544Z * [new branch] gh/xmfan/170/head -> origin/gh/xmfan/170/head 2025-12-04T09:23:32.8424676Z * [new branch] gh/xmfan/274/base -> origin/gh/xmfan/274/base 2025-12-04T09:23:32.8424796Z * [new branch] gh/xmfan/274/head -> origin/gh/xmfan/274/head 2025-12-04T09:23:32.8424925Z * [new branch] gh/xmfan/274/orig -> origin/gh/xmfan/274/orig 2025-12-04T09:23:32.8425274Z * [new branch] gh/xmfan/277/base -> origin/gh/xmfan/277/base 2025-12-04T09:23:32.8425436Z * [new branch] gh/xmfan/277/head -> origin/gh/xmfan/277/head 2025-12-04T09:23:32.8426236Z * [new branch] gh/xmfan/277/orig -> origin/gh/xmfan/277/orig 2025-12-04T09:23:32.8427354Z * [new branch] gh/xmfan/301/base -> origin/gh/xmfan/301/base 2025-12-04T09:23:32.8427565Z * [new branch] gh/xmfan/301/head -> origin/gh/xmfan/301/head 2025-12-04T09:23:32.8428410Z * [new branch] gh/xmfan/301/orig -> origin/gh/xmfan/301/orig 2025-12-04T09:23:32.8429260Z * [new branch] gh/xmfan/304/base -> origin/gh/xmfan/304/base 2025-12-04T09:23:32.8429893Z * [new branch] gh/xmfan/304/head -> origin/gh/xmfan/304/head 2025-12-04T09:23:32.8430395Z * [new branch] gh/xmfan/304/orig -> origin/gh/xmfan/304/orig 2025-12-04T09:23:32.8431625Z * [new branch] gh/xmfan/309/base -> origin/gh/xmfan/309/base 2025-12-04T09:23:32.8431874Z * [new branch] gh/xmfan/309/head -> origin/gh/xmfan/309/head 2025-12-04T09:23:32.8434250Z * [new branch] gh/xmfan/309/orig -> origin/gh/xmfan/309/orig 2025-12-04T09:23:32.8434432Z * [new branch] gh/xmfan/310/base -> origin/gh/xmfan/310/base 2025-12-04T09:23:32.8434581Z * [new branch] gh/xmfan/310/head -> origin/gh/xmfan/310/head 2025-12-04T09:23:32.8434749Z * [new branch] gh/xmfan/310/orig -> origin/gh/xmfan/310/orig 2025-12-04T09:23:32.8439064Z * [new branch] gh/xmfan/311/base -> origin/gh/xmfan/311/base 2025-12-04T09:23:32.8443870Z * [new branch] gh/xmfan/311/head -> origin/gh/xmfan/311/head 2025-12-04T09:23:32.8448103Z * [new branch] gh/xmfan/311/orig -> origin/gh/xmfan/311/orig 2025-12-04T09:23:32.8452657Z * [new branch] gh/xmfan/312/base -> origin/gh/xmfan/312/base 2025-12-04T09:23:32.8452825Z * [new branch] gh/xmfan/312/head -> origin/gh/xmfan/312/head 2025-12-04T09:23:32.8452965Z * [new branch] gh/xmfan/312/orig -> origin/gh/xmfan/312/orig 2025-12-04T09:23:32.8453107Z * [new branch] gh/xmfan/313/base -> origin/gh/xmfan/313/base 2025-12-04T09:23:32.8453233Z * [new branch] gh/xmfan/313/head -> origin/gh/xmfan/313/head 2025-12-04T09:23:32.8453384Z * [new branch] gh/xmfan/313/orig -> origin/gh/xmfan/313/orig 2025-12-04T09:23:32.8453540Z * [new branch] gh/xuanzhang816/27/base -> origin/gh/xuanzhang816/27/base 2025-12-04T09:23:32.8453695Z * [new branch] gh/xuanzhang816/27/head -> origin/gh/xuanzhang816/27/head 2025-12-04T09:23:32.8453845Z * [new branch] gh/xuanzhang816/27/orig -> origin/gh/xuanzhang816/27/orig 2025-12-04T09:23:32.8453988Z * [new branch] gh/xuanzhang816/32/base -> origin/gh/xuanzhang816/32/base 2025-12-04T09:23:32.8454139Z * [new branch] gh/xuanzhang816/32/head -> origin/gh/xuanzhang816/32/head 2025-12-04T09:23:32.8454283Z * [new branch] gh/xuanzhang816/32/orig -> origin/gh/xuanzhang816/32/orig 2025-12-04T09:23:32.8454429Z * [new branch] gh/xuanzhang816/33/base -> origin/gh/xuanzhang816/33/base 2025-12-04T09:23:32.8454567Z * [new branch] gh/xuanzhang816/33/head -> origin/gh/xuanzhang816/33/head 2025-12-04T09:23:32.8454704Z * [new branch] gh/xuanzhang816/33/orig -> origin/gh/xuanzhang816/33/orig 2025-12-04T09:23:32.8454849Z * [new branch] gh/xuanzhang816/34/base -> origin/gh/xuanzhang816/34/base 2025-12-04T09:23:32.8454989Z * [new branch] gh/xuanzhang816/34/head -> origin/gh/xuanzhang816/34/head 2025-12-04T09:23:32.8455282Z * [new branch] gh/xuanzhang816/34/orig -> origin/gh/xuanzhang816/34/orig 2025-12-04T09:23:32.8455430Z * [new branch] gh/xuanzhang816/35/base -> origin/gh/xuanzhang816/35/base 2025-12-04T09:23:32.8455572Z * [new branch] gh/xuanzhang816/35/head -> origin/gh/xuanzhang816/35/head 2025-12-04T09:23:32.8455772Z * [new branch] gh/xuanzhang816/35/orig -> origin/gh/xuanzhang816/35/orig 2025-12-04T09:23:32.8455928Z * [new branch] gh/yanbing-j/11/base -> origin/gh/yanbing-j/11/base 2025-12-04T09:23:32.8456055Z * [new branch] gh/yanbing-j/11/head -> origin/gh/yanbing-j/11/head 2025-12-04T09:23:32.8456432Z * [new branch] gh/yanbing-j/11/orig -> origin/gh/yanbing-j/11/orig 2025-12-04T09:23:32.8457840Z * [new branch] gh/yanbing-j/12/base -> origin/gh/yanbing-j/12/base 2025-12-04T09:23:32.8458272Z * [new branch] gh/yanbing-j/12/head -> origin/gh/yanbing-j/12/head 2025-12-04T09:23:32.8458996Z * [new branch] gh/yanbing-j/12/orig -> origin/gh/yanbing-j/12/orig 2025-12-04T09:23:32.8462065Z * [new branch] gh/yanbing-j/13/base -> origin/gh/yanbing-j/13/base 2025-12-04T09:23:32.8462226Z * [new branch] gh/yanbing-j/13/head -> origin/gh/yanbing-j/13/head 2025-12-04T09:23:32.8462367Z * [new branch] gh/yanbing-j/13/orig -> origin/gh/yanbing-j/13/orig 2025-12-04T09:23:32.8462503Z * [new branch] gh/yanbing-j/14/base -> origin/gh/yanbing-j/14/base 2025-12-04T09:23:32.8462630Z * [new branch] gh/yanbing-j/14/head -> origin/gh/yanbing-j/14/head 2025-12-04T09:23:32.8463521Z * [new branch] gh/yanbing-j/14/orig -> origin/gh/yanbing-j/14/orig 2025-12-04T09:23:32.8464161Z * [new branch] gh/yanbing-j/15/base -> origin/gh/yanbing-j/15/base 2025-12-04T09:23:32.8464721Z * [new branch] gh/yanbing-j/15/head -> origin/gh/yanbing-j/15/head 2025-12-04T09:23:32.8465287Z * [new branch] gh/yanbing-j/15/orig -> origin/gh/yanbing-j/15/orig 2025-12-04T09:23:32.8466452Z * [new branch] gh/yanbing-j/18/base -> origin/gh/yanbing-j/18/base 2025-12-04T09:23:32.8466732Z * [new branch] gh/yanbing-j/18/head -> origin/gh/yanbing-j/18/head 2025-12-04T09:23:32.8467719Z * [new branch] gh/yanbing-j/18/orig -> origin/gh/yanbing-j/18/orig 2025-12-04T09:23:32.8468528Z * [new branch] gh/yanbing-j/19/base -> origin/gh/yanbing-j/19/base 2025-12-04T09:23:32.8468882Z * [new branch] gh/yanbing-j/19/head -> origin/gh/yanbing-j/19/head 2025-12-04T09:23:32.8469793Z * [new branch] gh/yanbing-j/19/orig -> origin/gh/yanbing-j/19/orig 2025-12-04T09:23:32.8470590Z * [new branch] gh/yanbing-j/20/base -> origin/gh/yanbing-j/20/base 2025-12-04T09:23:32.8471027Z * [new branch] gh/yanbing-j/20/head -> origin/gh/yanbing-j/20/head 2025-12-04T09:23:32.8472091Z * [new branch] gh/yanbing-j/20/orig -> origin/gh/yanbing-j/20/orig 2025-12-04T09:23:32.8473172Z * [new branch] gh/yanbing-j/21/base -> origin/gh/yanbing-j/21/base 2025-12-04T09:23:32.8473377Z * [new branch] gh/yanbing-j/21/head -> origin/gh/yanbing-j/21/head 2025-12-04T09:23:32.8474696Z * [new branch] gh/yanbing-j/22/base -> origin/gh/yanbing-j/22/base 2025-12-04T09:23:32.8475028Z * [new branch] gh/yanbing-j/22/head -> origin/gh/yanbing-j/22/head 2025-12-04T09:23:32.8476115Z * [new branch] gh/yanbing-j/22/orig -> origin/gh/yanbing-j/22/orig 2025-12-04T09:23:32.8476965Z * [new branch] gh/yanbing-j/23/base -> origin/gh/yanbing-j/23/base 2025-12-04T09:23:32.8477421Z * [new branch] gh/yanbing-j/23/head -> origin/gh/yanbing-j/23/head 2025-12-04T09:23:32.8478282Z * [new branch] gh/yanbing-j/23/orig -> origin/gh/yanbing-j/23/orig 2025-12-04T09:23:32.8479344Z * [new branch] gh/yanbing-j/24/base -> origin/gh/yanbing-j/24/base 2025-12-04T09:23:32.8481987Z * [new branch] gh/yanbing-j/24/head -> origin/gh/yanbing-j/24/head 2025-12-04T09:23:32.8482408Z * [new branch] gh/yanbing-j/24/orig -> origin/gh/yanbing-j/24/orig 2025-12-04T09:23:32.8482568Z * [new branch] gh/yanbing-j/25/base -> origin/gh/yanbing-j/25/base 2025-12-04T09:23:32.8482704Z * [new branch] gh/yanbing-j/25/head -> origin/gh/yanbing-j/25/head 2025-12-04T09:23:32.8482851Z * [new branch] gh/yanbing-j/25/orig -> origin/gh/yanbing-j/25/orig 2025-12-04T09:23:32.8483347Z * [new branch] gh/yanbing-j/26/base -> origin/gh/yanbing-j/26/base 2025-12-04T09:23:32.8484110Z * [new branch] gh/yanbing-j/26/head -> origin/gh/yanbing-j/26/head 2025-12-04T09:23:32.8484998Z * [new branch] gh/yanbing-j/26/orig -> origin/gh/yanbing-j/26/orig 2025-12-04T09:23:32.8486231Z * [new branch] gh/yang-yu-hang/1/base -> origin/gh/yang-yu-hang/1/base 2025-12-04T09:23:32.8486782Z * [new branch] gh/yang-yu-hang/1/head -> origin/gh/yang-yu-hang/1/head 2025-12-04T09:23:32.8487738Z * [new branch] gh/yang-yu-hang/1/orig -> origin/gh/yang-yu-hang/1/orig 2025-12-04T09:23:32.8489176Z * [new branch] gh/yang-yu-hang/2/base -> origin/gh/yang-yu-hang/2/base 2025-12-04T09:23:32.8490191Z * [new branch] gh/yang-yu-hang/2/head -> origin/gh/yang-yu-hang/2/head 2025-12-04T09:23:32.8490347Z * [new branch] gh/yang-yu-hang/2/orig -> origin/gh/yang-yu-hang/2/orig 2025-12-04T09:23:32.8492913Z * [new branch] gh/yang-yu-hang/3/base -> origin/gh/yang-yu-hang/3/base 2025-12-04T09:23:32.8493108Z * [new branch] gh/yang-yu-hang/3/head -> origin/gh/yang-yu-hang/3/head 2025-12-04T09:23:32.8493245Z * [new branch] gh/yang-yu-hang/3/orig -> origin/gh/yang-yu-hang/3/orig 2025-12-04T09:23:32.8497636Z * [new branch] gh/yangw-dev/12/base -> origin/gh/yangw-dev/12/base 2025-12-04T09:23:32.8517482Z * [new branch] gh/yangw-dev/12/head -> origin/gh/yangw-dev/12/head 2025-12-04T09:23:32.8517652Z * [new branch] gh/yangw-dev/12/orig -> origin/gh/yangw-dev/12/orig 2025-12-04T09:23:32.8517850Z * [new branch] gh/yangw-dev/13/base -> origin/gh/yangw-dev/13/base 2025-12-04T09:23:32.8517978Z * [new branch] gh/yangw-dev/13/head -> origin/gh/yangw-dev/13/head 2025-12-04T09:23:32.8518124Z * [new branch] gh/yangw-dev/13/orig -> origin/gh/yangw-dev/13/orig 2025-12-04T09:23:32.8518262Z * [new branch] gh/yangw-dev/14/base -> origin/gh/yangw-dev/14/base 2025-12-04T09:23:32.8518395Z * [new branch] gh/yangw-dev/14/head -> origin/gh/yangw-dev/14/head 2025-12-04T09:23:32.8518535Z * [new branch] gh/yangw-dev/14/orig -> origin/gh/yangw-dev/14/orig 2025-12-04T09:23:32.8518661Z * [new branch] gh/yangw-dev/15/base -> origin/gh/yangw-dev/15/base 2025-12-04T09:23:32.8518789Z * [new branch] gh/yangw-dev/15/head -> origin/gh/yangw-dev/15/head 2025-12-04T09:23:32.8518923Z * [new branch] gh/yangw-dev/15/orig -> origin/gh/yangw-dev/15/orig 2025-12-04T09:23:32.8519048Z * [new branch] gh/yangw-dev/19/base -> origin/gh/yangw-dev/19/base 2025-12-04T09:23:32.8519185Z * [new branch] gh/yangw-dev/19/head -> origin/gh/yangw-dev/19/head 2025-12-04T09:23:32.8519318Z * [new branch] gh/yangw-dev/19/orig -> origin/gh/yangw-dev/19/orig 2025-12-04T09:23:32.8519447Z * [new branch] gh/yangw-dev/26/base -> origin/gh/yangw-dev/26/base 2025-12-04T09:23:32.8519792Z * [new branch] gh/yangw-dev/26/head -> origin/gh/yangw-dev/26/head 2025-12-04T09:23:32.8519920Z * [new branch] gh/yangw-dev/26/orig -> origin/gh/yangw-dev/26/orig 2025-12-04T09:23:32.8520059Z * [new branch] gh/yangw-dev/27/base -> origin/gh/yangw-dev/27/base 2025-12-04T09:23:32.8520245Z * [new branch] gh/yangw-dev/27/head -> origin/gh/yangw-dev/27/head 2025-12-04T09:23:32.8520372Z * [new branch] gh/yangw-dev/27/orig -> origin/gh/yangw-dev/27/orig 2025-12-04T09:23:32.8520516Z * [new branch] gh/ydwu4/292/base -> origin/gh/ydwu4/292/base 2025-12-04T09:23:32.8520642Z * [new branch] gh/ydwu4/292/head -> origin/gh/ydwu4/292/head 2025-12-04T09:23:32.8520763Z * [new branch] gh/ydwu4/292/orig -> origin/gh/ydwu4/292/orig 2025-12-04T09:23:32.8520893Z * [new branch] gh/ydwu4/294/base -> origin/gh/ydwu4/294/base 2025-12-04T09:23:32.8521014Z * [new branch] gh/ydwu4/294/head -> origin/gh/ydwu4/294/head 2025-12-04T09:23:32.8521143Z * [new branch] gh/ydwu4/294/orig -> origin/gh/ydwu4/294/orig 2025-12-04T09:23:32.8521262Z * [new branch] gh/ydwu4/295/base -> origin/gh/ydwu4/295/base 2025-12-04T09:23:32.8521384Z * [new branch] gh/ydwu4/295/head -> origin/gh/ydwu4/295/head 2025-12-04T09:23:32.8521511Z * [new branch] gh/ydwu4/295/orig -> origin/gh/ydwu4/295/orig 2025-12-04T09:23:32.8521630Z * [new branch] gh/ydwu4/296/base -> origin/gh/ydwu4/296/base 2025-12-04T09:23:32.8521757Z * [new branch] gh/ydwu4/296/head -> origin/gh/ydwu4/296/head 2025-12-04T09:23:32.8521888Z * [new branch] gh/ydwu4/296/orig -> origin/gh/ydwu4/296/orig 2025-12-04T09:23:32.8522008Z * [new branch] gh/ydwu4/306/base -> origin/gh/ydwu4/306/base 2025-12-04T09:23:32.8522139Z * [new branch] gh/ydwu4/306/head -> origin/gh/ydwu4/306/head 2025-12-04T09:23:32.8522258Z * [new branch] gh/ydwu4/306/orig -> origin/gh/ydwu4/306/orig 2025-12-04T09:23:32.8522384Z * [new branch] gh/ydwu4/312/base -> origin/gh/ydwu4/312/base 2025-12-04T09:23:32.8522506Z * [new branch] gh/ydwu4/312/head -> origin/gh/ydwu4/312/head 2025-12-04T09:23:32.8522626Z * [new branch] gh/ydwu4/312/orig -> origin/gh/ydwu4/312/orig 2025-12-04T09:23:32.8522752Z * [new branch] gh/ydwu4/322/base -> origin/gh/ydwu4/322/base 2025-12-04T09:23:32.8523042Z * [new branch] gh/ydwu4/322/head -> origin/gh/ydwu4/322/head 2025-12-04T09:23:32.8523293Z * [new branch] gh/ydwu4/322/orig -> origin/gh/ydwu4/322/orig 2025-12-04T09:23:32.8528838Z * [new branch] gh/ydwu4/327/base -> origin/gh/ydwu4/327/base 2025-12-04T09:23:32.8529169Z * [new branch] gh/ydwu4/327/head -> origin/gh/ydwu4/327/head 2025-12-04T09:23:32.8529335Z * [new branch] gh/ydwu4/327/orig -> origin/gh/ydwu4/327/orig 2025-12-04T09:23:32.8529464Z * [new branch] gh/ydwu4/328/base -> origin/gh/ydwu4/328/base 2025-12-04T09:23:32.8529605Z * [new branch] gh/ydwu4/328/head -> origin/gh/ydwu4/328/head 2025-12-04T09:23:32.8529732Z * [new branch] gh/ydwu4/328/orig -> origin/gh/ydwu4/328/orig 2025-12-04T09:23:32.8529989Z * [new branch] gh/ydwu4/329/base -> origin/gh/ydwu4/329/base 2025-12-04T09:23:32.8530265Z * [new branch] gh/ydwu4/329/head -> origin/gh/ydwu4/329/head 2025-12-04T09:23:32.8530400Z * [new branch] gh/ydwu4/329/orig -> origin/gh/ydwu4/329/orig 2025-12-04T09:23:32.8530712Z * [new branch] gh/ydwu4/330/base -> origin/gh/ydwu4/330/base 2025-12-04T09:23:32.8530841Z * [new branch] gh/ydwu4/330/head -> origin/gh/ydwu4/330/head 2025-12-04T09:23:32.8531004Z * [new branch] gh/ydwu4/330/orig -> origin/gh/ydwu4/330/orig 2025-12-04T09:23:32.8537487Z * [new branch] gh/ydwu4/331/base -> origin/gh/ydwu4/331/base 2025-12-04T09:23:32.8537988Z * [new branch] gh/ydwu4/331/head -> origin/gh/ydwu4/331/head 2025-12-04T09:23:32.8538250Z * [new branch] gh/ydwu4/331/orig -> origin/gh/ydwu4/331/orig 2025-12-04T09:23:32.8538409Z * [new branch] gh/ydwu4/332/base -> origin/gh/ydwu4/332/base 2025-12-04T09:23:32.8538535Z * [new branch] gh/ydwu4/332/head -> origin/gh/ydwu4/332/head 2025-12-04T09:23:32.8538672Z * [new branch] gh/ydwu4/332/orig -> origin/gh/ydwu4/332/orig 2025-12-04T09:23:32.8538949Z * [new branch] gh/ydwu4/333/base -> origin/gh/ydwu4/333/base 2025-12-04T09:23:32.8539378Z * [new branch] gh/ydwu4/333/head -> origin/gh/ydwu4/333/head 2025-12-04T09:23:32.8539543Z * [new branch] gh/ydwu4/333/orig -> origin/gh/ydwu4/333/orig 2025-12-04T09:23:32.8539669Z * [new branch] gh/ydwu4/334/base -> origin/gh/ydwu4/334/base 2025-12-04T09:23:32.8539811Z * [new branch] gh/ydwu4/334/head -> origin/gh/ydwu4/334/head 2025-12-04T09:23:32.8539950Z * [new branch] gh/ydwu4/334/orig -> origin/gh/ydwu4/334/orig 2025-12-04T09:23:32.8540083Z * [new branch] gh/ydwu4/335/base -> origin/gh/ydwu4/335/base 2025-12-04T09:23:32.8540217Z * [new branch] gh/ydwu4/335/head -> origin/gh/ydwu4/335/head 2025-12-04T09:23:32.8545541Z * [new branch] gh/ydwu4/335/orig -> origin/gh/ydwu4/335/orig 2025-12-04T09:23:32.8545871Z * [new branch] gh/ydwu4/337/base -> origin/gh/ydwu4/337/base 2025-12-04T09:23:32.8546041Z * [new branch] gh/ydwu4/337/head -> origin/gh/ydwu4/337/head 2025-12-04T09:23:32.8546199Z * [new branch] gh/ydwu4/337/orig -> origin/gh/ydwu4/337/orig 2025-12-04T09:23:32.8546343Z * [new branch] gh/ydwu4/339/base -> origin/gh/ydwu4/339/base 2025-12-04T09:23:32.8548677Z * [new branch] gh/ydwu4/339/head -> origin/gh/ydwu4/339/head 2025-12-04T09:23:32.8548852Z * [new branch] gh/ydwu4/339/orig -> origin/gh/ydwu4/339/orig 2025-12-04T09:23:32.8548990Z * [new branch] gh/yf225/133/base -> origin/gh/yf225/133/base 2025-12-04T09:23:32.8549115Z * [new branch] gh/yf225/133/head -> origin/gh/yf225/133/head 2025-12-04T09:23:32.8549259Z * [new branch] gh/yf225/93/base -> origin/gh/yf225/93/base 2025-12-04T09:23:32.8549407Z * [new branch] gh/yf225/93/head -> origin/gh/yf225/93/head 2025-12-04T09:23:32.8552464Z * [new branch] gh/yifuwang/152/base -> origin/gh/yifuwang/152/base 2025-12-04T09:23:32.8552884Z * [new branch] gh/yifuwang/152/head -> origin/gh/yifuwang/152/head 2025-12-04T09:23:32.8553039Z * [new branch] gh/yifuwang/152/orig -> origin/gh/yifuwang/152/orig 2025-12-04T09:23:32.8553192Z * [new branch] gh/yifuwang/195/base -> origin/gh/yifuwang/195/base 2025-12-04T09:23:32.8553330Z * [new branch] gh/yifuwang/195/head -> origin/gh/yifuwang/195/head 2025-12-04T09:23:32.8559422Z * [new branch] gh/yifuwang/195/orig -> origin/gh/yifuwang/195/orig 2025-12-04T09:23:32.8563469Z * [new branch] gh/yiming0416/1/base -> origin/gh/yiming0416/1/base 2025-12-04T09:23:32.8568187Z * [new branch] gh/yiming0416/1/head -> origin/gh/yiming0416/1/head 2025-12-04T09:23:32.8572860Z * [new branch] gh/yiming0416/2/base -> origin/gh/yiming0416/2/base 2025-12-04T09:23:32.8576953Z * [new branch] gh/yiming0416/2/head -> origin/gh/yiming0416/2/head 2025-12-04T09:23:32.8578757Z * [new branch] gh/yushangdi/1/base -> origin/gh/yushangdi/1/base 2025-12-04T09:23:32.8579036Z * [new branch] gh/yushangdi/1/head -> origin/gh/yushangdi/1/head 2025-12-04T09:23:32.8579230Z * [new branch] gh/yushangdi/10/base -> origin/gh/yushangdi/10/base 2025-12-04T09:23:32.8579363Z * [new branch] gh/yushangdi/10/head -> origin/gh/yushangdi/10/head 2025-12-04T09:23:32.8579500Z * [new branch] gh/yushangdi/10/orig -> origin/gh/yushangdi/10/orig 2025-12-04T09:23:32.8579629Z * [new branch] gh/yushangdi/11/base -> origin/gh/yushangdi/11/base 2025-12-04T09:23:32.8579756Z * [new branch] gh/yushangdi/11/head -> origin/gh/yushangdi/11/head 2025-12-04T09:23:32.8579897Z * [new branch] gh/yushangdi/11/orig -> origin/gh/yushangdi/11/orig 2025-12-04T09:23:32.8580027Z * [new branch] gh/yushangdi/2/base -> origin/gh/yushangdi/2/base 2025-12-04T09:23:32.8580165Z * [new branch] gh/yushangdi/2/head -> origin/gh/yushangdi/2/head 2025-12-04T09:23:32.8580296Z * [new branch] gh/yushangdi/7/base -> origin/gh/yushangdi/7/base 2025-12-04T09:23:32.8580420Z * [new branch] gh/yushangdi/7/head -> origin/gh/yushangdi/7/head 2025-12-04T09:23:32.8580555Z * [new branch] gh/yushangdi/7/orig -> origin/gh/yushangdi/7/orig 2025-12-04T09:23:32.8580681Z * [new branch] gh/yushangdi/8/base -> origin/gh/yushangdi/8/base 2025-12-04T09:23:32.8580817Z * [new branch] gh/yushangdi/8/head -> origin/gh/yushangdi/8/head 2025-12-04T09:23:32.8580950Z * [new branch] gh/yushangdi/8/orig -> origin/gh/yushangdi/8/orig 2025-12-04T09:23:32.8581075Z * [new branch] gh/yushangdi/9/base -> origin/gh/yushangdi/9/base 2025-12-04T09:23:32.8581209Z * [new branch] gh/yushangdi/9/head -> origin/gh/yushangdi/9/head 2025-12-04T09:23:32.8581333Z * [new branch] gh/yushangdi/9/orig -> origin/gh/yushangdi/9/orig 2025-12-04T09:23:32.8581470Z * [new branch] gh/zklaus/19/base -> origin/gh/zklaus/19/base 2025-12-04T09:23:32.8581592Z * [new branch] gh/zklaus/19/head -> origin/gh/zklaus/19/head 2025-12-04T09:23:32.8581709Z * [new branch] gh/zklaus/19/orig -> origin/gh/zklaus/19/orig 2025-12-04T09:23:32.8581835Z * [new branch] gh/zklaus/20/base -> origin/gh/zklaus/20/base 2025-12-04T09:23:32.8581957Z * [new branch] gh/zklaus/20/head -> origin/gh/zklaus/20/head 2025-12-04T09:23:32.8582079Z * [new branch] gh/zklaus/20/orig -> origin/gh/zklaus/20/orig 2025-12-04T09:23:32.8582205Z * [new branch] gh/zklaus/21/base -> origin/gh/zklaus/21/base 2025-12-04T09:23:32.8582327Z * [new branch] gh/zklaus/21/head -> origin/gh/zklaus/21/head 2025-12-04T09:23:32.8582478Z * [new branch] gh/zklaus/21/orig -> origin/gh/zklaus/21/orig 2025-12-04T09:23:32.8582612Z * [new branch] gh/zklaus/22/base -> origin/gh/zklaus/22/base 2025-12-04T09:23:32.8582754Z * [new branch] gh/zklaus/22/head -> origin/gh/zklaus/22/head 2025-12-04T09:23:32.8582906Z * [new branch] gh/zklaus/22/orig -> origin/gh/zklaus/22/orig 2025-12-04T09:23:32.8583108Z * [new branch] gh/zklaus/23/base -> origin/gh/zklaus/23/base 2025-12-04T09:23:32.8583247Z * [new branch] gh/zklaus/23/head -> origin/gh/zklaus/23/head 2025-12-04T09:23:32.8583484Z * [new branch] gh/zklaus/23/orig -> origin/gh/zklaus/23/orig 2025-12-04T09:23:32.8583948Z * [new branch] gh/zklaus/24/base -> origin/gh/zklaus/24/base 2025-12-04T09:23:32.8584535Z * [new branch] gh/zklaus/24/head -> origin/gh/zklaus/24/head 2025-12-04T09:23:32.8588163Z * [new branch] gh/zklaus/24/orig -> origin/gh/zklaus/24/orig 2025-12-04T09:23:32.8588480Z * [new branch] gh/zou3519/1197/base -> origin/gh/zou3519/1197/base 2025-12-04T09:23:32.8588638Z * [new branch] gh/zou3519/1197/head -> origin/gh/zou3519/1197/head 2025-12-04T09:23:32.8588770Z * [new branch] gh/zou3519/1197/orig -> origin/gh/zou3519/1197/orig 2025-12-04T09:23:32.8588898Z * [new branch] gh/zou3519/1199/base -> origin/gh/zou3519/1199/base 2025-12-04T09:23:32.8589037Z * [new branch] gh/zou3519/1199/head -> origin/gh/zou3519/1199/head 2025-12-04T09:23:32.8589173Z * [new branch] gh/zou3519/1199/orig -> origin/gh/zou3519/1199/orig 2025-12-04T09:23:32.8589344Z * [new branch] gh/zou3519/1200/base -> origin/gh/zou3519/1200/base 2025-12-04T09:23:32.8589494Z * [new branch] gh/zou3519/1200/head -> origin/gh/zou3519/1200/head 2025-12-04T09:23:32.8589758Z * [new branch] gh/zou3519/1200/orig -> origin/gh/zou3519/1200/orig 2025-12-04T09:23:32.8591200Z * [new branch] gh/zou3519/1201/base -> origin/gh/zou3519/1201/base 2025-12-04T09:23:32.8591750Z * [new branch] gh/zou3519/1201/head -> origin/gh/zou3519/1201/head 2025-12-04T09:23:32.8591914Z * [new branch] gh/zou3519/1201/orig -> origin/gh/zou3519/1201/orig 2025-12-04T09:23:32.8593175Z * [new branch] gh/zou3519/1202/base -> origin/gh/zou3519/1202/base 2025-12-04T09:23:32.8593353Z * [new branch] gh/zou3519/1202/head -> origin/gh/zou3519/1202/head 2025-12-04T09:23:32.8600931Z * [new branch] gh/zou3519/1202/orig -> origin/gh/zou3519/1202/orig 2025-12-04T09:23:32.8606241Z * [new branch] gh/zpcore/1/base -> origin/gh/zpcore/1/base 2025-12-04T09:23:32.8608382Z * [new branch] gh/zpcore/1/head -> origin/gh/zpcore/1/head 2025-12-04T09:23:32.8608571Z * [new branch] gh/zpcore/11/base -> origin/gh/zpcore/11/base 2025-12-04T09:23:32.8608709Z * [new branch] gh/zpcore/11/head -> origin/gh/zpcore/11/head 2025-12-04T09:23:32.8608875Z * [new branch] gh/zpcore/11/orig -> origin/gh/zpcore/11/orig 2025-12-04T09:23:32.8609007Z * [new branch] gh/zpcore/12/base -> origin/gh/zpcore/12/base 2025-12-04T09:23:32.8609151Z * [new branch] gh/zpcore/12/head -> origin/gh/zpcore/12/head 2025-12-04T09:23:32.8609288Z * [new branch] gh/zpcore/12/orig -> origin/gh/zpcore/12/orig 2025-12-04T09:23:32.8609444Z * [new branch] gh/zpcore/13/base -> origin/gh/zpcore/13/base 2025-12-04T09:23:32.8609591Z * [new branch] gh/zpcore/13/head -> origin/gh/zpcore/13/head 2025-12-04T09:23:32.8609731Z * [new branch] gh/zpcore/13/orig -> origin/gh/zpcore/13/orig 2025-12-04T09:23:32.8609879Z * [new branch] gh/zpcore/14/base -> origin/gh/zpcore/14/base 2025-12-04T09:23:32.8610015Z * [new branch] gh/zpcore/14/head -> origin/gh/zpcore/14/head 2025-12-04T09:23:32.8610150Z * [new branch] gh/zpcore/14/orig -> origin/gh/zpcore/14/orig 2025-12-04T09:23:32.8610286Z * [new branch] gh/zpcore/15/base -> origin/gh/zpcore/15/base 2025-12-04T09:23:32.8610421Z * [new branch] gh/zpcore/15/head -> origin/gh/zpcore/15/head 2025-12-04T09:23:32.8610564Z * [new branch] gh/zpcore/15/orig -> origin/gh/zpcore/15/orig 2025-12-04T09:23:32.8610918Z * [new branch] gh/zpcore/2/base -> origin/gh/zpcore/2/base 2025-12-04T09:23:32.8614866Z * [new branch] gh/zpcore/2/head -> origin/gh/zpcore/2/head 2025-12-04T09:23:32.8619756Z * [new branch] gh/zpcore/21/base -> origin/gh/zpcore/21/base 2025-12-04T09:23:32.8620467Z * [new branch] gh/zpcore/21/head -> origin/gh/zpcore/21/head 2025-12-04T09:23:32.8620664Z * [new branch] gh/zpcore/21/orig -> origin/gh/zpcore/21/orig 2025-12-04T09:23:32.8620807Z * [new branch] gh/zpcore/22/base -> origin/gh/zpcore/22/base 2025-12-04T09:23:32.8620953Z * [new branch] gh/zpcore/22/head -> origin/gh/zpcore/22/head 2025-12-04T09:23:32.8621091Z * [new branch] gh/zpcore/22/orig -> origin/gh/zpcore/22/orig 2025-12-04T09:23:32.8621228Z * [new branch] gh/zpcore/23/base -> origin/gh/zpcore/23/base 2025-12-04T09:23:32.8621391Z * [new branch] gh/zpcore/23/head -> origin/gh/zpcore/23/head 2025-12-04T09:23:32.8621529Z * [new branch] gh/zpcore/23/orig -> origin/gh/zpcore/23/orig 2025-12-04T09:23:32.8621683Z * [new branch] gh/zpcore/24/base -> origin/gh/zpcore/24/base 2025-12-04T09:23:32.8621824Z * [new branch] gh/zpcore/24/head -> origin/gh/zpcore/24/head 2025-12-04T09:23:32.8621964Z * [new branch] gh/zpcore/24/orig -> origin/gh/zpcore/24/orig 2025-12-04T09:23:32.8622103Z * [new branch] gh/zpcore/25/base -> origin/gh/zpcore/25/base 2025-12-04T09:23:32.8622243Z * [new branch] gh/zpcore/25/head -> origin/gh/zpcore/25/head 2025-12-04T09:23:32.8622382Z * [new branch] gh/zpcore/25/orig -> origin/gh/zpcore/25/orig 2025-12-04T09:23:32.8622528Z * [new branch] gh/zpcore/26/base -> origin/gh/zpcore/26/base 2025-12-04T09:23:32.8622693Z * [new branch] gh/zpcore/26/head -> origin/gh/zpcore/26/head 2025-12-04T09:23:32.8622822Z * [new branch] gh/zpcore/26/orig -> origin/gh/zpcore/26/orig 2025-12-04T09:23:32.8622942Z * [new branch] gh/zpcore/27/base -> origin/gh/zpcore/27/base 2025-12-04T09:23:32.8623124Z * [new branch] gh/zpcore/27/head -> origin/gh/zpcore/27/head 2025-12-04T09:23:32.8624113Z * [new branch] gh/zpcore/27/orig -> origin/gh/zpcore/27/orig 2025-12-04T09:23:32.8628151Z * [new branch] gh/zpcore/28/base -> origin/gh/zpcore/28/base 2025-12-04T09:23:32.8628317Z * [new branch] gh/zpcore/28/head -> origin/gh/zpcore/28/head 2025-12-04T09:23:32.8628448Z * [new branch] gh/zpcore/28/orig -> origin/gh/zpcore/28/orig 2025-12-04T09:23:32.8628586Z * [new branch] gh/zpcore/3/base -> origin/gh/zpcore/3/base 2025-12-04T09:23:32.8628751Z * [new branch] gh/zpcore/3/head -> origin/gh/zpcore/3/head 2025-12-04T09:23:32.8629145Z * [new branch] gh/zpcore/4/base -> origin/gh/zpcore/4/base 2025-12-04T09:23:32.8630600Z * [new branch] gh/zpcore/4/head -> origin/gh/zpcore/4/head 2025-12-04T09:23:32.8630798Z * [new branch] gh/zpcore/5/base -> origin/gh/zpcore/5/base 2025-12-04T09:23:32.8631298Z * [new branch] gh/zpcore/5/head -> origin/gh/zpcore/5/head 2025-12-04T09:23:32.8632260Z * [new branch] gh/zpcore/6/base -> origin/gh/zpcore/6/base 2025-12-04T09:23:32.8632572Z * [new branch] gh/zpcore/6/head -> origin/gh/zpcore/6/head 2025-12-04T09:23:32.8634472Z * [new branch] gh/zpcore/7/base -> origin/gh/zpcore/7/base 2025-12-04T09:23:32.8634772Z * [new branch] gh/zpcore/7/head -> origin/gh/zpcore/7/head 2025-12-04T09:23:32.8635247Z * [new branch] gh/zpcore/8/base -> origin/gh/zpcore/8/base 2025-12-04T09:23:32.8637504Z * [new branch] gh/zpcore/8/head -> origin/gh/zpcore/8/head 2025-12-04T09:23:32.8637662Z * [new branch] google-main -> origin/google-main 2025-12-04T09:23:32.8643783Z * [new branch] guangyey/external_stream -> origin/guangyey/external_stream 2025-12-04T09:23:32.8648840Z * [new branch] guangyey/test_2025 -> origin/guangyey/test_2025 2025-12-04T09:23:32.8651046Z * [new branch] guilhermeleobas/cherry-pick-55d87d9dfd9 -> origin/guilhermeleobas/cherry-pick-55d87d9dfd9 2025-12-04T09:23:32.8651381Z * [new branch] hameerabbasi/complex_tensor_subclass -> origin/hameerabbasi/complex_tensor_subclass 2025-12-04T09:23:32.8656706Z * [new branch] hameerabbasi/fix-ctensor-gradcheck-tests -> origin/hameerabbasi/fix-ctensor-gradcheck-tests 2025-12-04T09:23:32.8659044Z * [new branch] hameerabbasi/gradcheck-allclose -> origin/hameerabbasi/gradcheck-allclose 2025-12-04T09:23:32.8665113Z * [new branch] hc_baseline -> origin/hc_baseline 2025-12-04T09:23:32.8668495Z * [new branch] hhh_rand -> origin/hhh_rand 2025-12-04T09:23:32.8668659Z * [new branch] huba/f1 -> origin/huba/f1 2025-12-04T09:23:32.8669016Z * [new branch] increase-timeout-linux-jammy-cuda12_8-py3_10-gcc11-test -> origin/increase-timeout-linux-jammy-cuda12_8-py3_10-gcc11-test 2025-12-04T09:23:32.8669150Z * [new branch] inlining -> origin/inlining 2025-12-04T09:23:32.8669308Z * [new branch] inlining-ezyang -> origin/inlining-ezyang 2025-12-04T09:23:32.8669467Z * [new branch] install-torchao-0.13.0 -> origin/install-torchao-0.13.0 2025-12-04T09:23:32.8669798Z * [new branch] instrument-trunk-pull-linux-with-job-test-filters -> origin/instrument-trunk-pull-linux-with-job-test-filters 2025-12-04T09:23:32.8669939Z * [new branch] invoke-subgraph -> origin/invoke-subgraph 2025-12-04T09:23:32.8670085Z * [new branch] issue#58739 -> origin/issue#58739 2025-12-04T09:23:32.8670234Z * [new branch] jainapurva-patch-1 -> origin/jainapurva-patch-1 2025-12-04T09:23:32.8670350Z * [new branch] jathu/o3 -> origin/jathu/o3 2025-12-04T09:23:32.8670475Z * [new branch] jathu/sve -> origin/jathu/sve 2025-12-04T09:23:32.8670680Z * [new branch] jcaip/test-cusparselt-version-0.6.2 -> origin/jcaip/test-cusparselt-version-0.6.2 2025-12-04T09:23:32.8670865Z * [new branch] jcaip/update-cusparselt-0.6.2 -> origin/jcaip/update-cusparselt-0.6.2 2025-12-04T09:23:32.8671054Z * [new branch] jiannanWang/memorysnapshot_filter -> origin/jiannanWang/memorysnapshot_filter 2025-12-04T09:23:32.8671241Z * [new branch] jiannanWang/profilerstepwarning -> origin/jiannanWang/profilerstepwarning 2025-12-04T09:23:32.8671399Z * [new branch] jithunnair-amd-patch-1 -> origin/jithunnair-amd-patch-1 2025-12-04T09:23:32.8671552Z * [new branch] jithunnair-amd-patch-10 -> origin/jithunnair-amd-patch-10 2025-12-04T09:23:32.8671698Z * [new branch] jithunnair-amd-patch-2 -> origin/jithunnair-amd-patch-2 2025-12-04T09:23:32.8671847Z * [new branch] jithunnair-amd-patch-3 -> origin/jithunnair-amd-patch-3 2025-12-04T09:23:32.8671987Z * [new branch] jithunnair-amd-patch-4 -> origin/jithunnair-amd-patch-4 2025-12-04T09:23:32.8672134Z * [new branch] jithunnair-amd-patch-5 -> origin/jithunnair-amd-patch-5 2025-12-04T09:23:32.8672273Z * [new branch] jithunnair-amd-patch-6 -> origin/jithunnair-amd-patch-6 2025-12-04T09:23:32.8672410Z * [new branch] jithunnair-amd-patch-7 -> origin/jithunnair-amd-patch-7 2025-12-04T09:23:32.8672873Z * [new branch] jithunnair-amd-patch-8 -> origin/jithunnair-amd-patch-8 2025-12-04T09:23:32.8673032Z * [new branch] jithunnair-amd-patch-9 -> origin/jithunnair-amd-patch-9 2025-12-04T09:23:32.8673195Z * [new branch] justinchu/native-qdq -> origin/justinchu/native-qdq 2025-12-04T09:23:32.8673393Z * [new branch] kainan666/xlf_debug -> origin/kainan666/xlf_debug 2025-12-04T09:23:32.8673521Z * [new branch] kainan_test -> origin/kainan_test 2025-12-04T09:23:32.8673685Z * [new branch] larryliu0820-patch-1 -> origin/larryliu0820-patch-1 2025-12-04T09:23:32.8673879Z * [new branch] leslie/test_group_gemm_epilogues -> origin/leslie/test_group_gemm_epilogues 2025-12-04T09:23:32.8674086Z * [new branch] lessw2020/fix_cutlass_cache_error -> origin/lessw2020/fix_cutlass_cache_error 2025-12-04T09:23:32.8674246Z * [new branch] liaoxuan/shm_all_reduce -> origin/liaoxuan/shm_all_reduce 2025-12-04T09:23:32.8674459Z * [new branch] liaoxuan/test_fa_disable_softmax -> origin/liaoxuan/test_fa_disable_softmax 2025-12-04T09:23:32.8674622Z * [new branch] liaoxuan/test_int8_sdpa -> origin/liaoxuan/test_int8_sdpa 2025-12-04T09:23:32.8674759Z * [new branch] llama4-stable -> origin/llama4-stable 2025-12-04T09:23:32.8674903Z * [new branch] lts/release/1.8 -> origin/lts/release/1.8 2025-12-04T09:23:32.8675092Z * [new branch] lucaskabela/#94773 -> origin/lucaskabela/#94773 2025-12-04T09:23:32.8675241Z * [new branch] lucaskabela/fix_164876 -> origin/lucaskabela/fix_164876 2025-12-04T09:23:32.8675417Z * [new branch] lucaskabela/flop_counter -> origin/lucaskabela/flop_counter 2025-12-04T09:23:32.8675766Z * [new branch] lucaskabela/func_under_decomp -> origin/lucaskabela/func_under_decomp 2025-12-04T09:23:32.8675978Z * [new branch] lucaskabela/functional_in_dynamo -> origin/lucaskabela/functional_in_dynamo 2025-12-04T09:23:32.8676580Z * [new branch] lucaskabela/install_params_as_graph_attr -> origin/lucaskabela/install_params_as_graph_attr 2025-12-04T09:23:32.8678388Z * [new branch] lucaskabela/parameters_as_graph_attr -> origin/lucaskabela/parameters_as_graph_attr 2025-12-04T09:23:32.8678677Z * [new branch] lucaskabela/remove_aot_dispatcher_metadata -> origin/lucaskabela/remove_aot_dispatcher_metadata 2025-12-04T09:23:32.8679227Z * [new branch] lucaskabela/rnn_decomp -> origin/lucaskabela/rnn_decomp 2025-12-04T09:23:32.8680706Z * [new branch] lucaskabela/typing_backends -> origin/lucaskabela/typing_backends 2025-12-04T09:23:32.8681072Z * [new branch] lucaskabela/typing_ctx_manager -> origin/lucaskabela/typing_ctx_manager 2025-12-04T09:23:32.8681340Z * [new branch] lucaskabela/typing_nn_module -> origin/lucaskabela/typing_nn_module 2025-12-04T09:23:32.8681939Z * [new branch] lucaskabela/typing_user_defined -> origin/lucaskabela/typing_user_defined 2025-12-04T09:23:32.8683157Z * [new branch] lucaskabela/typing_variables -> origin/lucaskabela/typing_variables 2025-12-04T09:23:32.8683531Z * [new branch] lucaskabela/typing_variables_dicts -> origin/lucaskabela/typing_variables_dicts 2025-12-04T09:23:32.8684071Z * [new branch] lucaskabela/typing_variables_functions -> origin/lucaskabela/typing_variables_functions 2025-12-04T09:23:32.8687382Z * [new branch] lucaskabela/typing_variables_lists -> origin/lucaskabela/typing_variables_lists 2025-12-04T09:23:32.8687718Z * [new branch] lw/torch_box_by_ref -> origin/lw/torch_box_by_ref 2025-12-04T09:23:32.8687924Z * [new branch] main -> origin/main 2025-12-04T09:23:32.8688351Z * [new branch] malfet-patch-1 -> origin/malfet-patch-1 2025-12-04T09:23:32.8689007Z * [new branch] malfet-patch-2 -> origin/malfet-patch-2 2025-12-04T09:23:32.8689167Z * [new branch] malfet-patch-3 -> origin/malfet-patch-3 2025-12-04T09:23:32.8689805Z * [new branch] malfet-patch-4 -> origin/malfet-patch-4 2025-12-04T09:23:32.8690472Z * [new branch] malfet-patch-5 -> origin/malfet-patch-5 2025-12-04T09:23:32.8693897Z * [new branch] malfet-patch-6 -> origin/malfet-patch-6 2025-12-04T09:23:32.8694219Z * [new branch] malfet-patch-7 -> origin/malfet-patch-7 2025-12-04T09:23:32.8694407Z * [new branch] malfet-patch-8 -> origin/malfet-patch-8 2025-12-04T09:23:32.8694586Z * [new branch] malfet/add-3.14-ci -> origin/malfet/add-3.14-ci 2025-12-04T09:23:32.8694926Z * [new branch] malfet/be-do-not-make-typos-in-build-artifacts -> origin/malfet/be-do-not-make-typos-in-build-artifacts 2025-12-04T09:23:32.8695336Z * [new branch] malfet/be-move-more-settings-to-checkout-pytorch -> origin/malfet/be-move-more-settings-to-checkout-pytorch 2025-12-04T09:23:32.8695992Z * [new branch] malfet/be-remove-misisng-neon-headers -> origin/malfet/be-remove-misisng-neon-headers 2025-12-04T09:23:32.8698688Z * [new branch] malfet/mps-implement-col2im -> origin/malfet/mps-implement-col2im 2025-12-04T09:23:32.8699069Z * [new branch] manuel/aoti_metal_shimify-thread_safe -> origin/manuel/aoti_metal_shimify-thread_safe 2025-12-04T09:23:32.8699336Z * [new branch] manuel/inductor_link_openmp -> origin/manuel/inductor_link_openmp 2025-12-04T09:23:32.8699508Z * [new branch] masnesral/metaconda -> origin/masnesral/metaconda 2025-12-04T09:23:32.8699913Z * [new branch] mem_profiler_flaky_fix -> origin/mem_profiler_flaky_fix 2025-12-04T09:23:32.8701276Z * [new branch] mem_profiler_stack_trace -> origin/mem_profiler_stack_trace 2025-12-04T09:23:32.8701590Z * [new branch] memory_profiler_stack -> origin/memory_profiler_stack 2025-12-04T09:23:32.8704553Z * [new branch] metascroy-patch-1 -> origin/metascroy-patch-1 2025-12-04T09:23:32.8704880Z * [new branch] mingw_posix -> origin/mingw_posix 2025-12-04T09:23:32.8705115Z * [new branch] mlazos/S429861-debug -> origin/mlazos/S429861-debug 2025-12-04T09:23:32.8705320Z * [new branch] mlazos/aa -> origin/mlazos/aa 2025-12-04T09:23:32.8705476Z * [new branch] mlazos/acts -> origin/mlazos/acts 2025-12-04T09:23:32.8705975Z * [new branch] mlazos/arg-renames -> origin/mlazos/arg-renames 2025-12-04T09:23:32.8707070Z * [new branch] mlazos/bad-cudagraphs -> origin/mlazos/bad-cudagraphs 2025-12-04T09:23:32.8707745Z * [new branch] mlazos/baseline-graph-breaks -> origin/mlazos/baseline-graph-breaks 2025-12-04T09:23:32.8708147Z * [new branch] mlazos/beta-tensor -> origin/mlazos/beta-tensor 2025-12-04T09:23:32.8708730Z * [new branch] mlazos/buffers -> origin/mlazos/buffers 2025-12-04T09:23:32.8709218Z * [new branch] mlazos/buffers2 -> origin/mlazos/buffers2 2025-12-04T09:23:32.8710385Z * [new branch] mlazos/buffers3 -> origin/mlazos/buffers3 2025-12-04T09:23:32.8710928Z * [new branch] mlazos/bwd -> origin/mlazos/bwd 2025-12-04T09:23:32.8711536Z * [new branch] mlazos/combo-test -> origin/mlazos/combo-test 2025-12-04T09:23:32.8712459Z * [new branch] mlazos/ctx-cleanup -> origin/mlazos/ctx-cleanup 2025-12-04T09:23:32.8713026Z * [new branch] mlazos/cuda-cmd-log -> origin/mlazos/cuda-cmd-log 2025-12-04T09:23:32.8714365Z * [new branch] mlazos/cudagraph-tests -> origin/mlazos/cudagraph-tests 2025-12-04T09:23:32.8714799Z * [new branch] mlazos/cudagraphs-measurement -> origin/mlazos/cudagraphs-measurement 2025-12-04T09:23:32.8717587Z * [new branch] mlazos/cutlass-test -> origin/mlazos/cutlass-test 2025-12-04T09:23:32.8717991Z * [new branch] mlazos/cutlass-topo-bug -> origin/mlazos/cutlass-topo-bug 2025-12-04T09:23:32.8718174Z * [new branch] mlazos/dataclass-proxy -> origin/mlazos/dataclass-proxy 2025-12-04T09:23:32.8718310Z * [new branch] mlazos/dc-attrs -> origin/mlazos/dc-attrs 2025-12-04T09:23:32.8718469Z * [new branch] mlazos/dc-helion -> origin/mlazos/dc-helion 2025-12-04T09:23:32.8719011Z * [new branch] mlazos/dict-fix -> origin/mlazos/dict-fix 2025-12-04T09:23:32.8720725Z * [new branch] mlazos/disable-tf -> origin/mlazos/disable-tf 2025-12-04T09:23:32.8720888Z * [new branch] mlazos/dupe-fix -> origin/mlazos/dupe-fix 2025-12-04T09:23:32.8721250Z * [new branch] mlazos/dyn-batch -> origin/mlazos/dyn-batch 2025-12-04T09:23:32.8722878Z * [new branch] mlazos/evt -> origin/mlazos/evt 2025-12-04T09:23:32.8723089Z * [new branch] mlazos/extract-examples -> origin/mlazos/extract-examples 2025-12-04T09:23:32.8723248Z * [new branch] mlazos/foreach-op -> origin/mlazos/foreach-op 2025-12-04T09:23:32.8724118Z * [new branch] mlazos/fp8 -> origin/mlazos/fp8 2025-12-04T09:23:32.8726763Z * [new branch] mlazos/fp8-bias -> origin/mlazos/fp8-bias 2025-12-04T09:23:32.8726960Z * [new branch] mlazos/fp8-bias-fusion -> origin/mlazos/fp8-bias-fusion 2025-12-04T09:23:32.8727169Z * [new branch] mlazos/fp8-fixes -> origin/mlazos/fp8-fixes 2025-12-04T09:23:32.8727325Z * [new branch] mlazos/freezing -> origin/mlazos/freezing 2025-12-04T09:23:32.8727527Z * [new branch] mlazos/h-comp -> origin/mlazos/h-comp 2025-12-04T09:23:32.8733539Z * [new branch] mlazos/h-comp2 -> origin/mlazos/h-comp2 2025-12-04T09:23:32.8733740Z * [new branch] mlazos/hash-hop -> origin/mlazos/hash-hop 2025-12-04T09:23:32.8733867Z * [new branch] mlazos/hc -> origin/mlazos/hc 2025-12-04T09:23:32.8734005Z * [new branch] mlazos/hc-cycles -> origin/mlazos/hc-cycles 2025-12-04T09:23:32.8734148Z * [new branch] mlazos/hc-fixes -> origin/mlazos/hc-fixes 2025-12-04T09:23:32.8734275Z * [new branch] mlazos/hc-fixes3 -> origin/mlazos/hc-fixes3 2025-12-04T09:23:32.8734409Z * [new branch] mlazos/hc-fixes4 -> origin/mlazos/hc-fixes4 2025-12-04T09:23:32.8734552Z * [new branch] mlazos/hc-hf -> origin/mlazos/hc-hf 2025-12-04T09:23:32.8734828Z * [new branch] mlazos/hc-mut -> origin/mlazos/hc-mut 2025-12-04T09:23:32.8734967Z * [new branch] mlazos/hc10 -> origin/mlazos/hc10 2025-12-04T09:23:32.8735157Z * [new branch] mlazos/hc11 -> origin/mlazos/hc11 2025-12-04T09:23:32.8735289Z * [new branch] mlazos/hc12 -> origin/mlazos/hc12 2025-12-04T09:23:32.8735464Z * [new branch] mlazos/hc13 -> origin/mlazos/hc13 2025-12-04T09:23:32.8740332Z * [new branch] mlazos/hc14 -> origin/mlazos/hc14 2025-12-04T09:23:32.8740618Z * [new branch] mlazos/hc15 -> origin/mlazos/hc15 2025-12-04T09:23:32.8740749Z * [new branch] mlazos/hc2 -> origin/mlazos/hc2 2025-12-04T09:23:32.8741192Z * [new branch] mlazos/hc4 -> origin/mlazos/hc4 2025-12-04T09:23:32.8741320Z * [new branch] mlazos/hc5 -> origin/mlazos/hc5 2025-12-04T09:23:32.8741467Z * [new branch] mlazos/hc6 -> origin/mlazos/hc6 2025-12-04T09:23:32.8741585Z * [new branch] mlazos/hc7 -> origin/mlazos/hc7 2025-12-04T09:23:32.8741820Z * [new branch] mlazos/hc8 -> origin/mlazos/hc8 2025-12-04T09:23:32.8742012Z * [new branch] mlazos/hc9 -> origin/mlazos/hc9 2025-12-04T09:23:32.8742165Z * [new branch] mlazos/hc_baseline2 -> origin/mlazos/hc_baseline2 2025-12-04T09:23:32.8742418Z * [new branch] mlazos/inductor-streams -> origin/mlazos/inductor-streams 2025-12-04T09:23:32.8742577Z * [new branch] mlazos/main -> origin/mlazos/main 2025-12-04T09:23:32.8742701Z * [new branch] mlazos/mcg2 -> origin/mlazos/mcg2 2025-12-04T09:23:32.8747599Z * [new branch] mlazos/meta-guards -> origin/mlazos/meta-guards 2025-12-04T09:23:32.8747974Z * [new branch] mlazos/mlazos/foreach-map-adam -> origin/mlazos/mlazos/foreach-map-adam 2025-12-04T09:23:32.8748272Z * [new branch] mlazos/mlazos/tf-mode-backup -> origin/mlazos/mlazos/tf-mode-backup 2025-12-04T09:23:32.8748423Z * [new branch] mlazos/mod-fix -> origin/mlazos/mod-fix 2025-12-04T09:23:32.8748649Z * [new branch] mlazos/mode-fix -> origin/mlazos/mode-fix 2025-12-04T09:23:32.8749879Z * [new branch] mlazos/offsets -> origin/mlazos/offsets 2025-12-04T09:23:32.8750052Z * [new branch] mlazos/overguarding -> origin/mlazos/overguarding 2025-12-04T09:23:32.8750304Z * [new branch] mlazos/proxy-ctors -> origin/mlazos/proxy-ctors 2025-12-04T09:23:32.8750446Z * [new branch] mlazos/quant-fix -> origin/mlazos/quant-fix 2025-12-04T09:23:32.8750708Z * [new branch] mlazos/resnet-fix -> origin/mlazos/resnet-fix 2025-12-04T09:23:32.8750862Z * [new branch] mlazos/rm-buf-names -> origin/mlazos/rm-buf-names 2025-12-04T09:23:32.8751078Z * [new branch] mlazos/rm-code -> origin/mlazos/rm-code 2025-12-04T09:23:32.8751217Z * [new branch] mlazos/rm-spam -> origin/mlazos/rm-spam 2025-12-04T09:23:32.8751412Z * [new branch] mlazos/rtp -> origin/mlazos/rtp 2025-12-04T09:23:32.8751570Z * [new branch] mlazos/static-idx-dbg -> origin/mlazos/static-idx-dbg 2025-12-04T09:23:32.8751825Z * [new branch] mlazos/static-inputs-log -> origin/mlazos/static-inputs-log 2025-12-04T09:23:32.8751960Z * [new branch] mlazos/stests -> origin/mlazos/stests 2025-12-04T09:23:32.8753164Z * [new branch] mlazos/stream-ops -> origin/mlazos/stream-ops 2025-12-04T09:23:32.8753797Z * [new branch] mlazos/td-fix2 -> origin/mlazos/td-fix2 2025-12-04T09:23:32.8754042Z * [new branch] mlazos/tensor-hasattr2 -> origin/mlazos/tensor-hasattr2 2025-12-04T09:23:32.8758433Z * [new branch] mlazos/test -> origin/mlazos/test 2025-12-04T09:23:32.8758785Z * [new branch] mlazos/tf-mode -> origin/mlazos/tf-mode 2025-12-04T09:23:32.8759008Z * [new branch] mlazos/tf-mode-backup2 -> origin/mlazos/tf-mode-backup2 2025-12-04T09:23:32.8759257Z * [new branch] mlazos/tf-mode-reland -> origin/mlazos/tf-mode-reland 2025-12-04T09:23:32.8759442Z * [new branch] mlazos/tf-mode-reland2 -> origin/mlazos/tf-mode-reland2 2025-12-04T09:23:32.8760046Z * [new branch] mlazos/tf-mode-reland3 -> origin/mlazos/tf-mode-reland3 2025-12-04T09:23:32.8760394Z * [new branch] mlazos/triton-no-epi -> origin/mlazos/triton-no-epi 2025-12-04T09:23:32.8760542Z * [new branch] mlazos/tune-proto -> origin/mlazos/tune-proto 2025-12-04T09:23:32.8760830Z * [new branch] mlazos/tuple-fixes -> origin/mlazos/tuple-fixes 2025-12-04T09:23:32.8765012Z * [new branch] mlazos/tuple-fixes2 -> origin/mlazos/tuple-fixes2 2025-12-04T09:23:32.8765226Z * [new branch] mlazos/tuple-handling -> origin/mlazos/tuple-handling 2025-12-04T09:23:32.8765414Z * [new branch] mlazos/user-stream-base -> origin/mlazos/user-stream-base 2025-12-04T09:23:32.8765586Z * [new branch] mlazos/user-streams -> origin/mlazos/user-streams 2025-12-04T09:23:32.8765781Z * [new branch] mlazos/user-streams-backup -> origin/mlazos/user-streams-backup 2025-12-04T09:23:32.8767351Z * [new branch] mlazos/user-streams-backup2 -> origin/mlazos/user-streams-backup2 2025-12-04T09:23:32.8767698Z * [new branch] mlazos/vary-beta -> origin/mlazos/vary-beta 2025-12-04T09:23:32.8767891Z * [new branch] mlazos/vary-beta2 -> origin/mlazos/vary-beta2 2025-12-04T09:23:32.8768069Z * [new branch] mlazos/weird-perf1 -> origin/mlazos/weird-perf1 2025-12-04T09:23:32.8768296Z * [new branch] mm_out_dtype_compile -> origin/mm_out_dtype_compile 2025-12-04T09:23:32.8773469Z * [new branch] module-shim -> origin/module-shim 2025-12-04T09:23:32.8773772Z * [new branch] move_config -> origin/move_config 2025-12-04T09:23:32.8773928Z * [new branch] msaroufim/reduce -> origin/msaroufim/reduce 2025-12-04T09:23:32.8774066Z * [new branch] mtia/basic-cmake -> origin/mtia/basic-cmake 2025-12-04T09:23:32.8774379Z * [new branch] mwizak/fix-triton-block-shape -> origin/mwizak/fix-triton-block-shape 2025-12-04T09:23:32.8774925Z * [new branch] my_varlen_backup -> origin/my_varlen_backup 2025-12-04T09:23:32.8775112Z * [new branch] nativert_num_outputs -> origin/nativert_num_outputs 2025-12-04T09:23:32.8775259Z * [new branch] new-codegen -> origin/new-codegen 2025-12-04T09:23:32.8775407Z * [new branch] newtest-base -> origin/newtest-base 2025-12-04T09:23:32.8775560Z * [new branch] ngimel/addmm_dtype -> origin/ngimel/addmm_dtype 2025-12-04T09:23:32.8775838Z * [new branch] ngimel/div_inv -> origin/ngimel/div_inv 2025-12-04T09:23:32.8776272Z * [new branch] ngimel/error_index_list -> origin/ngimel/error_index_list 2025-12-04T09:23:32.8777058Z * [new branch] ngimel/gather_grid -> origin/ngimel/gather_grid 2025-12-04T09:23:32.8777470Z * [new branch] ngimel/gather_grid_release -> origin/ngimel/gather_grid_release 2025-12-04T09:23:32.8780157Z * [new branch] ngimel/gg_new -> origin/ngimel/gg_new 2025-12-04T09:23:32.8780476Z * [new branch] ngimel/hostalloc -> origin/ngimel/hostalloc 2025-12-04T09:23:32.8780663Z * [new branch] ngimel/storage_id -> origin/ngimel/storage_id 2025-12-04T09:23:32.8780823Z * [new branch] nightly -> origin/nightly 2025-12-04T09:23:32.8782391Z * [new branch] nikitaved/addmm_1_rowcol_lt_path_check -> origin/nikitaved/addmm_1_rowcol_lt_path_check 2025-12-04T09:23:32.8782788Z * [new branch] nikitaved/addmm_epilogue_fusions_2d_bias -> origin/nikitaved/addmm_epilogue_fusions_2d_bias 2025-12-04T09:23:32.8783110Z * [new branch] nikitaved/addmm_epilogue_fusions_inductor -> origin/nikitaved/addmm_epilogue_fusions_inductor 2025-12-04T09:23:32.8784867Z * [new branch] nikitaved/addmm_epilogue_fusions_scratch -> origin/nikitaved/addmm_epilogue_fusions_scratch 2025-12-04T09:23:32.8785424Z * [new branch] nikitaved/grad_addmm_epilogue_fusions -> origin/nikitaved/grad_addmm_epilogue_fusions 2025-12-04T09:23:32.8785735Z * [new branch] nikitaved/simpler_can_use_32bit_index -> origin/nikitaved/simpler_can_use_32bit_index 2025-12-04T09:23:32.8786241Z * [new branch] nikitaved/test -> origin/nikitaved/test 2025-12-04T09:23:32.8787886Z * [new branch] nmacchioni-perf-test-async-autotune -> origin/nmacchioni-perf-test-async-autotune 2025-12-04T09:23:32.8788077Z * [new branch] no_distributed_log_spew -> origin/no_distributed_log_spew 2025-12-04T09:23:32.8788492Z * [new branch] nofun-hack -> origin/nofun-hack 2025-12-04T09:23:32.8789856Z * [new branch] norm_bench -> origin/norm_bench 2025-12-04T09:23:32.8790326Z * [new branch] nullplay/fuse_matmul -> origin/nullplay/fuse_matmul 2025-12-04T09:23:32.8790960Z * [new branch] nullplay_fuse_matmul -> origin/nullplay_fuse_matmul 2025-12-04T09:23:32.8791920Z * [new branch] optimizer_test -> origin/optimizer_test 2025-12-04T09:23:32.8796513Z * [new branch] orig/release/1.10 -> origin/orig/release/1.10 2025-12-04T09:23:32.8796848Z * [new branch] orig/release/1.11 -> origin/orig/release/1.11 2025-12-04T09:23:32.8797062Z * [new branch] orig/release/1.12 -> origin/orig/release/1.12 2025-12-04T09:23:32.8797242Z * [new branch] orig/release/1.13 -> origin/orig/release/1.13 2025-12-04T09:23:32.8797396Z * [new branch] orig/release/1.6 -> origin/orig/release/1.6 2025-12-04T09:23:32.8797526Z * [new branch] orig/release/1.7 -> origin/orig/release/1.7 2025-12-04T09:23:32.8797766Z * [new branch] orig/release/1.8 -> origin/orig/release/1.8 2025-12-04T09:23:32.8799851Z * [new branch] orig/release/1.9 -> origin/orig/release/1.9 2025-12-04T09:23:32.8805863Z * [new branch] orig/release/2.0 -> origin/orig/release/2.0 2025-12-04T09:23:32.8806202Z * [new branch] orig/release/2.1 -> origin/orig/release/2.1 2025-12-04T09:23:32.8806390Z * [new branch] orig/release/2.2 -> origin/orig/release/2.2 2025-12-04T09:23:32.8806586Z * [new branch] orig/release/2.3 -> origin/orig/release/2.3 2025-12-04T09:23:32.8806731Z * [new branch] orig/release/2.4 -> origin/orig/release/2.4 2025-12-04T09:23:32.8806934Z * [new branch] orig/release/2.5 -> origin/orig/release/2.5 2025-12-04T09:23:32.8807543Z * [new branch] orig/release/2.6 -> origin/orig/release/2.6 2025-12-04T09:23:32.8807705Z * [new branch] orig/release/2.7 -> origin/orig/release/2.7 2025-12-04T09:23:32.8807858Z * [new branch] orig/release/2.8 -> origin/orig/release/2.8 2025-12-04T09:23:32.8807983Z * [new branch] orig/release/2.9 -> origin/orig/release/2.9 2025-12-04T09:23:32.8808156Z * [new branch] origin/gh/fxdawnn/1/base -> origin/origin/gh/fxdawnn/1/base 2025-12-04T09:23:32.8808473Z * [new branch] origin/gh/fxdawnn/1/orig -> origin/origin/gh/fxdawnn/1/orig 2025-12-04T09:23:32.8812888Z * [new branch] origin/gh/zpcore/14/orig -> origin/origin/gh/zpcore/14/orig 2025-12-04T09:23:32.8813221Z * [new branch] oulgen-patch-1 -> origin/oulgen-patch-1 2025-12-04T09:23:32.8813382Z * [new branch] oulgen-patch-2 -> origin/oulgen-patch-2 2025-12-04T09:23:32.8813517Z * [new branch] oulgen-patch-3 -> origin/oulgen-patch-3 2025-12-04T09:23:32.8813766Z * [new branch] oulgen-patch-4 -> origin/oulgen-patch-4 2025-12-04T09:23:32.8815179Z * [new branch] padded-tensor -> origin/padded-tensor 2025-12-04T09:23:32.8815685Z * [new branch] pca2 -> origin/pca2 2025-12-04T09:23:32.8815867Z * [new branch] per_channel_backup -> origin/per_channel_backup 2025-12-04T09:23:32.8816225Z * [new branch] perf_ops -> origin/perf_ops 2025-12-04T09:23:32.8816364Z * [new branch] perf_ops_2_9 -> origin/perf_ops_2_9 2025-12-04T09:23:32.8816687Z * [new branch] pianpwk-patch-1 -> origin/pianpwk-patch-1 2025-12-04T09:23:32.8817120Z * [new branch] pianpwk/__draft_debug_mode -> origin/pianpwk/__draft_debug_mode 2025-12-04T09:23:32.8818174Z * [new branch] pianpwk/_debug_mode_for_triton_draft -> origin/pianpwk/_debug_mode_for_triton_draft 2025-12-04T09:23:32.8821299Z * [new branch] pianpwk/_debug_nn_module_compile -> origin/pianpwk/_debug_nn_module_compile 2025-12-04T09:23:32.8821521Z * [new branch] pianpwk/_draft_triton_11_3 -> origin/pianpwk/_draft_triton_11_3 2025-12-04T09:23:32.8821693Z * [new branch] pianpwk/_manual_bucket_draft -> origin/pianpwk/_manual_bucket_draft 2025-12-04T09:23:32.8821880Z * [new branch] pianpwk/_profile_w_dispatch_keys -> origin/pianpwk/_profile_w_dispatch_keys 2025-12-04T09:23:32.8822050Z * [new branch] pianpwk/_super_draft_debug_mode -> origin/pianpwk/_super_draft_debug_mode 2025-12-04T09:23:32.8822603Z * [new branch] pianpwk/_unbacked_local_shard_size -> origin/pianpwk/_unbacked_local_shard_size 2025-12-04T09:23:32.8823213Z * [new branch] pianpwk/anomaly_tb -> origin/pianpwk/anomaly_tb 2025-12-04T09:23:32.8823837Z * [new branch] pianpwk/auto_fx_annotate -> origin/pianpwk/auto_fx_annotate 2025-12-04T09:23:32.8824720Z * [new branch] pianpwk/backed_size_oblivious_export -> origin/pianpwk/backed_size_oblivious_export 2025-12-04T09:23:32.8825185Z * [new branch] pianpwk/bert_dynamic_perf -> origin/pianpwk/bert_dynamic_perf 2025-12-04T09:23:32.8825858Z * [new branch] pianpwk/debug_fwd_stack_traces -> origin/pianpwk/debug_fwd_stack_traces 2025-12-04T09:23:32.8826585Z * [new branch] pianpwk/debug_hash_tensor -> origin/pianpwk/debug_hash_tensor 2025-12-04T09:23:32.8827733Z * [new branch] pianpwk/debug_mode_annotate -> origin/pianpwk/debug_mode_annotate 2025-12-04T09:23:32.8828117Z * [new branch] pianpwk/debug_mode_defaults -> origin/pianpwk/debug_mode_defaults 2025-12-04T09:23:32.8828991Z * [new branch] pianpwk/debug_mode_hacks -> origin/pianpwk/debug_mode_hacks 2025-12-04T09:23:32.8829381Z * [new branch] pianpwk/debug_mode_opcall_refactor -> origin/pianpwk/debug_mode_opcall_refactor 2025-12-04T09:23:32.8830068Z * [new branch] pianpwk/debug_mode_show_ids -> origin/pianpwk/debug_mode_show_ids 2025-12-04T09:23:32.8830630Z * [new branch] pianpwk/debug_mode_triton -> origin/pianpwk/debug_mode_triton 2025-12-04T09:23:32.8831657Z * [new branch] pianpwk/debug_show_stack_trace -> origin/pianpwk/debug_show_stack_trace 2025-12-04T09:23:32.8832091Z * [new branch] pianpwk/debug_wait_on_collective -> origin/pianpwk/debug_wait_on_collective 2025-12-04T09:23:32.8833069Z * [new branch] pianpwk/debugmode_compile_tf -> origin/pianpwk/debugmode_compile_tf 2025-12-04T09:23:32.8833630Z * [new branch] pianpwk/dispatch_key_debugging_for_debug -> origin/pianpwk/dispatch_key_debugging_for_debug 2025-12-04T09:23:32.8837064Z * [new branch] pianpwk/draft_debug_mode_tfcompile -> origin/pianpwk/draft_debug_mode_tfcompile 2025-12-04T09:23:32.8837280Z * [new branch] pianpwk/draft_multikernel_nn -> origin/pianpwk/draft_multikernel_nn 2025-12-04T09:23:32.8837482Z * [new branch] pianpwk/draft_multikernel_status_10_5 -> origin/pianpwk/draft_multikernel_status_10_5 2025-12-04T09:23:32.8837828Z * [new branch] pianpwk/dtensor_custom_chunk -> origin/pianpwk/dtensor_custom_chunk 2025-12-04T09:23:32.8838017Z * [new branch] pianpwk/dtensor_unbacked_keypath -> origin/pianpwk/dtensor_unbacked_keypath 2025-12-04T09:23:32.8838243Z * [new branch] pianpwk/event_list_tree -> origin/pianpwk/event_list_tree 2025-12-04T09:23:32.8838763Z * [new branch] pianpwk/false_numel_refs -> origin/pianpwk/false_numel_refs 2025-12-04T09:23:32.8839320Z * [new branch] pianpwk/maybe_guard_rel -> origin/pianpwk/maybe_guard_rel 2025-12-04T09:23:32.8840232Z * [new branch] pianpwk/multikernel_hints_draft -> origin/pianpwk/multikernel_hints_draft 2025-12-04T09:23:32.8840721Z * [new branch] pianpwk/no_size_oblivious_slice_scat -> origin/pianpwk/no_size_oblivious_slice_scat 2025-12-04T09:23:32.8841443Z * [new branch] pianpwk/oblivious_reshape_view_better -> origin/pianpwk/oblivious_reshape_view_better 2025-12-04T09:23:32.8841951Z * [new branch] pianpwk/pre_forward_hook -> origin/pianpwk/pre_forward_hook 2025-12-04T09:23:32.8844297Z * [new branch] pianpwk/skip_python_keys_alternate -> origin/pianpwk/skip_python_keys_alternate 2025-12-04T09:23:32.8844682Z * [new branch] pianpwk/skip_python_keys_in_guards -> origin/pianpwk/skip_python_keys_in_guards 2025-12-04T09:23:32.8844920Z * [new branch] pianpwk/sym_tokens_draft -> origin/pianpwk/sym_tokens_draft 2025-12-04T09:23:32.8845111Z * [new branch] pianpwk/symint_one_hot -> origin/pianpwk/symint_one_hot 2025-12-04T09:23:32.8845325Z * [new branch] pianpwk/test_pointwise_guard_or_false -> origin/pianpwk/test_pointwise_guard_or_false 2025-12-04T09:23:32.8846330Z * [new branch] pianpwk/totally_draft_sym_wrap -> origin/pianpwk/totally_draft_sym_wrap 2025-12-04T09:23:32.8846645Z * [new branch] pianpwk/try_dumb_stuff -> origin/pianpwk/try_dumb_stuff 2025-12-04T09:23:32.8850588Z * [new branch] pianpwk/try_dumb_stuff_2 -> origin/pianpwk/try_dumb_stuff_2 2025-12-04T09:23:32.8850929Z * [new branch] pianpwk/unbacked_dtensor_mm -> origin/pianpwk/unbacked_dtensor_mm 2025-12-04T09:23:32.8851191Z * [new branch] pianpwk/unbacked_tracing_12_2 -> origin/pianpwk/unbacked_tracing_12_2 2025-12-04T09:23:32.8851433Z * [new branch] pianpwk/user_symints -> origin/pianpwk/user_symints 2025-12-04T09:23:32.8851594Z * [new branch] pianpwk/wan21_reshape -> origin/pianpwk/wan21_reshape 2025-12-04T09:23:32.8851839Z * [new branch] piz/fix_partial_backward_1112 -> origin/piz/fix_partial_backward_1112 2025-12-04T09:23:32.8851990Z * [new branch] piz/prop_cache_clean -> origin/piz/prop_cache_clean 2025-12-04T09:23:32.8852139Z * [new branch] pool-separate -> origin/pool-separate 2025-12-04T09:23:32.8853526Z * [new branch] pr-156087 -> origin/pr-156087 2025-12-04T09:23:32.8853852Z * [new branch] pr/131860 -> origin/pr/131860 2025-12-04T09:23:32.8856276Z * [new branch] predispatch_to -> origin/predispatch_to 2025-12-04T09:23:32.8856596Z * [new branch] protect-c17 -> origin/protect-c17 2025-12-04T09:23:32.8856835Z * [new branch] pt-opt-cuda3 -> origin/pt-opt-cuda3 2025-12-04T09:23:32.8857068Z * [new branch] python_compiled_autograd -> origin/python_compiled_autograd 2025-12-04T09:23:32.8858872Z * [new branch] q1l1/fix_device_moved_constant_type_unknown -> origin/q1l1/fix_device_moved_constant_type_unknown 2025-12-04T09:23:32.8859292Z * [new branch] q1l1/fix_wrong_default_type_for_kernel_call_args -> origin/q1l1/fix_wrong_default_type_for_kernel_call_args 2025-12-04T09:23:32.8861443Z * [new branch] qchip/export-D54134695 -> origin/qchip/export-D54134695 2025-12-04T09:23:32.8861759Z * [new branch] quote-pytest_cache -> origin/quote-pytest_cache 2025-12-04T09:23:32.8862144Z * [new branch] reland-accgrad-stream-warn -> origin/reland-accgrad-stream-warn 2025-12-04T09:23:32.8867430Z * [new branch] release/1.10 -> origin/release/1.10 2025-12-04T09:23:32.8867758Z * [new branch] release/1.11 -> origin/release/1.11 2025-12-04T09:23:32.8867902Z * [new branch] release/1.12 -> origin/release/1.12 2025-12-04T09:23:32.8868049Z * [new branch] release/1.13 -> origin/release/1.13 2025-12-04T09:23:32.8868183Z * [new branch] release/1.4 -> origin/release/1.4 2025-12-04T09:23:32.8868318Z * [new branch] release/1.4.1 -> origin/release/1.4.1 2025-12-04T09:23:32.8868903Z * [new branch] release/1.5 -> origin/release/1.5 2025-12-04T09:23:32.8869151Z * [new branch] release/1.6 -> origin/release/1.6 2025-12-04T09:23:32.8869272Z * [new branch] release/1.7 -> origin/release/1.7 2025-12-04T09:23:32.8870088Z * [new branch] release/1.8 -> origin/release/1.8 2025-12-04T09:23:32.8870535Z * [new branch] release/1.9 -> origin/release/1.9 2025-12-04T09:23:32.8873426Z * [new branch] release/2.0 -> origin/release/2.0 2025-12-04T09:23:32.8880348Z * [new branch] release/2.1 -> origin/release/2.1 2025-12-04T09:23:32.8882488Z * [new branch] release/2.2 -> origin/release/2.2 2025-12-04T09:23:32.8882723Z * [new branch] release/2.3 -> origin/release/2.3 2025-12-04T09:23:32.8887820Z * [new branch] release/2.4 -> origin/release/2.4 2025-12-04T09:23:32.8889754Z * [new branch] release/2.5 -> origin/release/2.5 2025-12-04T09:23:32.8890007Z * [new branch] release/2.6 -> origin/release/2.6 2025-12-04T09:23:32.8894793Z * [new branch] release/2.7 -> origin/release/2.7 2025-12-04T09:23:32.8897115Z * [new branch] release/2.8 -> origin/release/2.8 2025-12-04T09:23:32.8901914Z * [new branch] release/2.9 -> origin/release/2.9 2025-12-04T09:23:32.8902098Z * [new branch] release_notes -> origin/release_notes 2025-12-04T09:23:32.8902249Z * [new branch] remove_pyinterpreter -> origin/remove_pyinterpreter 2025-12-04T09:23:32.8902473Z * [new branch] replace-pytorch-labs-20250812-195836 -> origin/replace-pytorch-labs-20250812-195836 2025-12-04T09:23:32.8902690Z * [new branch] replace-pytorch-labs-20250812-200248 -> origin/replace-pytorch-labs-20250812-200248 2025-12-04T09:23:32.8902903Z * [new branch] replace-pytorch-labs-20250812-200324 -> origin/replace-pytorch-labs-20250812-200324 2025-12-04T09:23:32.8903104Z * [new branch] replace-pytorch-labs-20250812-204020 -> origin/replace-pytorch-labs-20250812-204020 2025-12-04T09:23:32.8903333Z * [new branch] revert-131069-gh/krzysztofjordan/1/head -> origin/revert-131069-gh/krzysztofjordan/1/head 2025-12-04T09:23:32.8903526Z * [new branch] revert-131469-gh/andrewor14/51/head -> origin/revert-131469-gh/andrewor14/51/head 2025-12-04T09:23:32.8903707Z * [new branch] revert-152361-gh/fadara01/1/head -> origin/revert-152361-gh/fadara01/1/head 2025-12-04T09:23:32.8903879Z * [new branch] revert-156870-gh/skarjala/3/head -> origin/revert-156870-gh/skarjala/3/head 2025-12-04T09:23:32.8904163Z * [new branch] revert-157914-cherry-pick-157503-by-pytorch_bot_bot_ -> origin/revert-157914-cherry-pick-157503-by-pytorch_bot_bot_ 2025-12-04T09:23:32.8904481Z * [new branch] revert-hoo-invoke-subgraph -> origin/revert-hoo-invoke-subgraph 2025-12-04T09:23:32.8904651Z * [new branch] revert_always_build_distributed -> origin/revert_always_build_distributed 2025-12-04T09:23:32.8904784Z * [new branch] rms_norm_patch -> origin/rms_norm_patch 2025-12-04T09:23:32.8904992Z * [new branch] ruisi/fix_all_to_all_estimation -> origin/ruisi/fix_all_to_all_estimation 2025-12-04T09:23:32.8905149Z * [new branch] ruisi/fix_comm_estimation -> origin/ruisi/fix_comm_estimation 2025-12-04T09:23:32.8905325Z * [new branch] ruisi/fix_dynamic_shape_estimation -> origin/ruisi/fix_dynamic_shape_estimation 2025-12-04T09:23:32.8905499Z * [new branch] ruisi/fix_llama3_autobucketing -> origin/ruisi/fix_llama3_autobucketing 2025-12-04T09:23:32.8905831Z * [new branch] ruisi/fix_manual_bucketing_ep_pass -> origin/ruisi/fix_manual_bucketing_ep_pass 2025-12-04T09:23:32.8905990Z * [new branch] ruisi/manual_bucket_pass -> origin/ruisi/manual_bucket_pass 2025-12-04T09:23:32.8906235Z * [new branch] ryanguo99/cleanup-dynamo-expected-failures -> origin/ryanguo99/cleanup-dynamo-expected-failures 2025-12-04T09:23:32.8906401Z * [new branch] ryanguo99/fix-closure-var -> origin/ryanguo99/fix-closure-var 2025-12-04T09:23:32.8906544Z * [new branch] rzou/faketensor_bench -> origin/rzou/faketensor_bench 2025-12-04T09:23:32.8906663Z * [new branch] rzou/njt -> origin/rzou/njt 2025-12-04T09:23:32.8906774Z * [new branch] rzou/pca -> origin/rzou/pca 2025-12-04T09:23:32.8906896Z * [new branch] rzou/realprop -> origin/rzou/realprop 2025-12-04T09:23:32.8907018Z * [new branch] samplevllm -> origin/samplevllm 2025-12-04T09:23:32.8907302Z * [new branch] sanchitintel/weird_thing_with_test_cpu_select_algorithm -> origin/sanchitintel/weird_thing_with_test_cpu_select_algorithm 2025-12-04T09:23:32.8907472Z * [new branch] sapling-pr-archive-SS-JIA -> origin/sapling-pr-archive-SS-JIA 2025-12-04T09:23:32.8907843Z * [new branch] sapling-pr-archive-tushar00jain -> origin/sapling-pr-archive-tushar00jain 2025-12-04T09:23:32.8908479Z * [new branch] save -> origin/save 2025-12-04T09:23:32.8908660Z * [new branch] scaled_mm -> origin/scaled_mm 2025-12-04T09:23:32.8908808Z * [new branch] scan_attempt -> origin/scan_attempt 2025-12-04T09:23:32.8908929Z * [new branch] sdym/2.5.1 -> origin/sdym/2.5.1 2025-12-04T09:23:32.8909158Z * [new branch] sekyondaMeta-dynamoconfig-fix -> origin/sekyondaMeta-dynamoconfig-fix 2025-12-04T09:23:32.8909334Z * [new branch] shengf/fx-xform-perf -> origin/shengf/fx-xform-perf 2025-12-04T09:23:32.8909541Z * [new branch] shoumikhin-patch-1 -> origin/shoumikhin-patch-1 2025-12-04T09:23:32.8909706Z * [new branch] solve-accuracy-fix -> origin/solve-accuracy-fix 2025-12-04T09:23:32.8910353Z * [new branch] some_rocm_inductor_skips -> origin/some_rocm_inductor_skips 2025-12-04T09:23:32.8910708Z * [new branch] soulitzer/stash-tls-ac -> origin/soulitzer/stash-tls-ac 2025-12-04T09:23:32.8910882Z * [new branch] sparse-mm-bf16-support -> origin/sparse-mm-bf16-support 2025-12-04T09:23:32.8911885Z * [new branch] starterTaskUpdate -> origin/starterTaskUpdate 2025-12-04T09:23:32.8912321Z * [new branch] suo -> origin/suo 2025-12-04T09:23:32.8918685Z * [new branch] sve-poc -> origin/sve-poc 2025-12-04T09:23:32.8920782Z * [new branch] switch-bn -> origin/switch-bn 2025-12-04T09:23:32.8921403Z * [new branch] sy_annotation_in_autograd_hop -> origin/sy_annotation_in_autograd_hop 2025-12-04T09:23:32.8926544Z * [new branch] sy_aot_eager_record -> origin/sy_aot_eager_record 2025-12-04T09:23:32.8931119Z * [new branch] sy_custom_bucketing -> origin/sy_custom_bucketing 2025-12-04T09:23:32.8936037Z * [new branch] sy_debug_mode_test -> origin/sy_debug_mode_test 2025-12-04T09:23:32.8937632Z * [new branch] sy_deserialize -> origin/sy_deserialize 2025-12-04T09:23:32.8937782Z * [new branch] sy_dump_gm_code -> origin/sy_dump_gm_code 2025-12-04T09:23:32.8937905Z * [new branch] sy_exp -> origin/sy_exp 2025-12-04T09:23:32.8938046Z * [new branch] sy_export_annotation -> origin/sy_export_annotation 2025-12-04T09:23:32.8938171Z * [new branch] sy_invoke_subgraph -> origin/sy_invoke_subgraph 2025-12-04T09:23:32.8938329Z * [new branch] sy_kernel_bw_name -> origin/sy_kernel_bw_name 2025-12-04T09:23:32.8938447Z * [new branch] sy_multi_arch -> origin/sy_multi_arch 2025-12-04T09:23:32.8938578Z * [new branch] sy_nn_module_stack -> origin/sy_nn_module_stack 2025-12-04T09:23:32.8938716Z * [new branch] sy_original_dtensor -> origin/sy_original_dtensor 2025-12-04T09:23:32.8938842Z * [new branch] sy_profiler_cia -> origin/sy_profiler_cia 2025-12-04T09:23:32.8938970Z * [new branch] symm_mem_sync -> origin/symm_mem_sync 2025-12-04T09:23:32.8939130Z * [new branch] sympy-bottleneck-repro -> origin/sympy-bottleneck-repro 2025-12-04T09:23:32.8939286Z * [new branch] tensordict_integration -> origin/tensordict_integration 2025-12-04T09:23:32.8939440Z * [new branch] test-move-conda-builds -> origin/test-move-conda-builds 2025-12-04T09:23:32.8939557Z * [new branch] test-old -> origin/test-old 2025-12-04T09:23:32.8939682Z * [new branch] test/bmm_heur -> origin/test/bmm_heur 2025-12-04T09:23:32.8939852Z * [new branch] tianren/customOp_autotune_fix -> origin/tianren/customOp_autotune_fix 2025-12-04T09:23:32.8940042Z * [new branch] tianren/customOp_enable_max_autotune -> origin/tianren/customOp_enable_max_autotune 2025-12-04T09:23:32.8940197Z * [new branch] tianren/customOp_fusion -> origin/tianren/customOp_fusion 2025-12-04T09:23:32.8940409Z * [new branch] tianren/customop_collectiveop_benchmark -> origin/tianren/customop_collectiveop_benchmark 2025-12-04T09:23:32.8940640Z * [new branch] tianren/customop_collectiveop_benchmark_fix -> origin/tianren/customop_collectiveop_benchmark_fix 2025-12-04T09:23:32.8940812Z * [new branch] tianren/customop_dynamic_config -> origin/tianren/customop_dynamic_config 2025-12-04T09:23:32.8940970Z * [new branch] tianren/dynamic_range_input -> origin/tianren/dynamic_range_input 2025-12-04T09:23:32.8941149Z * [new branch] tianren/dynamic_range_input_fix -> origin/tianren/dynamic_range_input_fix 2025-12-04T09:23:32.8941323Z * [new branch] tianren/dynamic_range_input_merge -> origin/tianren/dynamic_range_input_merge 2025-12-04T09:23:32.8941498Z * [new branch] tianren/flex_paged_attn_fix_temp -> origin/tianren/flex_paged_attn_fix_temp 2025-12-04T09:23:32.8941637Z * [new branch] tianren/fx_codegen_dump -> origin/tianren/fx_codegen_dump 2025-12-04T09:23:32.8941784Z * [new branch] tianren/symmetric_memory -> origin/tianren/symmetric_memory 2025-12-04T09:23:32.8941910Z * [new branch] tianren/test -> origin/tianren/test 2025-12-04T09:23:32.8942055Z * [new branch] tidy_performance_cyy -> origin/tidy_performance_cyy 2025-12-04T09:23:32.8942248Z * [new branch] tmp -> origin/tmp 2025-12-04T09:23:32.8942391Z * [new branch] torchtitan_ep -> origin/torchtitan_ep 2025-12-04T09:23:32.8942537Z * [new branch] torchtitan_integration -> origin/torchtitan_integration 2025-12-04T09:23:32.8942736Z * [new branch] trace_fsdp_torchtune_lora -> origin/trace_fsdp_torchtune_lora 2025-12-04T09:23:32.8942975Z * [new branch] traceable_fsdp_unit_tests -> origin/traceable_fsdp_unit_tests 2025-12-04T09:23:32.8947882Z * [new branch] tree_loop_vec_base -> origin/tree_loop_vec_base 2025-12-04T09:23:32.8948208Z * [new branch] triton_kernel -> origin/triton_kernel 2025-12-04T09:23:32.8948350Z * [new branch] tt_pkg_1908 -> origin/tt_pkg_1908 2025-12-04T09:23:32.8948495Z * [new branch] type_dec -> origin/type_dec 2025-12-04T09:23:32.8948711Z * [new branch] udate-sphinx-dependancies -> origin/udate-sphinx-dependancies 2025-12-04T09:23:32.8949087Z * [new branch] update-audio-commit-hash/17630256502-1803-1 -> origin/update-audio-commit-hash/17630256502-1803-1 2025-12-04T09:23:32.8949804Z * [new branch] update-audio-commit-hash/19087141161-1916-1 -> origin/update-audio-commit-hash/19087141161-1916-1 2025-12-04T09:23:32.8950066Z * [new branch] update-audio-commit-hash/19250643381-1929-1 -> origin/update-audio-commit-hash/19250643381-1929-1 2025-12-04T09:23:32.8950303Z * [new branch] update-audio-commit-hash/19397724337-1935-1 -> origin/update-audio-commit-hash/19397724337-1935-1 2025-12-04T09:23:32.8950520Z * [new branch] update-audio-commit-hash/19555670148-1941-1 -> origin/update-audio-commit-hash/19555670148-1941-1 2025-12-04T09:23:32.8950930Z * [new branch] update-audio-commit-hash/19750627930-1946-1 -> origin/update-audio-commit-hash/19750627930-1946-1 2025-12-04T09:23:32.8952150Z * [new branch] update-triton-commit-hash/13663274526-1487-2 -> origin/update-triton-commit-hash/13663274526-1487-2 2025-12-04T09:23:32.8953064Z * [new branch] update-vision-commit-hash/19087141161-1916-1 -> origin/update-vision-commit-hash/19087141161-1916-1 2025-12-04T09:23:32.8953642Z * [new branch] update-vision-commit-hash/19184897099-1925-1 -> origin/update-vision-commit-hash/19184897099-1925-1 2025-12-04T09:23:32.8954225Z * [new branch] update-vision-commit-hash/19250643381-1929-1 -> origin/update-vision-commit-hash/19250643381-1929-1 2025-12-04T09:23:32.8954833Z * [new branch] update-vision-commit-hash/19381328640-1934-1 -> origin/update-vision-commit-hash/19381328640-1934-1 2025-12-04T09:23:32.8957589Z * [new branch] update-vision-commit-hash/19485237164-1938-1 -> origin/update-vision-commit-hash/19485237164-1938-1 2025-12-04T09:23:32.8958016Z * [new branch] update-vllm-commit-hash/18451675449-1879-1 -> origin/update-vllm-commit-hash/18451675449-1879-1 2025-12-04T09:23:32.8958322Z * [new branch] update-vllm-dockerfile -> origin/update-vllm-dockerfile 2025-12-04T09:23:32.8962491Z * [new branch] update-xla-commit-hash/19224287370-211-1 -> origin/update-xla-commit-hash/19224287370-211-1 2025-12-04T09:23:32.8962911Z * [new branch] update-xla-commit-hash/19422028566-212-1 -> origin/update-xla-commit-hash/19422028566-212-1 2025-12-04T09:23:32.8963255Z * [new branch] update-xla-commit-hash/19626841311-213-1 -> origin/update-xla-commit-hash/19626841311-213-1 2025-12-04T09:23:32.8963579Z * [new branch] update_docs_torch_multinomial_issue#125388 -> origin/update_docs_torch_multinomial_issue#125388 2025-12-04T09:23:32.8964098Z * [new branch] update_operator_readme -> origin/update_operator_readme 2025-12-04T09:23:32.8964471Z * [new branch] update_slow_tests_1722488736 -> origin/update_slow_tests_1722488736 2025-12-04T09:23:32.8964630Z * [new branch] update_slow_tests_1722879173 -> origin/update_slow_tests_1722879173 2025-12-04T09:23:32.8964788Z * [new branch] update_slow_tests_1762155677 -> origin/update_slow_tests_1762155677 2025-12-04T09:23:32.8964984Z * [new branch] update_slow_tests_1763365283 -> origin/update_slow_tests_1763365283 2025-12-04T09:23:32.8965141Z * [new branch] update_submodule_FBGEMM -> origin/update_submodule_FBGEMM 2025-12-04T09:23:32.8965480Z * [new branch] update_submodule_kineto -> origin/update_submodule_kineto 2025-12-04T09:23:32.8966161Z * [new branch] update_submodule_tensorpipe -> origin/update_submodule_tensorpipe 2025-12-04T09:23:32.8966852Z * [new branch] upload-tests-for-autorevert -> origin/upload-tests-for-autorevert 2025-12-04T09:23:32.8967367Z * [new branch] v0.1.2 -> origin/v0.1.2 2025-12-04T09:23:32.8970071Z * [new branch] v1.0.1 -> origin/v1.0.1 2025-12-04T09:23:32.8970370Z * [new branch] v1.0.3 -> origin/v1.0.3 2025-12-04T09:23:32.8970494Z * [new branch] v1.1.0 -> origin/v1.1.0 2025-12-04T09:23:32.8970662Z * [new branch] v1.2.0 -> origin/v1.2.0 2025-12-04T09:23:32.8972435Z * [new branch] v1.3.0 -> origin/v1.3.0 2025-12-04T09:23:32.8972729Z * [new branch] v1.3.1 -> origin/v1.3.1 2025-12-04T09:23:32.8972870Z * [new branch] validate_fn -> origin/validate_fn 2025-12-04T09:23:32.8975288Z * [new branch] validations_2.6 -> origin/validations_2.6 2025-12-04T09:23:32.8975607Z * [new branch] validations_2.8 -> origin/validations_2.8 2025-12-04T09:23:32.8975838Z * [new branch] varlen-api -> origin/varlen-api 2025-12-04T09:23:32.8975995Z * [new branch] varlen-api-backup -> origin/varlen-api-backup 2025-12-04T09:23:32.8976501Z * [new branch] varlen_batch_invariance -> origin/varlen_batch_invariance 2025-12-04T09:23:32.8980618Z * [new branch] viable/strict -> origin/viable/strict 2025-12-04T09:23:32.8981036Z * [new branch] vishal9-team/dtensor_parallelism_toy -> origin/vishal9-team/dtensor_parallelism_toy 2025-12-04T09:23:32.8981291Z * [new branch] vllmbuildci -> origin/vllmbuildci 2025-12-04T09:23:32.8981420Z * [new branch] vllmpin -> origin/vllmpin 2025-12-04T09:23:32.8981678Z * [new branch] vscode-recommend-pyrefly -> origin/vscode-recommend-pyrefly 2025-12-04T09:23:32.8982308Z * [new branch] wdvr-patch-1 -> origin/wdvr-patch-1 2025-12-04T09:23:32.8982706Z * [new branch] wdvr/iss_145259 -> origin/wdvr/iss_145259 2025-12-04T09:23:32.8984269Z * [new branch] whc/pei -> origin/whc/pei 2025-12-04T09:23:32.8984588Z * [new branch] whc/pp_fix -> origin/whc/pp_fix 2025-12-04T09:23:32.8984965Z * [new branch] whc/sharding -> origin/whc/sharding 2025-12-04T09:23:32.8986260Z * [new branch] whc/sharding2 -> origin/whc/sharding2 2025-12-04T09:23:32.8986421Z * [new branch] whc/uneven -> origin/whc/uneven 2025-12-04T09:23:32.8988382Z * [new branch] whc/uneven-merge -> origin/whc/uneven-merge 2025-12-04T09:23:32.8988546Z * [new branch] win_warnings -> origin/win_warnings 2025-12-04T09:23:32.8988699Z * [new branch] windows_libtorch_free -> origin/windows_libtorch_free 2025-12-04T09:23:32.8989749Z * [new branch] xmfan-war -> origin/xmfan-war 2025-12-04T09:23:32.8990804Z * [new branch] xmfan/ca_0516 -> origin/xmfan/ca_0516 2025-12-04T09:23:32.8991084Z * [new branch] xmfan/ca_1051b93192 -> origin/xmfan/ca_1051b93192 2025-12-04T09:23:32.8992347Z * [new branch] xmfan/ca_1a722f62c248391fc4a542e8851a5559aa356ae8 -> origin/xmfan/ca_1a722f62c248391fc4a542e8851a5559aa356ae8 2025-12-04T09:23:32.8992494Z * [new branch] xmfan/ca_5a2be192d1 -> origin/xmfan/ca_5a2be192d1 2025-12-04T09:23:32.8993024Z * [new branch] xmfan/ca_9d59b516e9 -> origin/xmfan/ca_9d59b516e9 2025-12-04T09:23:32.8996872Z * [new branch] xmfan/ca_apr8 -> origin/xmfan/ca_apr8 2025-12-04T09:23:32.8997024Z * [new branch] xmfan/ca_base -> origin/xmfan/ca_base 2025-12-04T09:23:32.8997160Z * [new branch] xmfan/ca_dynamic -> origin/xmfan/ca_dynamic 2025-12-04T09:23:32.8997319Z * [new branch] xmfan/ca_fix_dyn -> origin/xmfan/ca_fix_dyn 2025-12-04T09:23:32.8997604Z * [new branch] xmfan/ca_fix_lowering -> origin/xmfan/ca_fix_lowering 2025-12-04T09:23:32.9002050Z * [new branch] xmfan/ca_fix_polyfills -> origin/xmfan/ca_fix_polyfills 2025-12-04T09:23:32.9004179Z * [new branch] xmfan/ca_jan3 -> origin/xmfan/ca_jan3 2025-12-04T09:23:32.9009278Z * [new branch] xmfan/ca_jun18 -> origin/xmfan/ca_jun18 2025-12-04T09:23:32.9011531Z * [new branch] xmfan/ca_jun24 -> origin/xmfan/ca_jun24 2025-12-04T09:23:32.9016324Z * [new branch] xmfan/ca_nested -> origin/xmfan/ca_nested 2025-12-04T09:23:32.9020754Z * [new branch] xmfan/ca_overhead -> origin/xmfan/ca_overhead 2025-12-04T09:23:32.9025013Z * [new branch] xmfan/ca_overhead_0eba7e5451 -> origin/xmfan/ca_overhead_0eba7e5451 2025-12-04T09:23:32.9025217Z * [new branch] xmfan/cacu_jun18 -> origin/xmfan/cacu_jun18 2025-12-04T09:23:32.9025349Z * [new branch] xmfan/cacu_jun19 -> origin/xmfan/cacu_jun19 2025-12-04T09:23:32.9025478Z * [new branch] xmfan/cacu_jun4 -> origin/xmfan/cacu_jun4 2025-12-04T09:23:32.9025648Z * [new branch] xmfan/disable_duck_shape -> origin/xmfan/disable_duck_shape 2025-12-04T09:23:32.9025826Z * [new branch] xmfan/fca_cpp_node_passthrough -> origin/xmfan/fca_cpp_node_passthrough 2025-12-04T09:23:32.9026094Z * [new branch] xmfan/post_3945954741e2d37023c5d6954f9483008e0892f9 -> origin/xmfan/post_3945954741e2d37023c5d6954f9483008e0892f9 2025-12-04T09:23:32.9026337Z * [new branch] xmfan/pre_3945954741e2d37023c5d6954f9483008e0892f9 -> origin/xmfan/pre_3945954741e2d37023c5d6954f9483008e0892f9 2025-12-04T09:23:32.9026470Z * [new branch] xmfan/single_step -> origin/xmfan/single_step 2025-12-04T09:23:32.9026602Z * [new branch] xmfan/sth_0829 -> origin/xmfan/sth_0829 2025-12-04T09:23:32.9026717Z * [new branch] xmfan/test -> origin/xmfan/test 2025-12-04T09:23:32.9026880Z * [new branch] yguo/debug-0226-constexpr -> origin/yguo/debug-0226-constexpr 2025-12-04T09:23:32.9027025Z * [new branch] yguo/new_latest_changes -> origin/yguo/new_latest_changes 2025-12-04T09:23:32.9027197Z * [new branch] yguo/patch_constexpr_changes -> origin/yguo/patch_constexpr_changes 2025-12-04T09:23:32.9027331Z * [new branch] yiming/bootcamp -> origin/yiming/bootcamp 2025-12-04T09:23:32.9027500Z * [new branch] yiming/run_with_start_end_rng_hop -> origin/yiming/run_with_start_end_rng_hop 2025-12-04T09:23:32.9027627Z * [new branch] yolo-llama3 -> origin/yolo-llama3 2025-12-04T09:23:32.9027758Z * [new branch] zainr/canary-test -> origin/zainr/canary-test 2025-12-04T09:23:32.9028151Z * [new branch] zainr/cleanup-gh-runners -> origin/zainr/cleanup-gh-runners 2025-12-04T09:23:32.9028311Z * [new branch] zainr/pull-migration-c -> origin/zainr/pull-migration-c 2025-12-04T09:23:32.9028423Z * [new branch] zainr/test2 -> origin/zainr/test2 2025-12-04T09:23:32.9028650Z * [new branch] zasdfgbnm-patch-3 -> origin/zasdfgbnm-patch-3 2025-12-04T09:23:32.9028766Z * [new branch] zb2p -> origin/zb2p 2025-12-04T09:23:32.9028921Z * [new branch] zeros-and-scatter-part2 -> origin/zeros-and-scatter-part2 2025-12-04T09:23:32.9029078Z * [new branch] zhxchen17/ci/vllm_lora_oom -> origin/zhxchen17/ci/vllm_lora_oom 2025-12-04T09:23:32.9029252Z * [new branch] zhxchen17/ci/vllm_multimodal_oom -> origin/zhxchen17/ci/vllm_multimodal_oom 2025-12-04T09:23:32.9029392Z * [new branch] zhxchen17/ci/vllm_pin -> origin/zhxchen17/ci/vllm_pin 2025-12-04T09:23:32.9029614Z * [new branch] zhxchen17/dynamo/unsafe_drop_all_guards -> origin/zhxchen17/dynamo/unsafe_drop_all_guards 2025-12-04T09:23:32.9029789Z * [new branch] zhxchen17/export/call_override -> origin/zhxchen17/export/call_override 2025-12-04T09:23:32.9029956Z * [new branch] zhxchen17/export/codemod1 -> origin/zhxchen17/export/codemod1 2025-12-04T09:23:32.9030114Z * [new branch] zhxchen17/export/ctx_return -> origin/zhxchen17/export/ctx_return 2025-12-04T09:23:32.9030336Z * [new branch] zhxchen17/export/disable_side_effect_warn -> origin/zhxchen17/export/disable_side_effect_warn 2025-12-04T09:23:32.9030725Z * [new branch] zhxchen17/export/pytree_check -> origin/zhxchen17/export/pytree_check 2025-12-04T09:23:32.9031274Z * [new branch] zhxchen17/precompile/aoti -> origin/zhxchen17/precompile/aoti 2025-12-04T09:23:32.9032192Z * [new branch] zhxchen17/precompile/globals -> origin/zhxchen17/precompile/globals 2025-12-04T09:23:32.9032610Z * [new branch] zhxchen17/precompile/inductor_guards -> origin/zhxchen17/precompile/inductor_guards 2025-12-04T09:23:32.9034005Z * [new branch] zhxchen17/scratch/0 -> origin/zhxchen17/scratch/0 2025-12-04T09:23:32.9034558Z * [new branch] zhxchen17/torch_export_api_update -> origin/zhxchen17/torch_export_api_update 2025-12-04T09:23:32.9038291Z * [new branch] zhxhcen17/moodycamel -> origin/zhxhcen17/moodycamel 2025-12-04T09:23:32.9038496Z * [new branch] zxiiro/build-times -> origin/zxiiro/build-times 2025-12-04T09:23:32.9038657Z * [new branch] zxiiro/c7i.2xlarge -> origin/zxiiro/c7i.2xlarge 2025-12-04T09:23:32.9038826Z * [new branch] zxiiro/c7i.2xlarge.h100 -> origin/zxiiro/c7i.2xlarge.h100 2025-12-04T09:23:32.9043802Z * [new branch] zxiiro/main -> origin/zxiiro/main 2025-12-04T09:23:32.9048998Z * [new branch] zxiiro/risc64 -> origin/zxiiro/risc64 2025-12-04T09:23:32.9053092Z * [new branch] zxiiro/test-multicloud-arc -> origin/zxiiro/test-multicloud-arc 2025-12-04T09:23:32.9057816Z * [new tag] bc2caa7fdf006894eff7af936babde69ab5a40f8-huydhn-debug -> bc2caa7fdf006894eff7af936babde69ab5a40f8-huydhn-debug 2025-12-04T09:23:32.9057965Z * [new tag] ci/binaries/77164 -> ci/binaries/77164 2025-12-04T09:23:32.9058096Z * [new tag] ciflow/b200/115316 -> ciflow/b200/115316 2025-12-04T09:23:32.9058203Z * [new tag] ciflow/b200/160685 -> ciflow/b200/160685 2025-12-04T09:23:32.9058336Z * [new tag] ciflow/b200/161607 -> ciflow/b200/161607 2025-12-04T09:23:32.9058438Z * [new tag] ciflow/b200/161938 -> ciflow/b200/161938 2025-12-04T09:23:32.9058681Z * [new tag] ciflow/b200/167207 -> ciflow/b200/167207 2025-12-04T09:23:32.9058797Z * [new tag] ciflow/b200/167989 -> ciflow/b200/167989 2025-12-04T09:23:32.9058899Z * [new tag] ciflow/b200/168096 -> ciflow/b200/168096 2025-12-04T09:23:32.9059000Z * [new tag] ciflow/b200/168175 -> ciflow/b200/168175 2025-12-04T09:23:32.9059187Z * [new tag] ciflow/b200/168195 -> ciflow/b200/168195 2025-12-04T09:23:32.9059302Z * [new tag] ciflow/b200/169200 -> ciflow/b200/169200 2025-12-04T09:23:32.9059408Z * [new tag] ciflow/b200/169216 -> ciflow/b200/169216 2025-12-04T09:23:32.9059507Z * [new tag] ciflow/b200/169380 -> ciflow/b200/169380 2025-12-04T09:23:32.9059607Z * [new tag] ciflow/b200/169412 -> ciflow/b200/169412 2025-12-04T09:23:32.9059715Z * [new tag] ciflow/b200/169470 -> ciflow/b200/169470 2025-12-04T09:23:32.9059815Z * [new tag] ciflow/b200/169471 -> ciflow/b200/169471 2025-12-04T09:23:32.9059918Z * [new tag] ciflow/b200/169472 -> ciflow/b200/169472 2025-12-04T09:23:32.9060018Z * [new tag] ciflow/b200/169514 -> ciflow/b200/169514 2025-12-04T09:23:32.9060122Z * [new tag] ciflow/b200/169517 -> ciflow/b200/169517 2025-12-04T09:23:32.9060253Z * [new tag] ciflow/binaries/165922 -> ciflow/binaries/165922 2025-12-04T09:23:32.9060368Z * [new tag] ciflow/binaries/169510 -> ciflow/binaries/169510 2025-12-04T09:23:32.9060504Z * [new tag] ciflow/binaries_wheel/157994 -> ciflow/binaries_wheel/157994 2025-12-04T09:23:32.9060640Z * [new tag] ciflow/binaries_wheel/166829 -> ciflow/binaries_wheel/166829 2025-12-04T09:23:32.9060767Z * [new tag] ciflow/binaries_wheel/167972 -> ciflow/binaries_wheel/167972 2025-12-04T09:23:32.9060903Z * [new tag] ciflow/binaries_wheel/167981 -> ciflow/binaries_wheel/167981 2025-12-04T09:23:32.9061014Z * [new tag] ciflow/dynamo/167695 -> ciflow/dynamo/167695 2025-12-04T09:23:32.9061124Z * [new tag] ciflow/dynamo/168096 -> ciflow/dynamo/168096 2025-12-04T09:23:32.9061243Z * [new tag] ciflow/dynamo/169525 -> ciflow/dynamo/169525 2025-12-04T09:23:32.9061421Z * [new tag] ciflow/h100-cutlass-backend/161938 -> ciflow/h100-cutlass-backend/161938 2025-12-04T09:23:32.9061586Z * [new tag] ciflow/h100-cutlass-backend/161940 -> ciflow/h100-cutlass-backend/161940 2025-12-04T09:23:32.9061737Z * [new tag] ciflow/h100-distributed/168923 -> ciflow/h100-distributed/168923 2025-12-04T09:23:32.9061866Z * [new tag] ciflow/h100-symm-mem/167552 -> ciflow/h100-symm-mem/167552 2025-12-04T09:23:32.9061998Z * [new tag] ciflow/h100-symm-mem/168129 -> ciflow/h100-symm-mem/168129 2025-12-04T09:23:32.9062123Z * [new tag] ciflow/h100-symm-mem/168917 -> ciflow/h100-symm-mem/168917 2025-12-04T09:23:32.9062251Z * [new tag] ciflow/h100-symm-mem/169156 -> ciflow/h100-symm-mem/169156 2025-12-04T09:23:32.9062369Z * [new tag] ciflow/h100-symm-mem/169200 -> ciflow/h100-symm-mem/169200 2025-12-04T09:23:32.9062494Z * [new tag] ciflow/h100-symm-mem/169216 -> ciflow/h100-symm-mem/169216 2025-12-04T09:23:32.9062625Z * [new tag] ciflow/h100-symm-mem/169338 -> ciflow/h100-symm-mem/169338 2025-12-04T09:23:32.9063114Z * [new tag] ciflow/h100-symm-mem/169355 -> ciflow/h100-symm-mem/169355 2025-12-04T09:23:32.9065215Z * [new tag] ciflow/h100-symm-mem/169543 -> ciflow/h100-symm-mem/169543 2025-12-04T09:23:32.9065388Z * [new tag] ciflow/h100/115316 -> ciflow/h100/115316 2025-12-04T09:23:32.9065791Z * [new tag] ciflow/h100/160685 -> ciflow/h100/160685 2025-12-04T09:23:32.9066021Z * [new tag] ciflow/h100/160729 -> ciflow/h100/160729 2025-12-04T09:23:32.9066150Z * [new tag] ciflow/h100/161607 -> ciflow/h100/161607 2025-12-04T09:23:32.9066268Z * [new tag] ciflow/h100/161938 -> ciflow/h100/161938 2025-12-04T09:23:32.9066435Z * [new tag] ciflow/h100/167207 -> ciflow/h100/167207 2025-12-04T09:23:32.9066858Z * [new tag] ciflow/h100/167989 -> ciflow/h100/167989 2025-12-04T09:23:32.9068250Z * [new tag] ciflow/h100/168096 -> ciflow/h100/168096 2025-12-04T09:23:32.9068396Z * [new tag] ciflow/h100/168175 -> ciflow/h100/168175 2025-12-04T09:23:32.9068515Z * [new tag] ciflow/h100/168195 -> ciflow/h100/168195 2025-12-04T09:23:32.9070507Z * [new tag] ciflow/h100/168980 -> ciflow/h100/168980 2025-12-04T09:23:32.9071000Z * [new tag] ciflow/h100/169200 -> ciflow/h100/169200 2025-12-04T09:23:32.9071112Z * [new tag] ciflow/h100/169216 -> ciflow/h100/169216 2025-12-04T09:23:32.9071218Z * [new tag] ciflow/h100/169380 -> ciflow/h100/169380 2025-12-04T09:23:32.9071343Z * [new tag] ciflow/h100/169412 -> ciflow/h100/169412 2025-12-04T09:23:32.9071450Z * [new tag] ciflow/h100/169470 -> ciflow/h100/169470 2025-12-04T09:23:32.9071818Z * [new tag] ciflow/h100/169471 -> ciflow/h100/169471 2025-12-04T09:23:32.9072183Z * [new tag] ciflow/h100/169472 -> ciflow/h100/169472 2025-12-04T09:23:32.9072994Z * [new tag] ciflow/h100/169514 -> ciflow/h100/169514 2025-12-04T09:23:32.9079211Z * [new tag] ciflow/inductor-cu126/168096 -> ciflow/inductor-cu126/168096 2025-12-04T09:23:32.9084998Z * [new tag] ciflow/inductor-micro-benchmark-cpu-x86/168096 -> ciflow/inductor-micro-benchmark-cpu-x86/168096 2025-12-04T09:23:32.9086998Z * [new tag] ciflow/inductor-micro-benchmark/166165 -> ciflow/inductor-micro-benchmark/166165 2025-12-04T09:23:32.9087361Z * [new tag] ciflow/inductor-micro-benchmark/168096 -> ciflow/inductor-micro-benchmark/168096 2025-12-04T09:23:32.9087569Z * [new tag] ciflow/inductor-perf-compare/168096 -> ciflow/inductor-perf-compare/168096 2025-12-04T09:23:32.9087932Z * [new tag] ciflow/inductor-perf-test-nightly-rocm-mi300/168073 -> ciflow/inductor-perf-test-nightly-rocm-mi300/168073 2025-12-04T09:23:32.9088267Z * [new tag] ciflow/inductor-perf-test-nightly-rocm-mi300/168096 -> ciflow/inductor-perf-test-nightly-rocm-mi300/168096 2025-12-04T09:23:32.9088629Z * [new tag] ciflow/inductor-perf-test-nightly-rocm-mi300/169024 -> ciflow/inductor-perf-test-nightly-rocm-mi300/169024 2025-12-04T09:23:32.9089343Z * [new tag] ciflow/inductor-perf-test-nightly-rocm-mi355/169024 -> ciflow/inductor-perf-test-nightly-rocm-mi355/169024 2025-12-04T09:23:32.9089677Z * [new tag] ciflow/inductor-perf-test-nightly/168096 -> ciflow/inductor-perf-test-nightly/168096 2025-12-04T09:23:32.9089925Z * [new tag] ciflow/inductor-periodic/168096 -> ciflow/inductor-periodic/168096 2025-12-04T09:23:32.9090119Z * [new tag] ciflow/inductor-periodic/169024 -> ciflow/inductor-periodic/169024 2025-12-04T09:23:32.9090287Z * [new tag] ciflow/inductor-periodic/169425 -> ciflow/inductor-periodic/169425 2025-12-04T09:23:32.9090476Z * [new tag] ciflow/inductor-rocm-mi200/165545 -> ciflow/inductor-rocm-mi200/165545 2025-12-04T09:23:32.9090680Z * [new tag] ciflow/inductor-rocm-mi200/165997 -> ciflow/inductor-rocm-mi200/165997 2025-12-04T09:23:32.9090851Z * [new tag] ciflow/inductor-rocm-mi200/168096 -> ciflow/inductor-rocm-mi200/168096 2025-12-04T09:23:32.9091199Z * [new tag] ciflow/inductor-rocm-mi200/169063 -> ciflow/inductor-rocm-mi200/169063 2025-12-04T09:23:32.9091379Z * [new tag] ciflow/inductor-rocm-mi200/169425 -> ciflow/inductor-rocm-mi200/169425 2025-12-04T09:23:32.9091624Z * [new tag] ciflow/inductor-rocm-mi300/165545 -> ciflow/inductor-rocm-mi300/165545 2025-12-04T09:23:32.9091803Z * [new tag] ciflow/inductor-rocm-mi300/168096 -> ciflow/inductor-rocm-mi300/168096 2025-12-04T09:23:32.9091986Z * [new tag] ciflow/inductor-rocm-mi300/169063 -> ciflow/inductor-rocm-mi300/169063 2025-12-04T09:23:32.9092164Z * [new tag] ciflow/inductor-rocm-mi300/169425 -> ciflow/inductor-rocm-mi300/169425 2025-12-04T09:23:32.9092314Z * [new tag] ciflow/inductor-rocm/162052 -> ciflow/inductor-rocm/162052 2025-12-04T09:23:32.9092460Z * [new tag] ciflow/inductor-rocm/168971 -> ciflow/inductor-rocm/168971 2025-12-04T09:23:32.9092631Z * [new tag] ciflow/inductor-windows/168096 -> ciflow/inductor-windows/168096 2025-12-04T09:23:32.9092772Z * [new tag] ciflow/inductor/144542 -> ciflow/inductor/144542 2025-12-04T09:23:32.9092914Z * [new tag] ciflow/inductor/146506 -> ciflow/inductor/146506 2025-12-04T09:23:32.9093052Z * [new tag] ciflow/inductor/147990 -> ciflow/inductor/147990 2025-12-04T09:23:32.9093183Z * [new tag] ciflow/inductor/148294 -> ciflow/inductor/148294 2025-12-04T09:23:32.9093313Z * [new tag] ciflow/inductor/148492 -> ciflow/inductor/148492 2025-12-04T09:23:32.9093445Z * [new tag] ciflow/inductor/157149 -> ciflow/inductor/157149 2025-12-04T09:23:32.9093586Z * [new tag] ciflow/inductor/157994 -> ciflow/inductor/157994 2025-12-04T09:23:32.9093717Z * [new tag] ciflow/inductor/160685 -> ciflow/inductor/160685 2025-12-04T09:23:32.9093867Z * [new tag] ciflow/inductor/160686 -> ciflow/inductor/160686 2025-12-04T09:23:32.9093986Z * [new tag] ciflow/inductor/160687 -> ciflow/inductor/160687 2025-12-04T09:23:32.9094100Z * [new tag] ciflow/inductor/160688 -> ciflow/inductor/160688 2025-12-04T09:23:32.9094216Z * [new tag] ciflow/inductor/160706 -> ciflow/inductor/160706 2025-12-04T09:23:32.9094339Z * [new tag] ciflow/inductor/160729 -> ciflow/inductor/160729 2025-12-04T09:23:32.9094454Z * [new tag] ciflow/inductor/161938 -> ciflow/inductor/161938 2025-12-04T09:23:32.9094752Z * [new tag] ciflow/inductor/161939 -> ciflow/inductor/161939 2025-12-04T09:23:32.9094927Z * [new tag] ciflow/inductor/161940 -> ciflow/inductor/161940 2025-12-04T09:23:32.9095049Z * [new tag] ciflow/inductor/162052 -> ciflow/inductor/162052 2025-12-04T09:23:32.9095343Z * [new tag] ciflow/inductor/162275 -> ciflow/inductor/162275 2025-12-04T09:23:32.9095909Z * [new tag] ciflow/inductor/162795 -> ciflow/inductor/162795 2025-12-04T09:23:32.9096648Z * [new tag] ciflow/inductor/163245 -> ciflow/inductor/163245 2025-12-04T09:23:32.9096886Z * [new tag] ciflow/inductor/163335 -> ciflow/inductor/163335 2025-12-04T09:23:32.9097329Z * [new tag] ciflow/inductor/163503 -> ciflow/inductor/163503 2025-12-04T09:23:32.9097761Z * [new tag] ciflow/inductor/163942 -> ciflow/inductor/163942 2025-12-04T09:23:32.9099049Z * [new tag] ciflow/inductor/165270 -> ciflow/inductor/165270 2025-12-04T09:23:32.9099239Z * [new tag] ciflow/inductor/165274 -> ciflow/inductor/165274 2025-12-04T09:23:32.9099387Z * [new tag] ciflow/inductor/165322 -> ciflow/inductor/165322 2025-12-04T09:23:32.9099705Z * [new tag] ciflow/inductor/165597 -> ciflow/inductor/165597 2025-12-04T09:23:32.9100122Z * [new tag] ciflow/inductor/166063 -> ciflow/inductor/166063 2025-12-04T09:23:32.9100574Z * [new tag] ciflow/inductor/166075 -> ciflow/inductor/166075 2025-12-04T09:23:32.9101074Z * [new tag] ciflow/inductor/166165 -> ciflow/inductor/166165 2025-12-04T09:23:32.9102294Z * [new tag] ciflow/inductor/166254 -> ciflow/inductor/166254 2025-12-04T09:23:32.9102439Z * [new tag] ciflow/inductor/166483 -> ciflow/inductor/166483 2025-12-04T09:23:32.9102564Z * [new tag] ciflow/inductor/166494 -> ciflow/inductor/166494 2025-12-04T09:23:32.9103005Z * [new tag] ciflow/inductor/166545 -> ciflow/inductor/166545 2025-12-04T09:23:32.9103413Z * [new tag] ciflow/inductor/166788 -> ciflow/inductor/166788 2025-12-04T09:23:32.9104244Z * [new tag] ciflow/inductor/166846 -> ciflow/inductor/166846 2025-12-04T09:23:32.9104496Z * [new tag] ciflow/inductor/167300 -> ciflow/inductor/167300 2025-12-04T09:23:32.9104964Z * [new tag] ciflow/inductor/167407 -> ciflow/inductor/167407 2025-12-04T09:23:32.9107388Z * [new tag] ciflow/inductor/167536 -> ciflow/inductor/167536 2025-12-04T09:23:32.9107548Z * [new tag] ciflow/inductor/167552 -> ciflow/inductor/167552 2025-12-04T09:23:32.9107667Z * [new tag] ciflow/inductor/167555 -> ciflow/inductor/167555 2025-12-04T09:23:32.9107791Z * [new tag] ciflow/inductor/167583 -> ciflow/inductor/167583 2025-12-04T09:23:32.9107911Z * [new tag] ciflow/inductor/167599 -> ciflow/inductor/167599 2025-12-04T09:23:32.9108181Z * [new tag] ciflow/inductor/167647 -> ciflow/inductor/167647 2025-12-04T09:23:32.9108657Z * [new tag] ciflow/inductor/167677 -> ciflow/inductor/167677 2025-12-04T09:23:32.9109094Z * [new tag] ciflow/inductor/167680 -> ciflow/inductor/167680 2025-12-04T09:23:32.9109631Z * [new tag] ciflow/inductor/167695 -> ciflow/inductor/167695 2025-12-04T09:23:32.9110001Z * [new tag] ciflow/inductor/167742 -> ciflow/inductor/167742 2025-12-04T09:23:32.9110495Z * [new tag] ciflow/inductor/167768 -> ciflow/inductor/167768 2025-12-04T09:23:32.9112889Z * [new tag] ciflow/inductor/167773 -> ciflow/inductor/167773 2025-12-04T09:23:32.9113075Z * [new tag] ciflow/inductor/167781 -> ciflow/inductor/167781 2025-12-04T09:23:32.9113210Z * [new tag] ciflow/inductor/167880 -> ciflow/inductor/167880 2025-12-04T09:23:32.9113356Z * [new tag] ciflow/inductor/167887 -> ciflow/inductor/167887 2025-12-04T09:23:32.9113510Z * [new tag] ciflow/inductor/167972 -> ciflow/inductor/167972 2025-12-04T09:23:32.9113681Z * [new tag] ciflow/inductor/167989 -> ciflow/inductor/167989 2025-12-04T09:23:32.9114208Z * [new tag] ciflow/inductor/168002 -> ciflow/inductor/168002 2025-12-04T09:23:32.9114659Z * [new tag] ciflow/inductor/168050 -> ciflow/inductor/168050 2025-12-04T09:23:32.9115117Z * [new tag] ciflow/inductor/168051 -> ciflow/inductor/168051 2025-12-04T09:23:32.9116730Z * [new tag] ciflow/inductor/168052 -> ciflow/inductor/168052 2025-12-04T09:23:32.9117053Z * [new tag] ciflow/inductor/168073 -> ciflow/inductor/168073 2025-12-04T09:23:32.9117287Z * [new tag] ciflow/inductor/168096 -> ciflow/inductor/168096 2025-12-04T09:23:32.9117440Z * [new tag] ciflow/inductor/168114 -> ciflow/inductor/168114 2025-12-04T09:23:32.9117591Z * [new tag] ciflow/inductor/168115 -> ciflow/inductor/168115 2025-12-04T09:23:32.9118067Z * [new tag] ciflow/inductor/168127 -> ciflow/inductor/168127 2025-12-04T09:23:32.9118249Z * [new tag] ciflow/inductor/168129 -> ciflow/inductor/168129 2025-12-04T09:23:32.9119580Z * [new tag] ciflow/inductor/168157 -> ciflow/inductor/168157 2025-12-04T09:23:32.9119921Z * [new tag] ciflow/inductor/168175 -> ciflow/inductor/168175 2025-12-04T09:23:32.9120046Z * [new tag] ciflow/inductor/168185 -> ciflow/inductor/168185 2025-12-04T09:23:32.9120206Z * [new tag] ciflow/inductor/168195 -> ciflow/inductor/168195 2025-12-04T09:23:32.9120646Z * [new tag] ciflow/inductor/168209 -> ciflow/inductor/168209 2025-12-04T09:23:32.9122096Z * [new tag] ciflow/inductor/168266 -> ciflow/inductor/168266 2025-12-04T09:23:32.9122402Z * [new tag] ciflow/inductor/168316 -> ciflow/inductor/168316 2025-12-04T09:23:32.9122551Z * [new tag] ciflow/inductor/168326 -> ciflow/inductor/168326 2025-12-04T09:23:32.9122684Z * [new tag] ciflow/inductor/168368 -> ciflow/inductor/168368 2025-12-04T09:23:32.9123170Z * [new tag] ciflow/inductor/168894 -> ciflow/inductor/168894 2025-12-04T09:23:32.9123638Z * [new tag] ciflow/inductor/168934 -> ciflow/inductor/168934 2025-12-04T09:23:32.9125102Z * [new tag] ciflow/inductor/168939 -> ciflow/inductor/168939 2025-12-04T09:23:32.9125348Z * [new tag] ciflow/inductor/168946 -> ciflow/inductor/168946 2025-12-04T09:23:32.9125543Z * [new tag] ciflow/inductor/168950 -> ciflow/inductor/168950 2025-12-04T09:23:32.9125795Z * [new tag] ciflow/inductor/168951 -> ciflow/inductor/168951 2025-12-04T09:23:32.9127634Z * [new tag] ciflow/inductor/168952 -> ciflow/inductor/168952 2025-12-04T09:23:32.9127952Z * [new tag] ciflow/inductor/168955 -> ciflow/inductor/168955 2025-12-04T09:23:32.9128098Z * [new tag] ciflow/inductor/168971 -> ciflow/inductor/168971 2025-12-04T09:23:32.9128212Z * [new tag] ciflow/inductor/168979 -> ciflow/inductor/168979 2025-12-04T09:23:32.9128425Z * [new tag] ciflow/inductor/168980 -> ciflow/inductor/168980 2025-12-04T09:23:32.9128555Z * [new tag] ciflow/inductor/168983 -> ciflow/inductor/168983 2025-12-04T09:23:32.9129043Z * [new tag] ciflow/inductor/169006 -> ciflow/inductor/169006 2025-12-04T09:23:32.9129579Z * [new tag] ciflow/inductor/169023 -> ciflow/inductor/169023 2025-12-04T09:23:32.9129944Z * [new tag] ciflow/inductor/169024 -> ciflow/inductor/169024 2025-12-04T09:23:32.9130458Z * [new tag] ciflow/inductor/169025 -> ciflow/inductor/169025 2025-12-04T09:23:32.9130888Z * [new tag] ciflow/inductor/169066 -> ciflow/inductor/169066 2025-12-04T09:23:32.9131301Z * [new tag] ciflow/inductor/169091 -> ciflow/inductor/169091 2025-12-04T09:23:32.9131724Z * [new tag] ciflow/inductor/169102 -> ciflow/inductor/169102 2025-12-04T09:23:32.9132175Z * [new tag] ciflow/inductor/169103 -> ciflow/inductor/169103 2025-12-04T09:23:32.9132609Z * [new tag] ciflow/inductor/169121 -> ciflow/inductor/169121 2025-12-04T09:23:32.9133023Z * [new tag] ciflow/inductor/169134 -> ciflow/inductor/169134 2025-12-04T09:23:32.9133486Z * [new tag] ciflow/inductor/169135 -> ciflow/inductor/169135 2025-12-04T09:23:32.9134188Z * [new tag] ciflow/inductor/169141 -> ciflow/inductor/169141 2025-12-04T09:23:32.9134648Z * [new tag] ciflow/inductor/169151 -> ciflow/inductor/169151 2025-12-04T09:23:32.9135137Z * [new tag] ciflow/inductor/169161 -> ciflow/inductor/169161 2025-12-04T09:23:32.9135802Z * [new tag] ciflow/inductor/169167 -> ciflow/inductor/169167 2025-12-04T09:23:32.9136246Z * [new tag] ciflow/inductor/169177 -> ciflow/inductor/169177 2025-12-04T09:23:32.9139593Z * [new tag] ciflow/inductor/169185 -> ciflow/inductor/169185 2025-12-04T09:23:32.9139919Z * [new tag] ciflow/inductor/169196 -> ciflow/inductor/169196 2025-12-04T09:23:32.9140060Z * [new tag] ciflow/inductor/169200 -> ciflow/inductor/169200 2025-12-04T09:23:32.9140203Z * [new tag] ciflow/inductor/169204 -> ciflow/inductor/169204 2025-12-04T09:23:32.9140318Z * [new tag] ciflow/inductor/169216 -> ciflow/inductor/169216 2025-12-04T09:23:32.9140561Z * [new tag] ciflow/inductor/169219 -> ciflow/inductor/169219 2025-12-04T09:23:32.9141027Z * [new tag] ciflow/inductor/169220 -> ciflow/inductor/169220 2025-12-04T09:23:32.9141186Z * [new tag] ciflow/inductor/169230 -> ciflow/inductor/169230 2025-12-04T09:23:32.9141310Z * [new tag] ciflow/inductor/169242 -> ciflow/inductor/169242 2025-12-04T09:23:32.9141451Z * [new tag] ciflow/inductor/169245 -> ciflow/inductor/169245 2025-12-04T09:23:32.9141750Z * [new tag] ciflow/inductor/169260 -> ciflow/inductor/169260 2025-12-04T09:23:32.9142168Z * [new tag] ciflow/inductor/169282 -> ciflow/inductor/169282 2025-12-04T09:23:32.9142636Z * [new tag] ciflow/inductor/169286 -> ciflow/inductor/169286 2025-12-04T09:23:32.9143235Z * [new tag] ciflow/inductor/169299 -> ciflow/inductor/169299 2025-12-04T09:23:32.9143692Z * [new tag] ciflow/inductor/169304 -> ciflow/inductor/169304 2025-12-04T09:23:32.9145905Z * [new tag] ciflow/inductor/169305 -> ciflow/inductor/169305 2025-12-04T09:23:32.9146216Z * [new tag] ciflow/inductor/169308 -> ciflow/inductor/169308 2025-12-04T09:23:32.9146358Z * [new tag] ciflow/inductor/169319 -> ciflow/inductor/169319 2025-12-04T09:23:32.9146532Z * [new tag] ciflow/inductor/169326 -> ciflow/inductor/169326 2025-12-04T09:23:32.9146678Z * [new tag] ciflow/inductor/169332 -> ciflow/inductor/169332 2025-12-04T09:23:32.9146809Z * [new tag] ciflow/inductor/169333 -> ciflow/inductor/169333 2025-12-04T09:23:32.9147584Z * [new tag] ciflow/inductor/169336 -> ciflow/inductor/169336 2025-12-04T09:23:32.9148140Z * [new tag] ciflow/inductor/169340 -> ciflow/inductor/169340 2025-12-04T09:23:32.9148525Z * [new tag] ciflow/inductor/169341 -> ciflow/inductor/169341 2025-12-04T09:23:32.9148772Z * [new tag] ciflow/inductor/169343 -> ciflow/inductor/169343 2025-12-04T09:23:32.9149306Z * [new tag] ciflow/inductor/169346 -> ciflow/inductor/169346 2025-12-04T09:23:32.9149836Z * [new tag] ciflow/inductor/169348 -> ciflow/inductor/169348 2025-12-04T09:23:32.9150376Z * [new tag] ciflow/inductor/169350 -> ciflow/inductor/169350 2025-12-04T09:23:32.9150816Z * [new tag] ciflow/inductor/169355 -> ciflow/inductor/169355 2025-12-04T09:23:32.9151350Z * [new tag] ciflow/inductor/169370 -> ciflow/inductor/169370 2025-12-04T09:23:32.9152508Z * [new tag] ciflow/inductor/169375 -> ciflow/inductor/169375 2025-12-04T09:23:32.9152906Z * [new tag] ciflow/inductor/169389 -> ciflow/inductor/169389 2025-12-04T09:23:32.9153224Z * [new tag] ciflow/inductor/169391 -> ciflow/inductor/169391 2025-12-04T09:23:32.9153688Z * [new tag] ciflow/inductor/169393 -> ciflow/inductor/169393 2025-12-04T09:23:32.9154119Z * [new tag] ciflow/inductor/169399 -> ciflow/inductor/169399 2025-12-04T09:23:32.9158709Z * [new tag] ciflow/inductor/169400 -> ciflow/inductor/169400 2025-12-04T09:23:32.9158880Z * [new tag] ciflow/inductor/169415 -> ciflow/inductor/169415 2025-12-04T09:23:32.9159152Z * [new tag] ciflow/inductor/169417 -> ciflow/inductor/169417 2025-12-04T09:23:32.9159278Z * [new tag] ciflow/inductor/169418 -> ciflow/inductor/169418 2025-12-04T09:23:32.9159405Z * [new tag] ciflow/inductor/169430 -> ciflow/inductor/169430 2025-12-04T09:23:32.9159525Z * [new tag] ciflow/inductor/169432 -> ciflow/inductor/169432 2025-12-04T09:23:32.9159652Z * [new tag] ciflow/inductor/169436 -> ciflow/inductor/169436 2025-12-04T09:23:32.9159771Z * [new tag] ciflow/inductor/169437 -> ciflow/inductor/169437 2025-12-04T09:23:32.9160062Z * [new tag] ciflow/inductor/169438 -> ciflow/inductor/169438 2025-12-04T09:23:32.9160247Z * [new tag] ciflow/inductor/169441 -> ciflow/inductor/169441 2025-12-04T09:23:32.9160381Z * [new tag] ciflow/inductor/169446 -> ciflow/inductor/169446 2025-12-04T09:23:32.9161162Z * [new tag] ciflow/inductor/169447 -> ciflow/inductor/169447 2025-12-04T09:23:32.9161430Z * [new tag] ciflow/inductor/169452 -> ciflow/inductor/169452 2025-12-04T09:23:32.9165073Z * [new tag] ciflow/inductor/169455 -> ciflow/inductor/169455 2025-12-04T09:23:32.9165337Z * [new tag] ciflow/inductor/169459 -> ciflow/inductor/169459 2025-12-04T09:23:32.9165682Z * [new tag] ciflow/inductor/169463 -> ciflow/inductor/169463 2025-12-04T09:23:32.9165815Z * [new tag] ciflow/inductor/169476 -> ciflow/inductor/169476 2025-12-04T09:23:32.9165953Z * [new tag] ciflow/inductor/169485 -> ciflow/inductor/169485 2025-12-04T09:23:32.9166071Z * [new tag] ciflow/inductor/169493 -> ciflow/inductor/169493 2025-12-04T09:23:32.9166190Z * [new tag] ciflow/inductor/169496 -> ciflow/inductor/169496 2025-12-04T09:23:32.9166325Z * [new tag] ciflow/inductor/169497 -> ciflow/inductor/169497 2025-12-04T09:23:32.9166437Z * [new tag] ciflow/inductor/169503 -> ciflow/inductor/169503 2025-12-04T09:23:32.9166578Z * [new tag] ciflow/inductor/169504 -> ciflow/inductor/169504 2025-12-04T09:23:32.9168262Z * [new tag] ciflow/inductor/169505 -> ciflow/inductor/169505 2025-12-04T09:23:32.9168580Z * [new tag] ciflow/inductor/169508 -> ciflow/inductor/169508 2025-12-04T09:23:32.9168734Z * [new tag] ciflow/inductor/169509 -> ciflow/inductor/169509 2025-12-04T09:23:32.9169010Z * [new tag] ciflow/inductor/169513 -> ciflow/inductor/169513 2025-12-04T09:23:32.9169474Z * [new tag] ciflow/inductor/169514 -> ciflow/inductor/169514 2025-12-04T09:23:32.9169944Z * [new tag] ciflow/inductor/169515 -> ciflow/inductor/169515 2025-12-04T09:23:32.9170696Z * [new tag] ciflow/inductor/169517 -> ciflow/inductor/169517 2025-12-04T09:23:32.9171046Z * [new tag] ciflow/inductor/169519 -> ciflow/inductor/169519 2025-12-04T09:23:32.9171432Z * [new tag] ciflow/inductor/169520 -> ciflow/inductor/169520 2025-12-04T09:23:32.9172005Z * [new tag] ciflow/inductor/169521 -> ciflow/inductor/169521 2025-12-04T09:23:32.9172418Z * [new tag] ciflow/inductor/169524 -> ciflow/inductor/169524 2025-12-04T09:23:32.9172871Z * [new tag] ciflow/inductor/169527 -> ciflow/inductor/169527 2025-12-04T09:23:32.9173336Z * [new tag] ciflow/inductor/169528 -> ciflow/inductor/169528 2025-12-04T09:23:32.9175423Z * [new tag] ciflow/inductor/169532 -> ciflow/inductor/169532 2025-12-04T09:23:32.9175715Z * [new tag] ciflow/inductor/169535 -> ciflow/inductor/169535 2025-12-04T09:23:32.9175964Z * [new tag] ciflow/inductor/169536 -> ciflow/inductor/169536 2025-12-04T09:23:32.9176253Z * [new tag] ciflow/inductor/169547 -> ciflow/inductor/169547 2025-12-04T09:23:32.9176501Z * [new tag] ciflow/inductor/169548 -> ciflow/inductor/169548 2025-12-04T09:23:32.9176683Z * [new tag] ciflow/inductor/169549 -> ciflow/inductor/169549 2025-12-04T09:23:32.9176806Z * [new tag] ciflow/inductor/169551 -> ciflow/inductor/169551 2025-12-04T09:23:32.9177241Z * [new tag] ciflow/inductor/169552 -> ciflow/inductor/169552 2025-12-04T09:23:32.9177806Z * [new tag] ciflow/inductor/169553 -> ciflow/inductor/169553 2025-12-04T09:23:32.9178451Z * [new tag] ciflow/inductor/169557 -> ciflow/inductor/169557 2025-12-04T09:23:32.9179049Z * [new tag] ciflow/inductor/3b9a386 -> ciflow/inductor/3b9a386 2025-12-04T09:23:32.9179480Z * [new tag] ciflow/inductor/3d4b92b -> ciflow/inductor/3d4b92b 2025-12-04T09:23:32.9182185Z * [new tag] ciflow/inductor/d224ac7 -> ciflow/inductor/d224ac7 2025-12-04T09:23:32.9182369Z * [new tag] ciflow/linux-aarch64/157994 -> ciflow/linux-aarch64/157994 2025-12-04T09:23:32.9182506Z * [new tag] ciflow/linux-aarch64/166075 -> ciflow/linux-aarch64/166075 2025-12-04T09:23:32.9182642Z * [new tag] ciflow/linux-aarch64/166876 -> ciflow/linux-aarch64/166876 2025-12-04T09:23:32.9182765Z * [new tag] ciflow/linux-aarch64/167981 -> ciflow/linux-aarch64/167981 2025-12-04T09:23:32.9182944Z * [new tag] ciflow/mps/166254 -> ciflow/mps/166254 2025-12-04T09:23:32.9183200Z * [new tag] ciflow/mps/169017 -> ciflow/mps/169017 2025-12-04T09:23:32.9183956Z * [new tag] ciflow/mps/169372 -> ciflow/mps/169372 2025-12-04T09:23:32.9184251Z * [new tag] ciflow/mps/169478 -> ciflow/mps/169478 2025-12-04T09:23:32.9187561Z * [new tag] ciflow/op-benchmark/157994 -> ciflow/op-benchmark/157994 2025-12-04T09:23:32.9187746Z * [new tag] ciflow/op-benchmark/166075 -> ciflow/op-benchmark/166075 2025-12-04T09:23:32.9187883Z * [new tag] ciflow/op-benchmark/169544 -> ciflow/op-benchmark/169544 2025-12-04T09:23:32.9188076Z * [new tag] ciflow/periodic-rocm-mi200/165997 -> ciflow/periodic-rocm-mi200/165997 2025-12-04T09:23:32.9188237Z * [new tag] ciflow/periodic-rocm-mi200/166517 -> ciflow/periodic-rocm-mi200/166517 2025-12-04T09:23:32.9188395Z * [new tag] ciflow/periodic-rocm-mi200/169063 -> ciflow/periodic-rocm-mi200/169063 2025-12-04T09:23:32.9188561Z * [new tag] ciflow/periodic-rocm-mi200/169425 -> ciflow/periodic-rocm-mi200/169425 2025-12-04T09:23:32.9188840Z * [new tag] ciflow/periodic-rocm-mi300/166517 -> ciflow/periodic-rocm-mi300/166517 2025-12-04T09:23:32.9189116Z * [new tag] ciflow/periodic-rocm-mi300/169063 -> ciflow/periodic-rocm-mi300/169063 2025-12-04T09:23:32.9189561Z * [new tag] ciflow/periodic-rocm-mi300/169425 -> ciflow/periodic-rocm-mi300/169425 2025-12-04T09:23:32.9190620Z * [new tag] ciflow/periodic/054a2fd -> ciflow/periodic/054a2fd 2025-12-04T09:23:32.9190913Z * [new tag] ciflow/periodic/167207 -> ciflow/periodic/167207 2025-12-04T09:23:32.9191353Z * [new tag] ciflow/periodic/167978 -> ciflow/periodic/167978 2025-12-04T09:23:32.9191770Z * [new tag] ciflow/periodic/168096 -> ciflow/periodic/168096 2025-12-04T09:23:32.9192241Z * [new tag] ciflow/periodic/169286 -> ciflow/periodic/169286 2025-12-04T09:23:32.9192989Z * [new tag] ciflow/periodic/2a6d37d -> ciflow/periodic/2a6d37d 2025-12-04T09:23:32.9193676Z * [new tag] ciflow/periodic/317eeb8 -> ciflow/periodic/317eeb8 2025-12-04T09:23:32.9194218Z * [new tag] ciflow/periodic/3c32 -> ciflow/periodic/3c32 2025-12-04T09:23:32.9197116Z * [new tag] ciflow/periodic/3e98831 -> ciflow/periodic/3e98831 2025-12-04T09:23:32.9197656Z * [new tag] ciflow/periodic/7c648509a7470ace9fb2bae960dd4790f7e943e9 -> ciflow/periodic/7c648509a7470ace9fb2bae960dd4790f7e943e9 2025-12-04T09:23:32.9197947Z * [new tag] ciflow/periodic/94512-point -> ciflow/periodic/94512-point 2025-12-04T09:23:32.9198286Z * [new tag] ciflow/periodic/csl/test87519 -> ciflow/periodic/csl/test87519 2025-12-04T09:23:32.9198466Z * [new tag] ciflow/periodic/csltest88275 -> ciflow/periodic/csltest88275 2025-12-04T09:23:32.9198875Z * [new tag] ciflow/periodic/csltest88761 -> ciflow/periodic/csltest88761 2025-12-04T09:23:32.9199707Z * [new tag] ciflow/periodic/release_1.12 -> ciflow/periodic/release_1.12 2025-12-04T09:23:32.9200217Z * [new tag] ciflow/periodic/release_1.12.0 -> ciflow/periodic/release_1.12.0 2025-12-04T09:23:32.9202664Z * [new tag] ciflow/periodic/sha-ec5b83 -> ciflow/periodic/sha-ec5b83 2025-12-04T09:23:32.9202968Z * [new tag] ciflow/pull/167207 -> ciflow/pull/167207 2025-12-04T09:23:32.9203185Z * [new tag] ciflow/quantization-periodic/169207 -> ciflow/quantization-periodic/169207 2025-12-04T09:23:32.9203402Z * [new tag] ciflow/rocm-mi200/165545 -> ciflow/rocm-mi200/165545 2025-12-04T09:23:32.9203601Z * [new tag] ciflow/rocm-mi200/165997 -> ciflow/rocm-mi200/165997 2025-12-04T09:23:32.9204013Z * [new tag] ciflow/rocm-mi200/168096 -> ciflow/rocm-mi200/168096 2025-12-04T09:23:32.9204970Z * [new tag] ciflow/rocm-mi200/168275 -> ciflow/rocm-mi200/168275 2025-12-04T09:23:32.9205126Z * [new tag] ciflow/rocm-mi200/169063 -> ciflow/rocm-mi200/169063 2025-12-04T09:23:32.9205639Z * [new tag] ciflow/rocm-mi200/169356 -> ciflow/rocm-mi200/169356 2025-12-04T09:23:32.9205939Z * [new tag] ciflow/rocm-mi200/169425 -> ciflow/rocm-mi200/169425 2025-12-04T09:23:32.9209104Z * [new tag] ciflow/rocm-mi300/165545 -> ciflow/rocm-mi300/165545 2025-12-04T09:23:32.9209409Z * [new tag] ciflow/rocm-mi300/167157 -> ciflow/rocm-mi300/167157 2025-12-04T09:23:32.9209555Z * [new tag] ciflow/rocm-mi300/168096 -> ciflow/rocm-mi300/168096 2025-12-04T09:23:32.9209705Z * [new tag] ciflow/rocm-mi300/169063 -> ciflow/rocm-mi300/169063 2025-12-04T09:23:32.9209839Z * [new tag] ciflow/rocm-mi300/169425 -> ciflow/rocm-mi300/169425 2025-12-04T09:23:32.9210037Z * [new tag] ciflow/rocm-mi355/167157 -> ciflow/rocm-mi355/167157 2025-12-04T09:23:32.9210247Z * [new tag] ciflow/rocm-mi355/168275 -> ciflow/rocm-mi355/168275 2025-12-04T09:23:32.9210384Z * [new tag] ciflow/rocm-mi355/169425 -> ciflow/rocm-mi355/169425 2025-12-04T09:23:32.9210516Z * [new tag] ciflow/rocm-navi31/168275 -> ciflow/rocm-navi31/168275 2025-12-04T09:23:32.9210769Z * [new tag] ciflow/rocm-navi31/169425 -> ciflow/rocm-navi31/169425 2025-12-04T09:23:32.9212607Z * [new tag] ciflow/rocm/115316 -> ciflow/rocm/115316 2025-12-04T09:23:32.9212894Z * [new tag] ciflow/rocm/148492 -> ciflow/rocm/148492 2025-12-04T09:23:32.9213064Z * [new tag] ciflow/rocm/160685 -> ciflow/rocm/160685 2025-12-04T09:23:32.9213445Z * [new tag] ciflow/rocm/161607 -> ciflow/rocm/161607 2025-12-04T09:23:32.9213706Z * [new tag] ciflow/rocm/162052 -> ciflow/rocm/162052 2025-12-04T09:23:32.9213966Z * [new tag] ciflow/rocm/165997 -> ciflow/rocm/165997 2025-12-04T09:23:32.9214167Z * [new tag] ciflow/rocm/166165 -> ciflow/rocm/166165 2025-12-04T09:23:32.9214378Z * [new tag] ciflow/rocm/166517 -> ciflow/rocm/166517 2025-12-04T09:23:32.9214735Z * [new tag] ciflow/rocm/167207 -> ciflow/rocm/167207 2025-12-04T09:23:32.9215190Z * [new tag] ciflow/rocm/167536 -> ciflow/rocm/167536 2025-12-04T09:23:32.9215894Z * [new tag] ciflow/rocm/167781 -> ciflow/rocm/167781 2025-12-04T09:23:32.9219575Z * [new tag] ciflow/rocm/167989 -> ciflow/rocm/167989 2025-12-04T09:23:32.9219891Z * [new tag] ciflow/rocm/168073 -> ciflow/rocm/168073 2025-12-04T09:23:32.9220019Z * [new tag] ciflow/rocm/168195 -> ciflow/rocm/168195 2025-12-04T09:23:32.9220169Z * [new tag] ciflow/rocm/168939 -> ciflow/rocm/168939 2025-12-04T09:23:32.9220291Z * [new tag] ciflow/rocm/168971 -> ciflow/rocm/168971 2025-12-04T09:23:32.9220406Z * [new tag] ciflow/rocm/169024 -> ciflow/rocm/169024 2025-12-04T09:23:32.9220638Z * [new tag] ciflow/rocm/169200 -> ciflow/rocm/169200 2025-12-04T09:23:32.9220808Z * [new tag] ciflow/rocm/169216 -> ciflow/rocm/169216 2025-12-04T09:23:32.9221430Z * [new tag] ciflow/rocm/169312 -> ciflow/rocm/169312 2025-12-04T09:23:32.9221724Z * [new tag] ciflow/rocm/169380 -> ciflow/rocm/169380 2025-12-04T09:23:32.9221963Z * [new tag] ciflow/rocm/169427 -> ciflow/rocm/169427 2025-12-04T09:23:32.9222089Z * [new tag] ciflow/rocm/169455 -> ciflow/rocm/169455 2025-12-04T09:23:32.9222442Z * [new tag] ciflow/rocm/169470 -> ciflow/rocm/169470 2025-12-04T09:23:32.9222907Z * [new tag] ciflow/rocm/169471 -> ciflow/rocm/169471 2025-12-04T09:23:32.9223348Z * [new tag] ciflow/rocm/169472 -> ciflow/rocm/169472 2025-12-04T09:23:32.9223796Z * [new tag] ciflow/rocm/169514 -> ciflow/rocm/169514 2025-12-04T09:23:32.9224984Z * [new tag] ciflow/slow/01c7106 -> ciflow/slow/01c7106 2025-12-04T09:23:32.9225268Z * [new tag] ciflow/slow/0577043 -> ciflow/slow/0577043 2025-12-04T09:23:32.9225932Z * [new tag] ciflow/slow/0d5b74da0cab798fbfdb9caa53fad816999c8386-sdym -> ciflow/slow/0d5b74da0cab798fbfdb9caa53fad816999c8386-sdym 2025-12-04T09:23:32.9226233Z * [new tag] ciflow/slow/0e81104 -> ciflow/slow/0e81104 2025-12-04T09:23:32.9226683Z * [new tag] ciflow/slow/167207 -> ciflow/slow/167207 2025-12-04T09:23:32.9227128Z * [new tag] ciflow/slow/168050 -> ciflow/slow/168050 2025-12-04T09:23:32.9228625Z * [new tag] ciflow/slow/1732077 -> ciflow/slow/1732077 2025-12-04T09:23:32.9228790Z * [new tag] ciflow/slow/187eb7c -> ciflow/slow/187eb7c 2025-12-04T09:23:32.9229162Z * [new tag] ciflow/slow/1faef89 -> ciflow/slow/1faef89 2025-12-04T09:23:32.9230464Z * [new tag] ciflow/slow/3920ec1 -> ciflow/slow/3920ec1 2025-12-04T09:23:32.9230608Z * [new tag] ciflow/slow/3b7c6b2 -> ciflow/slow/3b7c6b2 2025-12-04T09:23:32.9231207Z * [new tag] ciflow/slow/59a3759 -> ciflow/slow/59a3759 2025-12-04T09:23:32.9231718Z * [new tag] ciflow/slow/70ef0bb -> ciflow/slow/70ef0bb 2025-12-04T09:23:32.9232232Z * [new tag] ciflow/slow/788ff06 -> ciflow/slow/788ff06 2025-12-04T09:23:32.9233416Z * [new tag] ciflow/slow/8751002215790a3a88750faa8f4366933e296693-sdym -> ciflow/slow/8751002215790a3a88750faa8f4366933e296693-sdym 2025-12-04T09:23:32.9233608Z * [new tag] ciflow/slow/9d85864 -> ciflow/slow/9d85864 2025-12-04T09:23:32.9234551Z * [new tag] ciflow/slow/9ffad5b -> ciflow/slow/9ffad5b 2025-12-04T09:23:32.9234865Z * [new tag] ciflow/slow/a206e8b -> ciflow/slow/a206e8b 2025-12-04T09:23:32.9235682Z * [new tag] ciflow/slow/a837609 -> ciflow/slow/a837609 2025-12-04T09:23:32.9235991Z * [new tag] ciflow/slow/af841f3 -> ciflow/slow/af841f3 2025-12-04T09:23:32.9238641Z * [new tag] ciflow/slow/da3aba1e46157c4df504b067477cdf2b3c96b194-sdym -> ciflow/slow/da3aba1e46157c4df504b067477cdf2b3c96b194-sdym 2025-12-04T09:23:32.9238833Z * [new tag] ciflow/torchbench/168175 -> ciflow/torchbench/168175 2025-12-04T09:23:32.9238957Z * [new tag] ciflow/trunk/148492 -> ciflow/trunk/148492 2025-12-04T09:23:32.9239072Z * [new tag] ciflow/trunk/157149 -> ciflow/trunk/157149 2025-12-04T09:23:32.9239359Z * [new tag] ciflow/trunk/157994 -> ciflow/trunk/157994 2025-12-04T09:23:32.9239486Z * [new tag] ciflow/trunk/159718 -> ciflow/trunk/159718 2025-12-04T09:23:32.9239749Z * [new tag] ciflow/trunk/160685 -> ciflow/trunk/160685 2025-12-04T09:23:32.9240195Z * [new tag] ciflow/trunk/160729 -> ciflow/trunk/160729 2025-12-04T09:23:32.9240915Z * [new tag] ciflow/trunk/162275 -> ciflow/trunk/162275 2025-12-04T09:23:32.9241382Z * [new tag] ciflow/trunk/162795 -> ciflow/trunk/162795 2025-12-04T09:23:32.9241686Z * [new tag] ciflow/trunk/163245 -> ciflow/trunk/163245 2025-12-04T09:23:32.9242130Z * [new tag] ciflow/trunk/163942 -> ciflow/trunk/163942 2025-12-04T09:23:32.9242553Z * [new tag] ciflow/trunk/165274 -> ciflow/trunk/165274 2025-12-04T09:23:32.9243861Z * [new tag] ciflow/trunk/165483 -> ciflow/trunk/165483 2025-12-04T09:23:32.9244014Z * [new tag] ciflow/trunk/165728 -> ciflow/trunk/165728 2025-12-04T09:23:32.9244449Z * [new tag] ciflow/trunk/165922 -> ciflow/trunk/165922 2025-12-04T09:23:32.9244883Z * [new tag] ciflow/trunk/166075 -> ciflow/trunk/166075 2025-12-04T09:23:32.9245460Z * [new tag] ciflow/trunk/166165 -> ciflow/trunk/166165 2025-12-04T09:23:32.9245932Z * [new tag] ciflow/trunk/166829 -> ciflow/trunk/166829 2025-12-04T09:23:32.9246458Z * [new tag] ciflow/trunk/166843 -> ciflow/trunk/166843 2025-12-04T09:23:32.9246867Z * [new tag] ciflow/trunk/166876 -> ciflow/trunk/166876 2025-12-04T09:23:32.9247324Z * [new tag] ciflow/trunk/167207 -> ciflow/trunk/167207 2025-12-04T09:23:32.9247809Z * [new tag] ciflow/trunk/167536 -> ciflow/trunk/167536 2025-12-04T09:23:32.9248256Z * [new tag] ciflow/trunk/167552 -> ciflow/trunk/167552 2025-12-04T09:23:32.9248902Z * [new tag] ciflow/trunk/167555 -> ciflow/trunk/167555 2025-12-04T09:23:32.9250013Z * [new tag] ciflow/trunk/167599 -> ciflow/trunk/167599 2025-12-04T09:23:32.9250150Z * [new tag] ciflow/trunk/167659 -> ciflow/trunk/167659 2025-12-04T09:23:32.9250625Z * [new tag] ciflow/trunk/167672 -> ciflow/trunk/167672 2025-12-04T09:23:32.9251088Z * [new tag] ciflow/trunk/167742 -> ciflow/trunk/167742 2025-12-04T09:23:32.9251529Z * [new tag] ciflow/trunk/167781 -> ciflow/trunk/167781 2025-12-04T09:23:32.9253064Z * [new tag] ciflow/trunk/167837 -> ciflow/trunk/167837 2025-12-04T09:23:32.9253217Z * [new tag] ciflow/trunk/167887 -> ciflow/trunk/167887 2025-12-04T09:23:32.9253329Z * [new tag] ciflow/trunk/167978 -> ciflow/trunk/167978 2025-12-04T09:23:32.9253835Z * [new tag] ciflow/trunk/168050 -> ciflow/trunk/168050 2025-12-04T09:23:32.9254062Z * [new tag] ciflow/trunk/168051 -> ciflow/trunk/168051 2025-12-04T09:23:32.9254498Z * [new tag] ciflow/trunk/168096 -> ciflow/trunk/168096 2025-12-04T09:23:32.9255001Z * [new tag] ciflow/trunk/168127 -> ciflow/trunk/168127 2025-12-04T09:23:32.9255426Z * [new tag] ciflow/trunk/168157 -> ciflow/trunk/168157 2025-12-04T09:23:32.9255959Z * [new tag] ciflow/trunk/168175 -> ciflow/trunk/168175 2025-12-04T09:23:32.9256355Z * [new tag] ciflow/trunk/168209 -> ciflow/trunk/168209 2025-12-04T09:23:32.9259708Z * [new tag] ciflow/trunk/168213 -> ciflow/trunk/168213 2025-12-04T09:23:32.9259869Z * [new tag] ciflow/trunk/168226 -> ciflow/trunk/168226 2025-12-04T09:23:32.9259996Z * [new tag] ciflow/trunk/168262 -> ciflow/trunk/168262 2025-12-04T09:23:32.9260118Z * [new tag] ciflow/trunk/168275 -> ciflow/trunk/168275 2025-12-04T09:23:32.9260231Z * [new tag] ciflow/trunk/168328 -> ciflow/trunk/168328 2025-12-04T09:23:32.9260346Z * [new tag] ciflow/trunk/168368 -> ciflow/trunk/168368 2025-12-04T09:23:32.9260466Z * [new tag] ciflow/trunk/168917 -> ciflow/trunk/168917 2025-12-04T09:23:32.9260605Z * [new tag] ciflow/trunk/168933 -> ciflow/trunk/168933 2025-12-04T09:23:32.9261131Z * [new tag] ciflow/trunk/168941 -> ciflow/trunk/168941 2025-12-04T09:23:32.9261587Z * [new tag] ciflow/trunk/168955 -> ciflow/trunk/168955 2025-12-04T09:23:32.9262112Z * [new tag] ciflow/trunk/168980 -> ciflow/trunk/168980 2025-12-04T09:23:32.9262697Z * [new tag] ciflow/trunk/169004 -> ciflow/trunk/169004 2025-12-04T09:23:32.9263073Z * [new tag] ciflow/trunk/169006 -> ciflow/trunk/169006 2025-12-04T09:23:32.9263596Z * [new tag] ciflow/trunk/169023 -> ciflow/trunk/169023 2025-12-04T09:23:32.9264041Z * [new tag] ciflow/trunk/169025 -> ciflow/trunk/169025 2025-12-04T09:23:32.9264469Z * [new tag] ciflow/trunk/169048 -> ciflow/trunk/169048 2025-12-04T09:23:32.9267584Z * [new tag] ciflow/trunk/169066 -> ciflow/trunk/169066 2025-12-04T09:23:32.9267735Z * [new tag] ciflow/trunk/169091 -> ciflow/trunk/169091 2025-12-04T09:23:32.9268251Z * [new tag] ciflow/trunk/169102 -> ciflow/trunk/169102 2025-12-04T09:23:32.9268384Z * [new tag] ciflow/trunk/169103 -> ciflow/trunk/169103 2025-12-04T09:23:32.9268496Z * [new tag] ciflow/trunk/169125 -> ciflow/trunk/169125 2025-12-04T09:23:32.9268648Z * [new tag] ciflow/trunk/169139 -> ciflow/trunk/169139 2025-12-04T09:23:32.9268758Z * [new tag] ciflow/trunk/169148 -> ciflow/trunk/169148 2025-12-04T09:23:32.9268879Z * [new tag] ciflow/trunk/169151 -> ciflow/trunk/169151 2025-12-04T09:23:32.9269255Z * [new tag] ciflow/trunk/169156 -> ciflow/trunk/169156 2025-12-04T09:23:32.9269727Z * [new tag] ciflow/trunk/169176 -> ciflow/trunk/169176 2025-12-04T09:23:32.9270134Z * [new tag] ciflow/trunk/169204 -> ciflow/trunk/169204 2025-12-04T09:23:32.9270686Z * [new tag] ciflow/trunk/169207 -> ciflow/trunk/169207 2025-12-04T09:23:32.9271074Z * [new tag] ciflow/trunk/169211 -> ciflow/trunk/169211 2025-12-04T09:23:32.9272020Z * [new tag] ciflow/trunk/169231 -> ciflow/trunk/169231 2025-12-04T09:23:32.9272311Z * [new tag] ciflow/trunk/169260 -> ciflow/trunk/169260 2025-12-04T09:23:32.9272994Z * [new tag] ciflow/trunk/169271 -> ciflow/trunk/169271 2025-12-04T09:23:32.9273565Z * [new tag] ciflow/trunk/169280 -> ciflow/trunk/169280 2025-12-04T09:23:32.9273942Z * [new tag] ciflow/trunk/169281 -> ciflow/trunk/169281 2025-12-04T09:23:32.9274751Z * [new tag] ciflow/trunk/169286 -> ciflow/trunk/169286 2025-12-04T09:23:32.9275056Z * [new tag] ciflow/trunk/169293 -> ciflow/trunk/169293 2025-12-04T09:23:32.9275594Z * [new tag] ciflow/trunk/169296 -> ciflow/trunk/169296 2025-12-04T09:23:32.9276086Z * [new tag] ciflow/trunk/169304 -> ciflow/trunk/169304 2025-12-04T09:23:32.9276578Z * [new tag] ciflow/trunk/169305 -> ciflow/trunk/169305 2025-12-04T09:23:32.9277122Z * [new tag] ciflow/trunk/169312 -> ciflow/trunk/169312 2025-12-04T09:23:32.9277688Z * [new tag] ciflow/trunk/169328 -> ciflow/trunk/169328 2025-12-04T09:23:32.9278069Z * [new tag] ciflow/trunk/169343 -> ciflow/trunk/169343 2025-12-04T09:23:32.9278540Z * [new tag] ciflow/trunk/169355 -> ciflow/trunk/169355 2025-12-04T09:23:32.9279023Z * [new tag] ciflow/trunk/169370 -> ciflow/trunk/169370 2025-12-04T09:23:32.9281654Z * [new tag] ciflow/trunk/169379 -> ciflow/trunk/169379 2025-12-04T09:23:32.9281797Z * [new tag] ciflow/trunk/169380 -> ciflow/trunk/169380 2025-12-04T09:23:32.9281928Z * [new tag] ciflow/trunk/169385 -> ciflow/trunk/169385 2025-12-04T09:23:32.9282044Z * [new tag] ciflow/trunk/169387 -> ciflow/trunk/169387 2025-12-04T09:23:32.9282154Z * [new tag] ciflow/trunk/169410 -> ciflow/trunk/169410 2025-12-04T09:23:32.9283895Z * [new tag] ciflow/trunk/169412 -> ciflow/trunk/169412 2025-12-04T09:23:32.9284194Z * [new tag] ciflow/trunk/169418 -> ciflow/trunk/169418 2025-12-04T09:23:32.9284363Z * [new tag] ciflow/trunk/169423 -> ciflow/trunk/169423 2025-12-04T09:23:32.9284550Z * [new tag] ciflow/trunk/169427 -> ciflow/trunk/169427 2025-12-04T09:23:32.9284817Z * [new tag] ciflow/trunk/169430 -> ciflow/trunk/169430 2025-12-04T09:23:32.9285256Z * [new tag] ciflow/trunk/169437 -> ciflow/trunk/169437 2025-12-04T09:23:32.9285723Z * [new tag] ciflow/trunk/169442 -> ciflow/trunk/169442 2025-12-04T09:23:32.9286511Z * [new tag] ciflow/trunk/169452 -> ciflow/trunk/169452 2025-12-04T09:23:32.9286635Z * [new tag] ciflow/trunk/169454 -> ciflow/trunk/169454 2025-12-04T09:23:32.9289875Z * [new tag] ciflow/trunk/169459 -> ciflow/trunk/169459 2025-12-04T09:23:32.9290173Z * [new tag] ciflow/trunk/169474 -> ciflow/trunk/169474 2025-12-04T09:23:32.9290299Z * [new tag] ciflow/trunk/169475 -> ciflow/trunk/169475 2025-12-04T09:23:32.9290496Z * [new tag] ciflow/trunk/169476 -> ciflow/trunk/169476 2025-12-04T09:23:32.9290620Z * [new tag] ciflow/trunk/169487 -> ciflow/trunk/169487 2025-12-04T09:23:32.9290735Z * [new tag] ciflow/trunk/169497 -> ciflow/trunk/169497 2025-12-04T09:23:32.9290984Z * [new tag] ciflow/trunk/169503 -> ciflow/trunk/169503 2025-12-04T09:23:32.9291578Z * [new tag] ciflow/trunk/169505 -> ciflow/trunk/169505 2025-12-04T09:23:32.9291722Z * [new tag] ciflow/trunk/169507 -> ciflow/trunk/169507 2025-12-04T09:23:32.9291847Z * [new tag] ciflow/trunk/169514 -> ciflow/trunk/169514 2025-12-04T09:23:32.9292320Z * [new tag] ciflow/trunk/169517 -> ciflow/trunk/169517 2025-12-04T09:23:32.9292662Z * [new tag] ciflow/trunk/169519 -> ciflow/trunk/169519 2025-12-04T09:23:32.9293094Z * [new tag] ciflow/trunk/169528 -> ciflow/trunk/169528 2025-12-04T09:23:32.9293605Z * [new tag] ciflow/trunk/169541 -> ciflow/trunk/169541 2025-12-04T09:23:32.9294190Z * [new tag] ciflow/trunk/169555 -> ciflow/trunk/169555 2025-12-04T09:23:32.9298299Z * [new tag] ciflow/unstable/123 -> ciflow/unstable/123 2025-12-04T09:23:32.9298622Z * [new tag] ciflow/vllm/165270 -> ciflow/vllm/165270 2025-12-04T09:23:32.9298753Z * [new tag] ciflow/vllm/165274 -> ciflow/vllm/165274 2025-12-04T09:23:32.9298942Z * [new tag] ciflow/vllm/166494 -> ciflow/vllm/166494 2025-12-04T09:23:32.9299080Z * [new tag] ciflow/vllm/169219 -> ciflow/vllm/169219 2025-12-04T09:23:32.9299269Z * [new tag] ciflow/vllm/169220 -> ciflow/vllm/169220 2025-12-04T09:23:32.9299424Z * [new tag] ciflow/xpu/157994 -> ciflow/xpu/157994 2025-12-04T09:23:32.9299630Z * [new tag] ciflow/xpu/159718 -> ciflow/xpu/159718 2025-12-04T09:23:32.9300259Z * [new tag] ciflow/xpu/161940 -> ciflow/xpu/161940 2025-12-04T09:23:32.9300403Z * [new tag] ciflow/xpu/163251 -> ciflow/xpu/163251 2025-12-04T09:23:32.9300542Z * [new tag] ciflow/xpu/166829 -> ciflow/xpu/166829 2025-12-04T09:23:32.9300656Z * [new tag] ciflow/xpu/166843 -> ciflow/xpu/166843 2025-12-04T09:23:32.9300873Z * [new tag] ciflow/xpu/167972 -> ciflow/xpu/167972 2025-12-04T09:23:32.9301308Z * [new tag] ciflow/xpu/167981 -> ciflow/xpu/167981 2025-12-04T09:23:32.9301827Z * [new tag] ciflow/xpu/168213 -> ciflow/xpu/168213 2025-12-04T09:23:32.9302194Z * [new tag] ciflow/xpu/168262 -> ciflow/xpu/168262 2025-12-04T09:23:32.9304923Z * [new tag] ciflow/xpu/168328 -> ciflow/xpu/168328 2025-12-04T09:23:32.9305233Z * [new tag] ciflow/xpu/168950 -> ciflow/xpu/168950 2025-12-04T09:23:32.9305452Z * [new tag] ciflow/xpu/169039 -> ciflow/xpu/169039 2025-12-04T09:23:32.9305568Z * [new tag] ciflow/xpu/169200 -> ciflow/xpu/169200 2025-12-04T09:23:32.9306001Z * [new tag] ciflow/xpu/169203 -> ciflow/xpu/169203 2025-12-04T09:23:32.9306168Z * [new tag] ciflow/xpu/169230 -> ciflow/xpu/169230 2025-12-04T09:23:32.9306307Z * [new tag] ciflow/xpu/169231 -> ciflow/xpu/169231 2025-12-04T09:23:32.9306668Z * [new tag] ciflow/xpu/169241 -> ciflow/xpu/169241 2025-12-04T09:23:32.9307103Z * [new tag] ciflow/xpu/169280 -> ciflow/xpu/169280 2025-12-04T09:23:32.9307584Z * [new tag] ciflow/xpu/169296 -> ciflow/xpu/169296 2025-12-04T09:23:32.9308427Z * [new tag] ciflow/xpu/169353 -> ciflow/xpu/169353 2025-12-04T09:23:32.9309059Z * [new tag] ciflow/xpu/169410 -> ciflow/xpu/169410 2025-12-04T09:23:32.9309203Z * [new tag] ciflow/xpu/169442 -> ciflow/xpu/169442 2025-12-04T09:23:32.9309483Z * [new tag] ciflow/xpu/169555 -> ciflow/xpu/169555 2025-12-04T09:23:32.9310227Z * [new tag] cslpull75 -> cslpull75 2025-12-04T09:23:32.9310646Z * [new tag] cslpull76 -> cslpull76 2025-12-04T09:23:32.9311433Z * [new tag] cslpull77 -> cslpull77 2025-12-04T09:23:32.9311819Z * [new tag] cslpull78 -> cslpull78 2025-12-04T09:23:32.9313178Z * [new tag] cslpull79 -> cslpull79 2025-12-04T09:23:32.9313312Z * [new tag] cslpull80 -> cslpull80 2025-12-04T09:23:32.9316194Z * [new tag] cslpull81 -> cslpull81 2025-12-04T09:23:32.9316487Z * [new tag] cslpull82 -> cslpull82 2025-12-04T09:23:32.9316613Z * [new tag] cslpull83 -> cslpull83 2025-12-04T09:23:32.9316718Z * [new tag] cslpull84 -> cslpull84 2025-12-04T09:23:32.9316829Z * [new tag] cslpull85 -> cslpull85 2025-12-04T09:23:32.9317051Z * [new tag] cslpull86 -> cslpull86 2025-12-04T09:23:32.9317214Z * [new tag] cslpull87 -> cslpull87 2025-12-04T09:23:32.9319657Z * [new tag] cslpull88 -> cslpull88 2025-12-04T09:23:32.9319970Z * [new tag] cslpull89 -> cslpull89 2025-12-04T09:23:32.9320094Z * [new tag] cslpull90 -> cslpull90 2025-12-04T09:23:32.9320203Z * [new tag] cslpull91 -> cslpull91 2025-12-04T09:23:32.9322640Z * [new tag] cslpull92 -> cslpull92 2025-12-04T09:23:32.9322927Z * [new tag] flight_5 -> flight_5 2025-12-04T09:23:32.9323056Z * [new tag] flight_5.1 -> flight_5.1 2025-12-04T09:23:32.9323171Z * [new tag] flight_5.2 -> flight_5.2 2025-12-04T09:23:32.9323293Z * [new tag] flight_5.3 -> flight_5.3 2025-12-04T09:23:32.9324487Z * [new tag] forpull1 -> forpull1 2025-12-04T09:23:32.9324673Z * [new tag] malfet/tag-2ef5611 -> malfet/tag-2ef5611 2025-12-04T09:23:32.9325227Z * [new tag] malfet/tag-317b1a0 -> malfet/tag-317b1a0 2025-12-04T09:23:32.9327382Z * [new tag] malfet/tag-ec6f767 -> malfet/tag-ec6f767 2025-12-04T09:23:32.9327675Z * [new tag] nightly-binary -> nightly-binary 2025-12-04T09:23:32.9327819Z * [new tag] sqzhang_flight4_plus -> sqzhang_flight4_plus 2025-12-04T09:23:32.9328013Z * [new tag] sqzhang_flight_3 -> sqzhang_flight_3 2025-12-04T09:23:32.9328478Z * [new tag] trunk/02d8bd6974cf84b721680d773dbdb1b6f40ce272 -> trunk/02d8bd6974cf84b721680d773dbdb1b6f40ce272 2025-12-04T09:23:32.9329336Z * [new tag] trunk/066997fb38ade71e00d78e9d572e380b5f02bd3e -> trunk/066997fb38ade71e00d78e9d572e380b5f02bd3e 2025-12-04T09:23:32.9330075Z * [new tag] trunk/076e7b19fa1d481ad778d06d2b49ba57d3ce8c88 -> trunk/076e7b19fa1d481ad778d06d2b49ba57d3ce8c88 2025-12-04T09:23:32.9330446Z * [new tag] trunk/07dcc0b83db3211653a38565a24e15acdba75654 -> trunk/07dcc0b83db3211653a38565a24e15acdba75654 2025-12-04T09:23:32.9331005Z * [new tag] trunk/082e96b68dfcd16cab7cfafc4d3d055767dab3eb -> trunk/082e96b68dfcd16cab7cfafc4d3d055767dab3eb 2025-12-04T09:23:32.9331552Z * [new tag] trunk/088048f2fea28ff7d450f65c72419ca45780d30b -> trunk/088048f2fea28ff7d450f65c72419ca45780d30b 2025-12-04T09:23:32.9332081Z * [new tag] trunk/09076941a95c76f4d9ad189d064dfd8baa39e672 -> trunk/09076941a95c76f4d9ad189d064dfd8baa39e672 2025-12-04T09:23:32.9332734Z * [new tag] trunk/0b80a4c62b94402844bf221791c096b0035c6d75 -> trunk/0b80a4c62b94402844bf221791c096b0035c6d75 2025-12-04T09:23:32.9333560Z * [new tag] trunk/0bbbdf1750567a980634ad907a325357ba8ba8f2 -> trunk/0bbbdf1750567a980634ad907a325357ba8ba8f2 2025-12-04T09:23:32.9334018Z * [new tag] trunk/0c281dd78773b2bc17c58ead0e4cd4ac46e775c5 -> trunk/0c281dd78773b2bc17c58ead0e4cd4ac46e775c5 2025-12-04T09:23:32.9334519Z * [new tag] trunk/135f3753c418a6879b1954904184937b67e61688 -> trunk/135f3753c418a6879b1954904184937b67e61688 2025-12-04T09:23:32.9336466Z * [new tag] trunk/15da21026cb13cd20257dc9e96830db108743c10 -> trunk/15da21026cb13cd20257dc9e96830db108743c10 2025-12-04T09:23:32.9336875Z * [new tag] trunk/166efdad2ac827f30fb02504c6017520257f88ec -> trunk/166efdad2ac827f30fb02504c6017520257f88ec 2025-12-04T09:23:32.9337204Z * [new tag] trunk/174272c15fae553d8488140af931f7d8050a313f -> trunk/174272c15fae553d8488140af931f7d8050a313f 2025-12-04T09:23:32.9337576Z * [new tag] trunk/18f3ca08f13b8de61307f5e8cd7d4cccb67e9d11 -> trunk/18f3ca08f13b8de61307f5e8cd7d4cccb67e9d11 2025-12-04T09:23:32.9337858Z * [new tag] trunk/1902eddfe655a15ebcf2c72bd81ade110fdeef63 -> trunk/1902eddfe655a15ebcf2c72bd81ade110fdeef63 2025-12-04T09:23:32.9338493Z * [new tag] trunk/195f92e98d3d66738577f11f22c4b5c8a1c76dd5 -> trunk/195f92e98d3d66738577f11f22c4b5c8a1c76dd5 2025-12-04T09:23:32.9339012Z * [new tag] trunk/1aa13e17de39e3c768ea7aebaad166ce72a06676 -> trunk/1aa13e17de39e3c768ea7aebaad166ce72a06676 2025-12-04T09:23:32.9339561Z * [new tag] trunk/1afe2832f58e24e54a5bfda5a5afa9b96fdea40e -> trunk/1afe2832f58e24e54a5bfda5a5afa9b96fdea40e 2025-12-04T09:23:32.9340013Z * [new tag] trunk/1c87554d74140eaee964ca8b1832cede67f5f520 -> trunk/1c87554d74140eaee964ca8b1832cede67f5f520 2025-12-04T09:23:32.9342586Z * [new tag] trunk/1ccb743b7b5be955f49736c162c4f5004b8a0dd8 -> trunk/1ccb743b7b5be955f49736c162c4f5004b8a0dd8 2025-12-04T09:23:32.9343014Z * [new tag] trunk/1cee47d6ce0a02227185b566593f002dd639ca0c -> trunk/1cee47d6ce0a02227185b566593f002dd639ca0c 2025-12-04T09:23:32.9343375Z * [new tag] trunk/1d21b4df2babe322e5d085ceb6de884eb260a62d -> trunk/1d21b4df2babe322e5d085ceb6de884eb260a62d 2025-12-04T09:23:32.9343733Z * [new tag] trunk/1e34fb2550e4aa650314f7a6d9f6daf4da7478a8 -> trunk/1e34fb2550e4aa650314f7a6d9f6daf4da7478a8 2025-12-04T09:23:32.9344410Z * [new tag] trunk/1e526fb5b1d93bfc70691c5c3955fdffc1b7b7de -> trunk/1e526fb5b1d93bfc70691c5c3955fdffc1b7b7de 2025-12-04T09:23:32.9344666Z * [new tag] trunk/1ee32a8b1f554a312d79bad01ded24f38cd95543 -> trunk/1ee32a8b1f554a312d79bad01ded24f38cd95543 2025-12-04T09:23:32.9344883Z * [new tag] trunk/201e2c4117eb9744594dad6a5c18213d7b4705d7 -> trunk/201e2c4117eb9744594dad6a5c18213d7b4705d7 2025-12-04T09:23:32.9345145Z * [new tag] trunk/2353a0f60eb4b4cb6675907a7fa9fbedc1c02e7f -> trunk/2353a0f60eb4b4cb6675907a7fa9fbedc1c02e7f 2025-12-04T09:23:32.9345485Z * [new tag] trunk/285779b1621cf9f073a062b0889a642d200308d9 -> trunk/285779b1621cf9f073a062b0889a642d200308d9 2025-12-04T09:23:32.9345819Z * [new tag] trunk/2887faaec6295d081580d09fce161201826c6d87 -> trunk/2887faaec6295d081580d09fce161201826c6d87 2025-12-04T09:23:32.9346296Z * [new tag] trunk/296e67c92635443c67b11c0ae1bd045f03ebb7bc -> trunk/296e67c92635443c67b11c0ae1bd045f03ebb7bc 2025-12-04T09:23:32.9346870Z * [new tag] trunk/29856679769b3dede478767e2fe6cfb51197cb25 -> trunk/29856679769b3dede478767e2fe6cfb51197cb25 2025-12-04T09:23:32.9347485Z * [new tag] trunk/29e5455a4740c326ab187c7aa7b5ef98034ea563 -> trunk/29e5455a4740c326ab187c7aa7b5ef98034ea563 2025-12-04T09:23:32.9348019Z * [new tag] trunk/2ac3ef882afb23136adc188975f0a8802fc68adf -> trunk/2ac3ef882afb23136adc188975f0a8802fc68adf 2025-12-04T09:23:32.9348568Z * [new tag] trunk/2bec68e73b64715354af076ad309335f943e36cd -> trunk/2bec68e73b64715354af076ad309335f943e36cd 2025-12-04T09:23:32.9349199Z * [new tag] trunk/2c87367e6f88662cd5cedbd1537748b7948c38e1 -> trunk/2c87367e6f88662cd5cedbd1537748b7948c38e1 2025-12-04T09:23:32.9349948Z * [new tag] trunk/2d1f78fe3ec13820f136a2e0336da12a25f41708 -> trunk/2d1f78fe3ec13820f136a2e0336da12a25f41708 2025-12-04T09:23:32.9350421Z * [new tag] trunk/2df6058f116a65722a0e03073402feb242572d35 -> trunk/2df6058f116a65722a0e03073402feb242572d35 2025-12-04T09:23:32.9350971Z * [new tag] trunk/2e0c2e170fe658c440775c8e5c44228aafcc47ec -> trunk/2e0c2e170fe658c440775c8e5c44228aafcc47ec 2025-12-04T09:23:32.9351848Z * [new tag] trunk/2f9b7dad7b5419b063bd0f2e204de192720ebb94 -> trunk/2f9b7dad7b5419b063bd0f2e204de192720ebb94 2025-12-04T09:23:32.9352263Z * [new tag] trunk/305168768a95d69c444df5cd334bb774edfe06f1 -> trunk/305168768a95d69c444df5cd334bb774edfe06f1 2025-12-04T09:23:32.9353065Z * [new tag] trunk/31fc12773026e8e00f054dd79ad9b2491e693b48 -> trunk/31fc12773026e8e00f054dd79ad9b2491e693b48 2025-12-04T09:23:32.9353503Z * [new tag] trunk/320de0c6b0a3e7c6d2693ea5c28d5d0156ba7991 -> trunk/320de0c6b0a3e7c6d2693ea5c28d5d0156ba7991 2025-12-04T09:23:32.9354163Z * [new tag] trunk/3418bd29475dff06695045fcdf93e7d0dac67da8 -> trunk/3418bd29475dff06695045fcdf93e7d0dac67da8 2025-12-04T09:23:32.9354717Z * [new tag] trunk/34a98608afa0cb5b48f0d6d30432fdd0a2614ddf -> trunk/34a98608afa0cb5b48f0d6d30432fdd0a2614ddf 2025-12-04T09:23:32.9355332Z * [new tag] trunk/35b7a9a26c5923d98aebaa41a031dae21788a9ee -> trunk/35b7a9a26c5923d98aebaa41a031dae21788a9ee 2025-12-04T09:23:32.9355934Z * [new tag] trunk/39d07dbf03a911bdd45d1af78d8638dc92074938 -> trunk/39d07dbf03a911bdd45d1af78d8638dc92074938 2025-12-04T09:23:32.9356626Z * [new tag] trunk/3cd98b4205ada151042cc7ff097a82d4a4b18725 -> trunk/3cd98b4205ada151042cc7ff097a82d4a4b18725 2025-12-04T09:23:32.9356943Z * [new tag] trunk/3d35fd20a78ff4d016fa80f4e5fad37191d7bcae -> trunk/3d35fd20a78ff4d016fa80f4e5fad37191d7bcae 2025-12-04T09:23:32.9357492Z * [new tag] trunk/409a5fee945c46a3edaf5df162812f201bfd7b2f -> trunk/409a5fee945c46a3edaf5df162812f201bfd7b2f 2025-12-04T09:23:32.9358182Z * [new tag] trunk/42e9005cda22da3f1c559c3649218cebd671027c -> trunk/42e9005cda22da3f1c559c3649218cebd671027c 2025-12-04T09:23:32.9358594Z * [new tag] trunk/43b94713bbf340d3c124fde02d0f73add4021247 -> trunk/43b94713bbf340d3c124fde02d0f73add4021247 2025-12-04T09:23:32.9359132Z * [new tag] trunk/44ac69388a4a5eb463dbd2a13f00d1e3b924566c -> trunk/44ac69388a4a5eb463dbd2a13f00d1e3b924566c 2025-12-04T09:23:32.9359703Z * [new tag] trunk/45d14e2497292be06ad36eaa1aaaf7c630a2586a -> trunk/45d14e2497292be06ad36eaa1aaaf7c630a2586a 2025-12-04T09:23:32.9360211Z * [new tag] trunk/45d310ad84854dff730c0b12e577d7998d978686 -> trunk/45d310ad84854dff730c0b12e577d7998d978686 2025-12-04T09:23:32.9361491Z * [new tag] trunk/47b28ddf7bd74b50fa93b307a7d3b183a6d77f54 -> trunk/47b28ddf7bd74b50fa93b307a7d3b183a6d77f54 2025-12-04T09:23:32.9361764Z * [new tag] trunk/481e5ab336275bd3acd5fa8a611b05b4469012af -> trunk/481e5ab336275bd3acd5fa8a611b05b4469012af 2025-12-04T09:23:32.9362178Z * [new tag] trunk/491731647f6b8a9345dcfb3bc9416aea254a7d96 -> trunk/491731647f6b8a9345dcfb3bc9416aea254a7d96 2025-12-04T09:23:32.9363808Z * [new tag] trunk/49a04d26088acc17d948ddd66920f3e16371e873 -> trunk/49a04d26088acc17d948ddd66920f3e16371e873 2025-12-04T09:23:32.9364237Z * [new tag] trunk/4bebc827c47d2f1f0fa1a417a5201a97aef3d985 -> trunk/4bebc827c47d2f1f0fa1a417a5201a97aef3d985 2025-12-04T09:23:32.9364783Z * [new tag] trunk/4c246677784c6a14bc2dbb9ff8773ef0a3a3222f -> trunk/4c246677784c6a14bc2dbb9ff8773ef0a3a3222f 2025-12-04T09:23:32.9365037Z * [new tag] trunk/4cfb47ff548b6d996641058cf04a70e311a4c3aa -> trunk/4cfb47ff548b6d996641058cf04a70e311a4c3aa 2025-12-04T09:23:32.9365345Z * [new tag] trunk/4e0061c1aa52f606dda8cfab0bd7591e588faf2c -> trunk/4e0061c1aa52f606dda8cfab0bd7591e588faf2c 2025-12-04T09:23:32.9366902Z * [new tag] trunk/4fefb8e7e942386ffac764a41b232241f82bea3a -> trunk/4fefb8e7e942386ffac764a41b232241f82bea3a 2025-12-04T09:23:32.9367253Z * [new tag] trunk/503b2640023521f5a35cd9a52fc8033d73a95d0d -> trunk/503b2640023521f5a35cd9a52fc8033d73a95d0d 2025-12-04T09:23:32.9367501Z * [new tag] trunk/518c2b1b3dab9a2ef2849e04b3bc2f20c1c41db9 -> trunk/518c2b1b3dab9a2ef2849e04b3bc2f20c1c41db9 2025-12-04T09:23:32.9372771Z * [new tag] trunk/5191b2fa68ba19960912bfd7fd721c79d76bb1f3 -> trunk/5191b2fa68ba19960912bfd7fd721c79d76bb1f3 2025-12-04T09:23:32.9373216Z * [new tag] trunk/52ac0f0dc4acacd219f1317fbc28ec631c01e07a -> trunk/52ac0f0dc4acacd219f1317fbc28ec631c01e07a 2025-12-04T09:23:32.9378849Z * [new tag] trunk/539ba711b029de9f191070f4f0d12f18f5b7f292 -> trunk/539ba711b029de9f191070f4f0d12f18f5b7f292 2025-12-04T09:23:32.9379285Z * [new tag] trunk/556375b55deebebbc56cb7aef81f4d52f031ba28 -> trunk/556375b55deebebbc56cb7aef81f4d52f031ba28 2025-12-04T09:23:32.9379572Z * [new tag] trunk/55c4ab554845481d0a69a3811937575fe8bb1a66 -> trunk/55c4ab554845481d0a69a3811937575fe8bb1a66 2025-12-04T09:23:32.9379918Z * [new tag] trunk/5634469fda9e5d98869c82c7d03bb08914245f96 -> trunk/5634469fda9e5d98869c82c7d03bb08914245f96 2025-12-04T09:23:32.9380594Z * [new tag] trunk/5778f6ff894686a975a9a23645178ae4c87ad5dc -> trunk/5778f6ff894686a975a9a23645178ae4c87ad5dc 2025-12-04T09:23:32.9380891Z * [new tag] trunk/587d63a3e07de5dc91065f9ef70bcacda9989068 -> trunk/587d63a3e07de5dc91065f9ef70bcacda9989068 2025-12-04T09:23:32.9381128Z * [new tag] trunk/597930f6b568852356ca9795dac76f9e4653adbd -> trunk/597930f6b568852356ca9795dac76f9e4653adbd 2025-12-04T09:23:32.9381356Z * [new tag] trunk/597df3a4e2a67b9fdbe1a89b2f4d74f822274db6 -> trunk/597df3a4e2a67b9fdbe1a89b2f4d74f822274db6 2025-12-04T09:23:32.9381594Z * [new tag] trunk/59abd50e931f4efb21b053f7a2911f5d8a49d883 -> trunk/59abd50e931f4efb21b053f7a2911f5d8a49d883 2025-12-04T09:23:32.9381823Z * [new tag] trunk/5a607febc04c3a2b5824c75f3f60307867439a2c -> trunk/5a607febc04c3a2b5824c75f3f60307867439a2c 2025-12-04T09:23:32.9382058Z * [new tag] trunk/5bf1cdf4755c54ef462b44cb8041b0a57311556b -> trunk/5bf1cdf4755c54ef462b44cb8041b0a57311556b 2025-12-04T09:23:32.9382277Z * [new tag] trunk/5f0030ba63d334d7e8c93a09e41403b89e4c573c -> trunk/5f0030ba63d334d7e8c93a09e41403b89e4c573c 2025-12-04T09:23:32.9382499Z * [new tag] trunk/5f21d27e71268464d362a96c9ac09ea475f7f202 -> trunk/5f21d27e71268464d362a96c9ac09ea475f7f202 2025-12-04T09:23:32.9382728Z * [new tag] trunk/5fafc13038c9988d9ac21fa793fbd5890604b447 -> trunk/5fafc13038c9988d9ac21fa793fbd5890604b447 2025-12-04T09:23:32.9382953Z * [new tag] trunk/61be54a31dc09b59d99b62176fb935aee0b924ef -> trunk/61be54a31dc09b59d99b62176fb935aee0b924ef 2025-12-04T09:23:32.9383181Z * [new tag] trunk/62d3ccd71484ed6a760d909b41487101bbc65719 -> trunk/62d3ccd71484ed6a760d909b41487101bbc65719 2025-12-04T09:23:32.9383399Z * [new tag] trunk/641cdb68ae27668eb441d0e49c87a0602c120c2b -> trunk/641cdb68ae27668eb441d0e49c87a0602c120c2b 2025-12-04T09:23:32.9383616Z * [new tag] trunk/65c4620d6bb0c6029f69762c22b91dda2294da9a -> trunk/65c4620d6bb0c6029f69762c22b91dda2294da9a 2025-12-04T09:23:32.9383839Z * [new tag] trunk/66004b993744b4106bf8afaba71f3c228a804206 -> trunk/66004b993744b4106bf8afaba71f3c228a804206 2025-12-04T09:23:32.9384200Z * [new tag] trunk/6658a04c7ca67acb64512341342e7b3ee13ee386 -> trunk/6658a04c7ca67acb64512341342e7b3ee13ee386 2025-12-04T09:23:32.9384433Z * [new tag] trunk/6864e309092a71f8ab0ca6a4dc7f8a4073fd31c4 -> trunk/6864e309092a71f8ab0ca6a4dc7f8a4073fd31c4 2025-12-04T09:23:32.9384701Z * [new tag] trunk/6c261c6cb07892c90ca19ed51c9705b1659a3f7d -> trunk/6c261c6cb07892c90ca19ed51c9705b1659a3f7d 2025-12-04T09:23:32.9384921Z * [new tag] trunk/6c8b6a043f1628188b6396b3a2a6e000ca68362b -> trunk/6c8b6a043f1628188b6396b3a2a6e000ca68362b 2025-12-04T09:23:32.9385164Z * [new tag] trunk/6ceb4a32f92ae67ce5d7d97931d17401ebf5ffa5 -> trunk/6ceb4a32f92ae67ce5d7d97931d17401ebf5ffa5 2025-12-04T09:23:32.9385387Z * [new tag] trunk/6e404e9b7d6f5fb0de86aa73888c3038248c17f8 -> trunk/6e404e9b7d6f5fb0de86aa73888c3038248c17f8 2025-12-04T09:23:32.9385628Z * [new tag] trunk/6ec30b490aee1db6bcdc7340abddef25784f08ec -> trunk/6ec30b490aee1db6bcdc7340abddef25784f08ec 2025-12-04T09:23:32.9385848Z * [new tag] trunk/6f2783a6c08e1db34275ff25176ffe9aebc30a71 -> trunk/6f2783a6c08e1db34275ff25176ffe9aebc30a71 2025-12-04T09:23:32.9386078Z * [new tag] trunk/6f53fefeb90ad3281119b5cfc4aa9ffd8a066e3d -> trunk/6f53fefeb90ad3281119b5cfc4aa9ffd8a066e3d 2025-12-04T09:23:32.9386310Z * [new tag] trunk/6f7dcf51e46d0c880db1a2f5c70de57adb576f4a -> trunk/6f7dcf51e46d0c880db1a2f5c70de57adb576f4a 2025-12-04T09:23:32.9386540Z * [new tag] trunk/6ff831180d2fa436c7f1c1af3adac641fce9d60e -> trunk/6ff831180d2fa436c7f1c1af3adac641fce9d60e 2025-12-04T09:23:32.9386945Z * [new tag] trunk/70076464a63ab218a7ceefb0e76ccd7131deb8f8 -> trunk/70076464a63ab218a7ceefb0e76ccd7131deb8f8 2025-12-04T09:23:32.9387197Z * [new tag] trunk/70d797a5fc109b20a517646fcaa819477cd0d485 -> trunk/70d797a5fc109b20a517646fcaa819477cd0d485 2025-12-04T09:23:32.9387442Z * [new tag] trunk/7348cb355ff0a6f79cd4871215aea72185748734 -> trunk/7348cb355ff0a6f79cd4871215aea72185748734 2025-12-04T09:23:32.9387740Z * [new tag] trunk/74fe26a1ebe32931783569f2e762e3c2c974901f -> trunk/74fe26a1ebe32931783569f2e762e3c2c974901f 2025-12-04T09:23:32.9388324Z * [new tag] trunk/76aeb8c7e0f795b3fddca134cbea9a69da3ee696 -> trunk/76aeb8c7e0f795b3fddca134cbea9a69da3ee696 2025-12-04T09:23:32.9388593Z * [new tag] trunk/7716da9fb23f27a65b41f9f016a2afadf281c18f -> trunk/7716da9fb23f27a65b41f9f016a2afadf281c18f 2025-12-04T09:23:32.9388853Z * [new tag] trunk/7741edd4ed665f3988052e260863efb508d61a03 -> trunk/7741edd4ed665f3988052e260863efb508d61a03 2025-12-04T09:23:32.9389385Z * [new tag] trunk/78adb3b3df41b45d2368b67226d2f864b78939a6 -> trunk/78adb3b3df41b45d2368b67226d2f864b78939a6 2025-12-04T09:23:32.9389950Z * [new tag] trunk/79d7b178225e5ed24d4e1db74e5abbff848f5fb7 -> trunk/79d7b178225e5ed24d4e1db74e5abbff848f5fb7 2025-12-04T09:23:32.9390407Z * [new tag] trunk/7a1e316115fc6996b3f2336822ba5d5f6179f0c3 -> trunk/7a1e316115fc6996b3f2336822ba5d5f6179f0c3 2025-12-04T09:23:32.9391051Z * [new tag] trunk/7a41b66367c38d0af3e8a90f7be48d6b281e7bca -> trunk/7a41b66367c38d0af3e8a90f7be48d6b281e7bca 2025-12-04T09:23:32.9391510Z * [new tag] trunk/7b7af390ea8541c611d1ce2018a6934188fc197b -> trunk/7b7af390ea8541c611d1ce2018a6934188fc197b 2025-12-04T09:23:32.9392152Z * [new tag] trunk/7ba4680f3755a560af81aa0f688791e367aa3609 -> trunk/7ba4680f3755a560af81aa0f688791e367aa3609 2025-12-04T09:23:32.9392813Z * [new tag] trunk/7bc2a66ded06a0b2549aa51d807edc5dc3e73d1b -> trunk/7bc2a66ded06a0b2549aa51d807edc5dc3e73d1b 2025-12-04T09:23:32.9393354Z * [new tag] trunk/7c648509a7470ace9fb2bae960dd4790f7e943e9 -> trunk/7c648509a7470ace9fb2bae960dd4790f7e943e9 2025-12-04T09:23:32.9393908Z * [new tag] trunk/7cbc2d034cecd21ab5c9707d0a9c525c17143fb8 -> trunk/7cbc2d034cecd21ab5c9707d0a9c525c17143fb8 2025-12-04T09:23:32.9394361Z * [new tag] trunk/7d1bbaf4ba301ea3fba6f3c7bc02d58f6417aaed -> trunk/7d1bbaf4ba301ea3fba6f3c7bc02d58f6417aaed 2025-12-04T09:23:32.9395071Z * [new tag] trunk/7d2a33e4ebf60b217a3cd77feae19231eb996fc8 -> trunk/7d2a33e4ebf60b217a3cd77feae19231eb996fc8 2025-12-04T09:23:32.9395587Z * [new tag] trunk/7eb625920054b1126a7d2d99818aaa188c6ba95e -> trunk/7eb625920054b1126a7d2d99818aaa188c6ba95e 2025-12-04T09:23:32.9396033Z * [new tag] trunk/7f55ba19c456a3d6cc443dd9edb6bb7cca677ead -> trunk/7f55ba19c456a3d6cc443dd9edb6bb7cca677ead 2025-12-04T09:23:32.9396716Z * [new tag] trunk/81af382128efa094d8702e18f2c133760904c718 -> trunk/81af382128efa094d8702e18f2c133760904c718 2025-12-04T09:23:32.9398200Z * [new tag] trunk/84149583d483e9c973c9a0feda70e4f3964947b0 -> trunk/84149583d483e9c973c9a0feda70e4f3964947b0 2025-12-04T09:23:32.9398459Z * [new tag] trunk/85a315917efe82c24306be805c584ec044951c75 -> trunk/85a315917efe82c24306be805c584ec044951c75 2025-12-04T09:23:32.9398818Z * [new tag] trunk/87329491c82a5f8c1cc4ec11d8f55a5de2551ece -> trunk/87329491c82a5f8c1cc4ec11d8f55a5de2551ece 2025-12-04T09:23:32.9399312Z * [new tag] trunk/892640e25aeefa8007c5af837214b4502b6b62a6 -> trunk/892640e25aeefa8007c5af837214b4502b6b62a6 2025-12-04T09:23:32.9400662Z * [new tag] trunk/89e3bbcb5b5321dc8b9520b4d5a8ee60cea1d0b4 -> trunk/89e3bbcb5b5321dc8b9520b4d5a8ee60cea1d0b4 2025-12-04T09:23:32.9400924Z * [new tag] trunk/8c73bbbb02159223c0c97d268a0a74cb78158a1c -> trunk/8c73bbbb02159223c0c97d268a0a74cb78158a1c 2025-12-04T09:23:32.9401381Z * [new tag] trunk/8d56e98c8db988a22cb2dfaeefb30bc7d2a3cc43 -> trunk/8d56e98c8db988a22cb2dfaeefb30bc7d2a3cc43 2025-12-04T09:23:32.9401851Z * [new tag] trunk/8d9dd9603e5ee26c01007f0cd4f018e584840922 -> trunk/8d9dd9603e5ee26c01007f0cd4f018e584840922 2025-12-04T09:23:32.9402912Z * [new tag] trunk/8ef0c0b02b062d75e7c9be2594914a3e784d23ca -> trunk/8ef0c0b02b062d75e7c9be2594914a3e784d23ca 2025-12-04T09:23:32.9403163Z * [new tag] trunk/90b27e7e8352cde97d32ddad24740ef819633f38 -> trunk/90b27e7e8352cde97d32ddad24740ef819633f38 2025-12-04T09:23:32.9403629Z * [new tag] trunk/90f0139e64b2951815d524b6a373bed20c4fbf90 -> trunk/90f0139e64b2951815d524b6a373bed20c4fbf90 2025-12-04T09:23:32.9404028Z * [new tag] trunk/93d0d6838c56af59b0dba794e6aa08f0c1c7799c -> trunk/93d0d6838c56af59b0dba794e6aa08f0c1c7799c 2025-12-04T09:23:32.9404824Z * [new tag] trunk/94ca8d5f1e81fea3ae488650a0fb6795049a9f87 -> trunk/94ca8d5f1e81fea3ae488650a0fb6795049a9f87 2025-12-04T09:23:32.9405215Z * [new tag] trunk/9844fbeadd5cebdf1281d6fbf79164139c352693 -> trunk/9844fbeadd5cebdf1281d6fbf79164139c352693 2025-12-04T09:23:32.9406110Z * [new tag] trunk/99024dec888ec1e50b546822a32b6fb2f35e5eaa -> trunk/99024dec888ec1e50b546822a32b6fb2f35e5eaa 2025-12-04T09:23:32.9406654Z * [new tag] trunk/9a296e640fc88aa44d275b48cd9cc30c573b169d -> trunk/9a296e640fc88aa44d275b48cd9cc30c573b169d 2025-12-04T09:23:32.9408038Z * [new tag] trunk/9b3e34d8589b29f7b4e7fab6f78711b7ca6e4639 -> trunk/9b3e34d8589b29f7b4e7fab6f78711b7ca6e4639 2025-12-04T09:23:32.9408282Z * [new tag] trunk/9cd055e547e9b67a5f9827f8999c38d7eda1bcb8 -> trunk/9cd055e547e9b67a5f9827f8999c38d7eda1bcb8 2025-12-04T09:23:32.9408533Z * [new tag] trunk/9f0df5686cb4ada94f94620acba2e3c3f363b11d -> trunk/9f0df5686cb4ada94f94620acba2e3c3f363b11d 2025-12-04T09:23:32.9409047Z * [new tag] trunk/9f7fceb887d0cfa0326a59b887821c63ff11340a -> trunk/9f7fceb887d0cfa0326a59b887821c63ff11340a 2025-12-04T09:23:32.9410301Z * [new tag] trunk/9f8ef8855d3078d70f7b782540ff2aaf158d6742 -> trunk/9f8ef8855d3078d70f7b782540ff2aaf158d6742 2025-12-04T09:23:32.9410993Z * [new tag] trunk/9fb52efc797b47a1f425a03aa5e47b866d8b1098 -> trunk/9fb52efc797b47a1f425a03aa5e47b866d8b1098 2025-12-04T09:23:32.9411441Z * [new tag] trunk/9ff4a2ebc5762d46c73e46b1b523d7ff349fedfa -> trunk/9ff4a2ebc5762d46c73e46b1b523d7ff349fedfa 2025-12-04T09:23:32.9417545Z * [new tag] trunk/a0f3937b94422354538ebbd47202d5b0e8a3fd0d -> trunk/a0f3937b94422354538ebbd47202d5b0e8a3fd0d 2025-12-04T09:23:32.9422156Z * [new tag] trunk/a15066c28b3145e6edbfc88359d0411d14cfc70c -> trunk/a15066c28b3145e6edbfc88359d0411d14cfc70c 2025-12-04T09:23:32.9423999Z * [new tag] trunk/a20f775e82564d2a9979221ed7f3b8d7cf54ce90 -> trunk/a20f775e82564d2a9979221ed7f3b8d7cf54ce90 2025-12-04T09:23:32.9424255Z * [new tag] trunk/a2973fb00ec002dd4b6bbf07385f066efb259b8c -> trunk/a2973fb00ec002dd4b6bbf07385f066efb259b8c 2025-12-04T09:23:32.9424517Z * [new tag] trunk/a7dc6dab9ad911259d4801c502907e531594db45 -> trunk/a7dc6dab9ad911259d4801c502907e531594db45 2025-12-04T09:23:32.9424758Z * [new tag] trunk/a951a9cee65c01660bbc6e6fded90ecb10fa6109 -> trunk/a951a9cee65c01660bbc6e6fded90ecb10fa6109 2025-12-04T09:23:32.9425005Z * [new tag] trunk/abfa1a6d65c7c159e35c72c25979b9da4971689e -> trunk/abfa1a6d65c7c159e35c72c25979b9da4971689e 2025-12-04T09:23:32.9425234Z * [new tag] trunk/ae3a2395bf66151078e2d201716f7d63ce1c6f3e -> trunk/ae3a2395bf66151078e2d201716f7d63ce1c6f3e 2025-12-04T09:23:32.9425467Z * [new tag] trunk/afdff7f0325080dedac44d080cb5a3b0e65e6c5e -> trunk/afdff7f0325080dedac44d080cb5a3b0e65e6c5e 2025-12-04T09:23:32.9425695Z * [new tag] trunk/b1aed4e7a72c03a38f44543aaea0dae2e9b76d48 -> trunk/b1aed4e7a72c03a38f44543aaea0dae2e9b76d48 2025-12-04T09:23:32.9425937Z * [new tag] trunk/b1decff555cd50e2123c8c6e25cc0d447c411f62 -> trunk/b1decff555cd50e2123c8c6e25cc0d447c411f62 2025-12-04T09:23:32.9426204Z * [new tag] trunk/b2b6b034c9fd08672c40e63ef243556ad4c49bd2 -> trunk/b2b6b034c9fd08672c40e63ef243556ad4c49bd2 2025-12-04T09:23:32.9426443Z * [new tag] trunk/b39813b4a04931682b0491adba2138d01d716d99 -> trunk/b39813b4a04931682b0491adba2138d01d716d99 2025-12-04T09:23:32.9426694Z * [new tag] trunk/b3a7edb2311367974cc7cd764cfb11a5d6758b24 -> trunk/b3a7edb2311367974cc7cd764cfb11a5d6758b24 2025-12-04T09:23:32.9426946Z * [new tag] trunk/b4cc1329c86acaef6d42c1fac7169b8d870ab0d7 -> trunk/b4cc1329c86acaef6d42c1fac7169b8d870ab0d7 2025-12-04T09:23:32.9427194Z * [new tag] trunk/b555c39217f765759954a4f9f9bd1e9b87bed11a -> trunk/b555c39217f765759954a4f9f9bd1e9b87bed11a 2025-12-04T09:23:32.9427435Z * [new tag] trunk/b6b6c80379388b7f9932c3e6a0f9907bf430e417 -> trunk/b6b6c80379388b7f9932c3e6a0f9907bf430e417 2025-12-04T09:23:32.9427688Z * [new tag] trunk/b6b6d912df0b6f4082f8e50b18bd1de1dd7325f4 -> trunk/b6b6d912df0b6f4082f8e50b18bd1de1dd7325f4 2025-12-04T09:23:32.9427930Z * [new tag] trunk/b7d60685f8cbc939b68a20871e90db67e729329b -> trunk/b7d60685f8cbc939b68a20871e90db67e729329b 2025-12-04T09:23:32.9428185Z * [new tag] trunk/b7f6b9a4fc6259f7af068f31868b3119bb1bac3e -> trunk/b7f6b9a4fc6259f7af068f31868b3119bb1bac3e 2025-12-04T09:23:32.9428435Z * [new tag] trunk/b8c4ba3593761e7b2a3ebd86f040fb07b47c02cf -> trunk/b8c4ba3593761e7b2a3ebd86f040fb07b47c02cf 2025-12-04T09:23:32.9428707Z * [new tag] trunk/b9c8f3a4884befb965ff42620ce44a71b04887f5 -> trunk/b9c8f3a4884befb965ff42620ce44a71b04887f5 2025-12-04T09:23:32.9428956Z * [new tag] trunk/ba1412546f3082c0958c077acc2025e4dbc33f1f -> trunk/ba1412546f3082c0958c077acc2025e4dbc33f1f 2025-12-04T09:23:32.9429209Z * [new tag] trunk/bac403c0b38c63bdbcc0c31f1c2b0bc0260f610f -> trunk/bac403c0b38c63bdbcc0c31f1c2b0bc0260f610f 2025-12-04T09:23:32.9429594Z * [new tag] trunk/bb3034198b459401fabeab254e1b99f0115046e2 -> trunk/bb3034198b459401fabeab254e1b99f0115046e2 2025-12-04T09:23:32.9429842Z * [new tag] trunk/bc39b2b3bc7a6e19a42e62bd576974035086fe55 -> trunk/bc39b2b3bc7a6e19a42e62bd576974035086fe55 2025-12-04T09:23:32.9430140Z * [new tag] trunk/bc43d5b297f207a11d83d77ddf0152bdaabe15a8 -> trunk/bc43d5b297f207a11d83d77ddf0152bdaabe15a8 2025-12-04T09:23:32.9430398Z * [new tag] trunk/bc6a4863c7246a6493d16d4ea6eee71ec07c6a09 -> trunk/bc6a4863c7246a6493d16d4ea6eee71ec07c6a09 2025-12-04T09:23:32.9432414Z * [new tag] trunk/bea4912944defdbcb8b061800caab6cbbbd01df5 -> trunk/bea4912944defdbcb8b061800caab6cbbbd01df5 2025-12-04T09:23:32.9432852Z * [new tag] trunk/c04e2c656f48d82d1521b867bbbf03967b9b7564 -> trunk/c04e2c656f48d82d1521b867bbbf03967b9b7564 2025-12-04T09:23:32.9433127Z * [new tag] trunk/c0660bcee27e7d7731634e274576a7081882bede -> trunk/c0660bcee27e7d7731634e274576a7081882bede 2025-12-04T09:23:32.9433406Z * [new tag] trunk/c178ed43d3d99cbefe84fbfb21d6f282b20d62ac -> trunk/c178ed43d3d99cbefe84fbfb21d6f282b20d62ac 2025-12-04T09:23:32.9433642Z * [new tag] trunk/c55b1e8f61d041ee436d697449eb028931d574fb -> trunk/c55b1e8f61d041ee436d697449eb028931d574fb 2025-12-04T09:23:32.9433990Z * [new tag] trunk/c6ae7579fe12fe75f1a8f7043a494c90567273f1 -> trunk/c6ae7579fe12fe75f1a8f7043a494c90567273f1 2025-12-04T09:23:32.9436980Z * [new tag] trunk/c8210e7d94bad5ae21ac389fa4ba8a463c76c4d0 -> trunk/c8210e7d94bad5ae21ac389fa4ba8a463c76c4d0 2025-12-04T09:23:32.9437661Z * [new tag] trunk/cc0853af42122f8185321f542616f4474e717f09 -> trunk/cc0853af42122f8185321f542616f4474e717f09 2025-12-04T09:23:32.9437955Z * [new tag] trunk/cddec6562eabfa390d014fa3741a5659cf9c94c9 -> trunk/cddec6562eabfa390d014fa3741a5659cf9c94c9 2025-12-04T09:23:32.9438260Z * [new tag] trunk/ce5e7e3bf1f4b69a4f4f93d288ba75b906df492a -> trunk/ce5e7e3bf1f4b69a4f4f93d288ba75b906df492a 2025-12-04T09:23:32.9438514Z * [new tag] trunk/d038b0130ec7c20ebcac219301292fd8e98a1ace -> trunk/d038b0130ec7c20ebcac219301292fd8e98a1ace 2025-12-04T09:23:32.9438784Z * [new tag] trunk/d16447dacaf2420ea175f0c275c75da951f57d39 -> trunk/d16447dacaf2420ea175f0c275c75da951f57d39 2025-12-04T09:23:32.9439028Z * [new tag] trunk/d19f1e8cab6810bb2e99141f9976665954c67a50 -> trunk/d19f1e8cab6810bb2e99141f9976665954c67a50 2025-12-04T09:23:32.9444604Z * [new tag] trunk/d1c9f03b2a5af4104721712f8cdffe9b4f340c01 -> trunk/d1c9f03b2a5af4104721712f8cdffe9b4f340c01 2025-12-04T09:23:32.9448952Z * [new tag] trunk/d40f4950f2b7f7aa380a22fe0f6166e71680fbcf -> trunk/d40f4950f2b7f7aa380a22fe0f6166e71680fbcf 2025-12-04T09:23:32.9449236Z * [new tag] trunk/d5038950bacfe36bbf24a47a455fe76901deb8e8 -> trunk/d5038950bacfe36bbf24a47a455fe76901deb8e8 2025-12-04T09:23:32.9449552Z * [new tag] trunk/d54ff42903c2ae0533931ff11d23b35f875bdb3d -> trunk/d54ff42903c2ae0533931ff11d23b35f875bdb3d 2025-12-04T09:23:32.9449799Z * [new tag] trunk/d76697633a2d2b9cced1ae21161849b33bfe7e47 -> trunk/d76697633a2d2b9cced1ae21161849b33bfe7e47 2025-12-04T09:23:32.9450064Z * [new tag] trunk/d78f52b199c547106d4cd9d2856dd0805c118bf1 -> trunk/d78f52b199c547106d4cd9d2856dd0805c118bf1 2025-12-04T09:23:32.9450308Z * [new tag] trunk/d8fd5c6eed28e5004150691d048a3f6785e19a8e -> trunk/d8fd5c6eed28e5004150691d048a3f6785e19a8e 2025-12-04T09:23:32.9450548Z * [new tag] trunk/d900f5e86745dec76713f4b0ef07005ef36b2f5a -> trunk/d900f5e86745dec76713f4b0ef07005ef36b2f5a 2025-12-04T09:23:32.9450800Z * [new tag] trunk/d973dc6b87d763859fe1c5bd1287e3b6b1c49d1b -> trunk/d973dc6b87d763859fe1c5bd1287e3b6b1c49d1b 2025-12-04T09:23:32.9451199Z * [new tag] trunk/d998c03304cb6ede76e1ed535b4ddeb6c2bf40ec -> trunk/d998c03304cb6ede76e1ed535b4ddeb6c2bf40ec 2025-12-04T09:23:32.9451457Z * [new tag] trunk/d9cb8a70833101dbbe16b99520cfbdd70d0a87bf -> trunk/d9cb8a70833101dbbe16b99520cfbdd70d0a87bf 2025-12-04T09:23:32.9451797Z * [new tag] trunk/d9d5e91b43f70eb8637af55db6856d49be391ffd -> trunk/d9d5e91b43f70eb8637af55db6856d49be391ffd 2025-12-04T09:23:32.9452045Z * [new tag] trunk/dd18a75336a4fbd7497955cc5665904724fce889 -> trunk/dd18a75336a4fbd7497955cc5665904724fce889 2025-12-04T09:23:32.9452304Z * [new tag] trunk/ded9bcd61a059bf723e6e84689552962b480ea77 -> trunk/ded9bcd61a059bf723e6e84689552962b480ea77 2025-12-04T09:23:32.9452556Z * [new tag] trunk/dfbd3714d15c37a7b83b322a6b60f997fc00f50c -> trunk/dfbd3714d15c37a7b83b322a6b60f997fc00f50c 2025-12-04T09:23:32.9452812Z * [new tag] trunk/e115f9f4e4b039f8e9a642aaa2bd8254a920541b -> trunk/e115f9f4e4b039f8e9a642aaa2bd8254a920541b 2025-12-04T09:23:32.9453046Z * [new tag] trunk/e3f24fd73ad74c6e7176687986436956c7c18235 -> trunk/e3f24fd73ad74c6e7176687986436956c7c18235 2025-12-04T09:23:32.9453291Z * [new tag] trunk/e7d24d3ff93d1503ba63860b7057438ad93f918e -> trunk/e7d24d3ff93d1503ba63860b7057438ad93f918e 2025-12-04T09:23:32.9453541Z * [new tag] trunk/ea7035f462a0d2830865ee86c832bd101e1427fc -> trunk/ea7035f462a0d2830865ee86c832bd101e1427fc 2025-12-04T09:23:32.9453778Z * [new tag] trunk/eabb7ad2128580ef674446027b95bcf4e21e8df3 -> trunk/eabb7ad2128580ef674446027b95bcf4e21e8df3 2025-12-04T09:23:32.9454024Z * [new tag] trunk/eb5c63652a33da42e7018c23df5f20a3eb4c6ccf -> trunk/eb5c63652a33da42e7018c23df5f20a3eb4c6ccf 2025-12-04T09:23:32.9454263Z * [new tag] trunk/ec2c71f5c85021b8938cdafadce24c15a36fd93e -> trunk/ec2c71f5c85021b8938cdafadce24c15a36fd93e 2025-12-04T09:23:32.9454515Z * [new tag] trunk/ecbcc3f6bf327856b435b259ac63cc2f328c4b4e -> trunk/ecbcc3f6bf327856b435b259ac63cc2f328c4b4e 2025-12-04T09:23:32.9454765Z * [new tag] trunk/ee87bbe876c42575e961b32a0827d76bc9782ca2 -> trunk/ee87bbe876c42575e961b32a0827d76bc9782ca2 2025-12-04T09:23:32.9455012Z * [new tag] trunk/ef019d1d431c4c5a95b594cb90d40a50cd00f5e4 -> trunk/ef019d1d431c4c5a95b594cb90d40a50cd00f5e4 2025-12-04T09:23:32.9455272Z * [new tag] trunk/ef8ecc13830a86c4b231f1aad9aba7851db61b53 -> trunk/ef8ecc13830a86c4b231f1aad9aba7851db61b53 2025-12-04T09:23:32.9455496Z * [new tag] trunk/f1076f5510920044912247b1abb8760cb820f598 -> trunk/f1076f5510920044912247b1abb8760cb820f598 2025-12-04T09:23:32.9455753Z * [new tag] trunk/f2d6a75a00a1d648ca9a0abc6a33e14c3dea6c40 -> trunk/f2d6a75a00a1d648ca9a0abc6a33e14c3dea6c40 2025-12-04T09:23:32.9455999Z * [new tag] trunk/f47dd0ddef1359e5b43e4b962412f67b30ecde56 -> trunk/f47dd0ddef1359e5b43e4b962412f67b30ecde56 2025-12-04T09:23:32.9456242Z * [new tag] trunk/f49d32dfa4730dcfb1b60eeeb369b5889da983c8 -> trunk/f49d32dfa4730dcfb1b60eeeb369b5889da983c8 2025-12-04T09:23:32.9456498Z * [new tag] trunk/f4dedf78fc30fd4b93975787ca6074ee89db9467 -> trunk/f4dedf78fc30fd4b93975787ca6074ee89db9467 2025-12-04T09:23:32.9456746Z * [new tag] trunk/f7c0d03819ebed05c4038f095d66d1b8c54aca17 -> trunk/f7c0d03819ebed05c4038f095d66d1b8c54aca17 2025-12-04T09:23:32.9462692Z * [new tag] trunk/f7e1bd80a063e17453c361837ba6ea2570920a73 -> trunk/f7e1bd80a063e17453c361837ba6ea2570920a73 2025-12-04T09:23:32.9467059Z * [new tag] trunk/f9bd6c53624c7c0ea3772de78498326e84c2f0e7 -> trunk/f9bd6c53624c7c0ea3772de78498326e84c2f0e7 2025-12-04T09:23:32.9469314Z * [new tag] trunk/fb5be221a46b51bfc9509013b0d85bc5a9d4f15b -> trunk/fb5be221a46b51bfc9509013b0d85bc5a9d4f15b 2025-12-04T09:23:32.9469573Z * [new tag] trunk/fdf863d5e1de3b2688c9511e96876e34581dbfd7 -> trunk/fdf863d5e1de3b2688c9511e96876e34581dbfd7 2025-12-04T09:23:32.9470029Z * [new tag] trunk/fe0e65adfc0e7ca6e5f57e6ea8b16bd5cc967307 -> trunk/fe0e65adfc0e7ca6e5f57e6ea8b16bd5cc967307 2025-12-04T09:23:32.9470264Z * [new tag] trunk/fec710bf89173f5355468a7ce1afe9157c3d9009 -> trunk/fec710bf89173f5355468a7ce1afe9157c3d9009 2025-12-04T09:23:32.9470555Z * [new tag] trunk/ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 -> trunk/ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:23:32.9470689Z * [new tag] v0.1.1 -> v0.1.1 2025-12-04T09:23:32.9470791Z * [new tag] v0.1.10 -> v0.1.10 2025-12-04T09:23:32.9470889Z * [new tag] v0.1.11 -> v0.1.11 2025-12-04T09:23:32.9470988Z * [new tag] v0.1.12 -> v0.1.12 2025-12-04T09:23:32.9471081Z * [new tag] v0.1.2 -> v0.1.2 2025-12-04T09:23:32.9471183Z * [new tag] v0.1.3 -> v0.1.3 2025-12-04T09:23:32.9471270Z * [new tag] v0.1.4 -> v0.1.4 2025-12-04T09:23:32.9471358Z * [new tag] v0.1.5 -> v0.1.5 2025-12-04T09:23:32.9471452Z * [new tag] v0.1.6 -> v0.1.6 2025-12-04T09:23:32.9471542Z * [new tag] v0.1.7 -> v0.1.7 2025-12-04T09:23:32.9471637Z * [new tag] v0.1.8 -> v0.1.8 2025-12-04T09:23:32.9471725Z * [new tag] v0.1.9 -> v0.1.9 2025-12-04T09:23:32.9471818Z * [new tag] v0.2.0 -> v0.2.0 2025-12-04T09:23:32.9471921Z * [new tag] v0.3.0 -> v0.3.0 2025-12-04T09:23:32.9472013Z * [new tag] v0.3.1 -> v0.3.1 2025-12-04T09:23:32.9472105Z * [new tag] v0.4.0 -> v0.4.0 2025-12-04T09:23:32.9472213Z * [new tag] v0.4.1 -> v0.4.1 2025-12-04T09:23:32.9472309Z * [new tag] v1.0.0 -> v1.0.0 2025-12-04T09:23:32.9472425Z * [new tag] v1.0.0a0 -> v1.0.0a0 2025-12-04T09:23:32.9472520Z * [new tag] v1.0.1 -> v1.0.1 2025-12-04T09:23:32.9472805Z * [new tag] v1.0rc0 -> v1.0rc0 2025-12-04T09:23:32.9473244Z * [new tag] v1.0rc1 -> v1.0rc1 2025-12-04T09:23:32.9473380Z * [new tag] v1.1.0 -> v1.1.0 2025-12-04T09:23:32.9473507Z * [new tag] v1.1.0a0 -> v1.1.0a0 2025-12-04T09:23:32.9473618Z * [new tag] v1.10.0 -> v1.10.0 2025-12-04T09:23:32.9476459Z * [new tag] v1.10.0-rc1 -> v1.10.0-rc1 2025-12-04T09:23:32.9476634Z * [new tag] v1.10.0-rc2 -> v1.10.0-rc2 2025-12-04T09:23:32.9476757Z * [new tag] v1.10.0-rc3 -> v1.10.0-rc3 2025-12-04T09:23:32.9483047Z * [new tag] v1.10.1 -> v1.10.1 2025-12-04T09:23:32.9486869Z * [new tag] v1.10.1-rc1 -> v1.10.1-rc1 2025-12-04T09:23:32.9491813Z * [new tag] v1.10.2 -> v1.10.2 2025-12-04T09:23:32.9496184Z * [new tag] v1.10.2-rc1 -> v1.10.2-rc1 2025-12-04T09:23:32.9496322Z * [new tag] v1.11.0 -> v1.11.0 2025-12-04T09:23:32.9496447Z * [new tag] v1.11.0-rc1 -> v1.11.0-rc1 2025-12-04T09:23:32.9496550Z * [new tag] v1.11.0-rc2 -> v1.11.0-rc2 2025-12-04T09:23:32.9496675Z * [new tag] v1.11.0-rc3 -> v1.11.0-rc3 2025-12-04T09:23:32.9496775Z * [new tag] v1.11.0-rc4 -> v1.11.0-rc4 2025-12-04T09:23:32.9497040Z * [new tag] v1.11.0-rc5 -> v1.11.0-rc5 2025-12-04T09:23:32.9497150Z * [new tag] v1.11.0-rc6 -> v1.11.0-rc6 2025-12-04T09:23:32.9497249Z * [new tag] v1.11.0-rc7 -> v1.11.0-rc7 2025-12-04T09:23:32.9497408Z * [new tag] v1.12.0 -> v1.12.0 2025-12-04T09:23:32.9497510Z * [new tag] v1.12.0-rc1 -> v1.12.0-rc1 2025-12-04T09:23:32.9497609Z * [new tag] v1.12.0-rc2 -> v1.12.0-rc2 2025-12-04T09:23:32.9497715Z * [new tag] v1.12.0-rc3 -> v1.12.0-rc3 2025-12-04T09:23:32.9497813Z * [new tag] v1.12.0-rc4 -> v1.12.0-rc4 2025-12-04T09:23:32.9497910Z * [new tag] v1.12.0-rc5 -> v1.12.0-rc5 2025-12-04T09:23:32.9498017Z * [new tag] v1.12.0-rc6 -> v1.12.0-rc6 2025-12-04T09:23:32.9498119Z * [new tag] v1.12.0-rc7 -> v1.12.0-rc7 2025-12-04T09:23:32.9498225Z * [new tag] v1.12.0-rc8 -> v1.12.0-rc8 2025-12-04T09:23:32.9498321Z * [new tag] v1.12.1 -> v1.12.1 2025-12-04T09:23:32.9498420Z * [new tag] v1.12.1-rc1 -> v1.12.1-rc1 2025-12-04T09:23:32.9498529Z * [new tag] v1.12.1-rc2 -> v1.12.1-rc2 2025-12-04T09:23:32.9498630Z * [new tag] v1.12.1-rc3 -> v1.12.1-rc3 2025-12-04T09:23:32.9498736Z * [new tag] v1.12.1-rc4 -> v1.12.1-rc4 2025-12-04T09:23:32.9498835Z * [new tag] v1.12.1-rc5 -> v1.12.1-rc5 2025-12-04T09:23:32.9498930Z * [new tag] v1.13.0 -> v1.13.0 2025-12-04T09:23:32.9499035Z * [new tag] v1.13.0-rc1 -> v1.13.0-rc1 2025-12-04T09:23:32.9499137Z * [new tag] v1.13.0-rc2 -> v1.13.0-rc2 2025-12-04T09:23:32.9499234Z * [new tag] v1.13.0-rc3 -> v1.13.0-rc3 2025-12-04T09:23:32.9499340Z * [new tag] v1.13.0-rc4 -> v1.13.0-rc4 2025-12-04T09:23:32.9499438Z * [new tag] v1.13.0-rc5 -> v1.13.0-rc5 2025-12-04T09:23:32.9499547Z * [new tag] v1.13.0-rc6 -> v1.13.0-rc6 2025-12-04T09:23:32.9499643Z * [new tag] v1.13.1 -> v1.13.1 2025-12-04T09:23:32.9499741Z * [new tag] v1.13.1-rc1 -> v1.13.1-rc1 2025-12-04T09:23:32.9499852Z * [new tag] v1.2.0 -> v1.2.0 2025-12-04T09:23:32.9499956Z * [new tag] v1.2.0a0 -> v1.2.0a0 2025-12-04T09:23:32.9500059Z * [new tag] v1.3.0 -> v1.3.0 2025-12-04T09:23:32.9500621Z * [new tag] v1.3.0a0 -> v1.3.0a0 2025-12-04T09:23:32.9500719Z * [new tag] v1.3.1 -> v1.3.1 2025-12-04T09:23:32.9500822Z * [new tag] v1.4.0 -> v1.4.0 2025-12-04T09:23:32.9500923Z * [new tag] v1.4.0a0 -> v1.4.0a0 2025-12-04T09:23:32.9501025Z * [new tag] v1.4.1 -> v1.4.1 2025-12-04T09:23:32.9501129Z * [new tag] v1.5.0 -> v1.5.0 2025-12-04T09:23:32.9501237Z * [new tag] v1.5.0-rc1 -> v1.5.0-rc1 2025-12-04T09:23:32.9501374Z * [new tag] v1.5.0-rc2 -> v1.5.0-rc2 2025-12-04T09:23:32.9506349Z * [new tag] v1.5.0-rc3 -> v1.5.0-rc3 2025-12-04T09:23:32.9509997Z * [new tag] v1.5.0-rc4 -> v1.5.0-rc4 2025-12-04T09:23:32.9510136Z * [new tag] v1.5.0-rc5 -> v1.5.0-rc5 2025-12-04T09:23:32.9510493Z * [new tag] v1.5.1 -> v1.5.1 2025-12-04T09:23:32.9510599Z * [new tag] v1.5.1-rc1 -> v1.5.1-rc1 2025-12-04T09:23:32.9510721Z * [new tag] v1.6.0 -> v1.6.0 2025-12-04T09:23:32.9510825Z * [new tag] v1.6.0-rc1 -> v1.6.0-rc1 2025-12-04T09:23:32.9510984Z * [new tag] v1.6.0-rc2 -> v1.6.0-rc2 2025-12-04T09:23:32.9511094Z * [new tag] v1.6.0-rc3 -> v1.6.0-rc3 2025-12-04T09:23:32.9511190Z * [new tag] v1.6.0-rc4 -> v1.6.0-rc4 2025-12-04T09:23:32.9511297Z * [new tag] v1.6.0-rc5 -> v1.6.0-rc5 2025-12-04T09:23:32.9511395Z * [new tag] v1.6.0-rc6 -> v1.6.0-rc6 2025-12-04T09:23:32.9511494Z * [new tag] v1.6.0-rc7 -> v1.6.0-rc7 2025-12-04T09:23:32.9511604Z * [new tag] v1.7.0 -> v1.7.0 2025-12-04T09:23:32.9511703Z * [new tag] v1.7.0-rc1 -> v1.7.0-rc1 2025-12-04T09:23:32.9516693Z * [new tag] v1.7.0-rc2 -> v1.7.0-rc2 2025-12-04T09:23:32.9518650Z * [new tag] v1.7.0-rc3 -> v1.7.0-rc3 2025-12-04T09:23:32.9518787Z * [new tag] v1.7.0-rc4 -> v1.7.0-rc4 2025-12-04T09:23:32.9518913Z * [new tag] v1.7.1 -> v1.7.1 2025-12-04T09:23:32.9519021Z * [new tag] v1.7.1-rc1 -> v1.7.1-rc1 2025-12-04T09:23:32.9519144Z * [new tag] v1.7.1-rc2 -> v1.7.1-rc2 2025-12-04T09:23:32.9519244Z * [new tag] v1.7.1-rc3 -> v1.7.1-rc3 2025-12-04T09:23:32.9519348Z * [new tag] v1.8.0 -> v1.8.0 2025-12-04T09:23:32.9519465Z * [new tag] v1.8.0-rc1 -> v1.8.0-rc1 2025-12-04T09:23:32.9519565Z * [new tag] v1.8.0-rc2 -> v1.8.0-rc2 2025-12-04T09:23:32.9519667Z * [new tag] v1.8.0-rc3 -> v1.8.0-rc3 2025-12-04T09:23:32.9519779Z * [new tag] v1.8.0-rc4 -> v1.8.0-rc4 2025-12-04T09:23:32.9519886Z * [new tag] v1.8.0-rc5 -> v1.8.0-rc5 2025-12-04T09:23:32.9519994Z * [new tag] v1.8.1 -> v1.8.1 2025-12-04T09:23:32.9520096Z * [new tag] v1.8.1-rc1 -> v1.8.1-rc1 2025-12-04T09:23:32.9520196Z * [new tag] v1.8.1-rc2 -> v1.8.1-rc2 2025-12-04T09:23:32.9524758Z * [new tag] v1.8.1-rc3 -> v1.8.1-rc3 2025-12-04T09:23:32.9528542Z * [new tag] v1.8.2 -> v1.8.2 2025-12-04T09:23:32.9530955Z * [new tag] v1.8.2-rc1 -> v1.8.2-rc1 2025-12-04T09:23:32.9534417Z * [new tag] v1.9.0 -> v1.9.0 2025-12-04T09:23:32.9534602Z * [new tag] v1.9.0-rc1 -> v1.9.0-rc1 2025-12-04T09:23:32.9534732Z * [new tag] v1.9.0-rc2 -> v1.9.0-rc2 2025-12-04T09:23:32.9534849Z * [new tag] v1.9.0-rc3 -> v1.9.0-rc3 2025-12-04T09:23:32.9535062Z * [new tag] v1.9.0-rc4 -> v1.9.0-rc4 2025-12-04T09:23:32.9535241Z * [new tag] v1.9.1 -> v1.9.1 2025-12-04T09:23:32.9535352Z * [new tag] v1.9.1-rc1 -> v1.9.1-rc1 2025-12-04T09:23:32.9535531Z * [new tag] v1.9.1-rc2 -> v1.9.1-rc2 2025-12-04T09:23:32.9540847Z * [new tag] v2.0.0 -> v2.0.0 2025-12-04T09:23:32.9543156Z * [new tag] v2.0.0-rc1 -> v2.0.0-rc1 2025-12-04T09:23:32.9543612Z * [new tag] v2.0.0-rc2 -> v2.0.0-rc2 2025-12-04T09:23:32.9543801Z * [new tag] v2.0.0-rc3 -> v2.0.0-rc3 2025-12-04T09:23:32.9543906Z * [new tag] v2.0.0-rc4 -> v2.0.0-rc4 2025-12-04T09:23:32.9544129Z * [new tag] v2.0.0-rc5 -> v2.0.0-rc5 2025-12-04T09:23:32.9544544Z * [new tag] v2.0.0-rc6 -> v2.0.0-rc6 2025-12-04T09:23:32.9544770Z * [new tag] v2.0.1 -> v2.0.1 2025-12-04T09:23:32.9545022Z * [new tag] v2.0.1-rc1 -> v2.0.1-rc1 2025-12-04T09:23:32.9545126Z * [new tag] v2.0.1-rc2 -> v2.0.1-rc2 2025-12-04T09:23:32.9545362Z * [new tag] v2.0.1-rc3 -> v2.0.1-rc3 2025-12-04T09:23:32.9545794Z * [new tag] v2.0.1-rc4 -> v2.0.1-rc4 2025-12-04T09:23:32.9545966Z * [new tag] v2.1.0 -> v2.1.0 2025-12-04T09:23:32.9546070Z * [new tag] v2.1.0-rc1 -> v2.1.0-rc1 2025-12-04T09:23:32.9546167Z * [new tag] v2.1.0-rc2 -> v2.1.0-rc2 2025-12-04T09:23:32.9546276Z * [new tag] v2.1.0-rc3 -> v2.1.0-rc3 2025-12-04T09:23:32.9546399Z * [new tag] v2.1.0-rc4 -> v2.1.0-rc4 2025-12-04T09:23:32.9546503Z * [new tag] v2.1.0-rc5 -> v2.1.0-rc5 2025-12-04T09:23:32.9546601Z * [new tag] v2.1.0-rc6 -> v2.1.0-rc6 2025-12-04T09:23:32.9546699Z * [new tag] v2.1.1 -> v2.1.1 2025-12-04T09:23:32.9546803Z * [new tag] v2.1.1-rc1 -> v2.1.1-rc1 2025-12-04T09:23:32.9546899Z * [new tag] v2.1.1-rc2 -> v2.1.1-rc2 2025-12-04T09:23:32.9546991Z * [new tag] v2.1.1-rc3 -> v2.1.1-rc3 2025-12-04T09:23:32.9547097Z * [new tag] v2.1.1-rc4 -> v2.1.1-rc4 2025-12-04T09:23:32.9547188Z * [new tag] v2.1.1-rc5 -> v2.1.1-rc5 2025-12-04T09:23:32.9547287Z * [new tag] v2.1.1-rc6 -> v2.1.1-rc6 2025-12-04T09:23:32.9547376Z * [new tag] v2.1.2 -> v2.1.2 2025-12-04T09:23:32.9547474Z * [new tag] v2.1.2-rc1 -> v2.1.2-rc1 2025-12-04T09:23:32.9547575Z * [new tag] v2.1.2-rc2 -> v2.1.2-rc2 2025-12-04T09:23:32.9547667Z * [new tag] v2.1.2-rc3 -> v2.1.2-rc3 2025-12-04T09:23:32.9547765Z * [new tag] v2.2.0 -> v2.2.0 2025-12-04T09:23:32.9547857Z * [new tag] v2.2.0-rc1 -> v2.2.0-rc1 2025-12-04T09:23:32.9547950Z * [new tag] v2.2.0-rc2 -> v2.2.0-rc2 2025-12-04T09:23:32.9548058Z * [new tag] v2.2.0-rc3 -> v2.2.0-rc3 2025-12-04T09:23:32.9548156Z * [new tag] v2.2.0-rc4 -> v2.2.0-rc4 2025-12-04T09:23:32.9548253Z * [new tag] v2.2.0-rc5 -> v2.2.0-rc5 2025-12-04T09:23:32.9548353Z * [new tag] v2.2.0-rc6 -> v2.2.0-rc6 2025-12-04T09:23:32.9548792Z * [new tag] v2.2.0-rc7 -> v2.2.0-rc7 2025-12-04T09:23:32.9549070Z * [new tag] v2.2.0-rc8 -> v2.2.0-rc8 2025-12-04T09:23:32.9550101Z * [new tag] v2.2.1 -> v2.2.1 2025-12-04T09:23:32.9550355Z * [new tag] v2.2.1-rc1 -> v2.2.1-rc1 2025-12-04T09:23:32.9550864Z * [new tag] v2.2.1-rc2 -> v2.2.1-rc2 2025-12-04T09:23:32.9554647Z * [new tag] v2.2.1-rc3 -> v2.2.1-rc3 2025-12-04T09:23:32.9560702Z * [new tag] v2.2.2 -> v2.2.2 2025-12-04T09:23:32.9565570Z * [new tag] v2.2.2-rc1 -> v2.2.2-rc1 2025-12-04T09:23:32.9570335Z * [new tag] v2.2.2-rc2 -> v2.2.2-rc2 2025-12-04T09:23:32.9575185Z * [new tag] v2.2.2-rc3 -> v2.2.2-rc3 2025-12-04T09:23:32.9577452Z * [new tag] v2.3.0 -> v2.3.0 2025-12-04T09:23:32.9577633Z * [new tag] v2.3.0-rc1 -> v2.3.0-rc1 2025-12-04T09:23:32.9577742Z * [new tag] v2.3.0-rc10 -> v2.3.0-rc10 2025-12-04T09:23:32.9577849Z * [new tag] v2.3.0-rc11 -> v2.3.0-rc11 2025-12-04T09:23:32.9577944Z * [new tag] v2.3.0-rc12 -> v2.3.0-rc12 2025-12-04T09:23:32.9578044Z * [new tag] v2.3.0-rc2 -> v2.3.0-rc2 2025-12-04T09:23:32.9578147Z * [new tag] v2.3.0-rc3 -> v2.3.0-rc3 2025-12-04T09:23:32.9578250Z * [new tag] v2.3.0-rc4 -> v2.3.0-rc4 2025-12-04T09:23:32.9578352Z * [new tag] v2.3.0-rc5 -> v2.3.0-rc5 2025-12-04T09:23:32.9578446Z * [new tag] v2.3.0-rc6 -> v2.3.0-rc6 2025-12-04T09:23:32.9578544Z * [new tag] v2.3.0-rc7 -> v2.3.0-rc7 2025-12-04T09:23:32.9578649Z * [new tag] v2.3.0-rc8 -> v2.3.0-rc8 2025-12-04T09:23:32.9578743Z * [new tag] v2.3.0-rc9 -> v2.3.0-rc9 2025-12-04T09:23:32.9578849Z * [new tag] v2.3.1 -> v2.3.1 2025-12-04T09:23:32.9578941Z * [new tag] v2.3.1-rc1 -> v2.3.1-rc1 2025-12-04T09:23:32.9579034Z * [new tag] v2.3.1-rc2 -> v2.3.1-rc2 2025-12-04T09:23:32.9579134Z * [new tag] v2.3.1-rc3 -> v2.3.1-rc3 2025-12-04T09:23:32.9579231Z * [new tag] v2.4.0 -> v2.4.0 2025-12-04T09:23:32.9579324Z * [new tag] v2.4.0-rc1 -> v2.4.0-rc1 2025-12-04T09:23:32.9579426Z * [new tag] v2.4.0-rc2 -> v2.4.0-rc2 2025-12-04T09:23:32.9579519Z * [new tag] v2.4.0-rc3 -> v2.4.0-rc3 2025-12-04T09:23:32.9579621Z * [new tag] v2.4.0-rc4 -> v2.4.0-rc4 2025-12-04T09:23:32.9579715Z * [new tag] v2.4.0-rc5 -> v2.4.0-rc5 2025-12-04T09:23:32.9579809Z * [new tag] v2.4.0-rc6 -> v2.4.0-rc6 2025-12-04T09:23:32.9579911Z * [new tag] v2.4.0-rc7 -> v2.4.0-rc7 2025-12-04T09:23:32.9580005Z * [new tag] v2.4.0-rc8 -> v2.4.0-rc8 2025-12-04T09:23:32.9580100Z * [new tag] v2.4.0-rc9 -> v2.4.0-rc9 2025-12-04T09:23:32.9580203Z * [new tag] v2.4.1 -> v2.4.1 2025-12-04T09:23:32.9580295Z * [new tag] v2.4.1-rc1 -> v2.4.1-rc1 2025-12-04T09:23:32.9580396Z * [new tag] v2.4.1-rc2 -> v2.4.1-rc2 2025-12-04T09:23:32.9580488Z * [new tag] v2.4.1-rc3 -> v2.4.1-rc3 2025-12-04T09:23:32.9580582Z * [new tag] v2.5.0 -> v2.5.0 2025-12-04T09:23:32.9580683Z * [new tag] v2.5.0-rc1 -> v2.5.0-rc1 2025-12-04T09:23:32.9580781Z * [new tag] v2.5.0-rc10 -> v2.5.0-rc10 2025-12-04T09:23:32.9580882Z * [new tag] v2.5.0-rc2 -> v2.5.0-rc2 2025-12-04T09:23:32.9580975Z * [new tag] v2.5.0-rc3 -> v2.5.0-rc3 2025-12-04T09:23:32.9581067Z * [new tag] v2.5.0-rc4 -> v2.5.0-rc4 2025-12-04T09:23:32.9581167Z * [new tag] v2.5.0-rc5 -> v2.5.0-rc5 2025-12-04T09:23:32.9581311Z * [new tag] v2.5.0-rc6 -> v2.5.0-rc6 2025-12-04T09:23:32.9581405Z * [new tag] v2.5.0-rc7 -> v2.5.0-rc7 2025-12-04T09:23:32.9581510Z * [new tag] v2.5.0-rc8 -> v2.5.0-rc8 2025-12-04T09:23:32.9581635Z * [new tag] v2.5.0-rc9 -> v2.5.0-rc9 2025-12-04T09:23:32.9581736Z * [new tag] v2.5.1 -> v2.5.1 2025-12-04T09:23:32.9581829Z * [new tag] v2.5.1-rc1 -> v2.5.1-rc1 2025-12-04T09:23:32.9581920Z * [new tag] v2.6.0 -> v2.6.0 2025-12-04T09:23:32.9582024Z * [new tag] v2.6.0-rc1 -> v2.6.0-rc1 2025-12-04T09:23:32.9582127Z * [new tag] v2.6.0-rc2 -> v2.6.0-rc2 2025-12-04T09:23:32.9582231Z * [new tag] v2.6.0-rc3 -> v2.6.0-rc3 2025-12-04T09:23:32.9582329Z * [new tag] v2.6.0-rc4 -> v2.6.0-rc4 2025-12-04T09:23:32.9582424Z * [new tag] v2.6.0-rc5 -> v2.6.0-rc5 2025-12-04T09:23:32.9582537Z * [new tag] v2.6.0-rc6 -> v2.6.0-rc6 2025-12-04T09:23:32.9582630Z * [new tag] v2.6.0-rc7 -> v2.6.0-rc7 2025-12-04T09:23:32.9584933Z * [new tag] v2.6.0-rc8 -> v2.6.0-rc8 2025-12-04T09:23:32.9585070Z * [new tag] v2.6.0-rc9 -> v2.6.0-rc9 2025-12-04T09:23:32.9585175Z * [new tag] v2.7.0 -> v2.7.0 2025-12-04T09:23:32.9587047Z * [new tag] v2.7.0-rc1 -> v2.7.0-rc1 2025-12-04T09:23:32.9587199Z * [new tag] v2.7.0-rc10 -> v2.7.0-rc10 2025-12-04T09:23:32.9587316Z * [new tag] v2.7.0-rc2 -> v2.7.0-rc2 2025-12-04T09:23:32.9587439Z * [new tag] v2.7.0-rc3 -> v2.7.0-rc3 2025-12-04T09:23:32.9587544Z * [new tag] v2.7.0-rc4 -> v2.7.0-rc4 2025-12-04T09:23:32.9587651Z * [new tag] v2.7.0-rc5 -> v2.7.0-rc5 2025-12-04T09:23:32.9587752Z * [new tag] v2.7.0-rc6 -> v2.7.0-rc6 2025-12-04T09:23:32.9588144Z * [new tag] v2.7.0-rc7 -> v2.7.0-rc7 2025-12-04T09:23:32.9588483Z * [new tag] v2.7.0-rc8 -> v2.7.0-rc8 2025-12-04T09:23:32.9589205Z * [new tag] v2.7.0-rc9 -> v2.7.0-rc9 2025-12-04T09:23:32.9589475Z * [new tag] v2.7.1 -> v2.7.1 2025-12-04T09:23:32.9590524Z * [new tag] v2.7.1-rc1 -> v2.7.1-rc1 2025-12-04T09:23:32.9590860Z * [new tag] v2.7.1-rc2 -> v2.7.1-rc2 2025-12-04T09:23:32.9592182Z * [new tag] v2.7.1-rc3 -> v2.7.1-rc3 2025-12-04T09:23:32.9592395Z * [new tag] v2.7.1-rc4 -> v2.7.1-rc4 2025-12-04T09:23:32.9593051Z * [new tag] v2.7.1-rc5 -> v2.7.1-rc5 2025-12-04T09:23:32.9593572Z * [new tag] v2.8.0 -> v2.8.0 2025-12-04T09:23:32.9594382Z * [new tag] v2.8.0-rc1 -> v2.8.0-rc1 2025-12-04T09:23:32.9594823Z * [new tag] v2.8.0-rc2 -> v2.8.0-rc2 2025-12-04T09:23:32.9595765Z * [new tag] v2.8.0-rc3 -> v2.8.0-rc3 2025-12-04T09:23:32.9596433Z * [new tag] v2.8.0-rc4 -> v2.8.0-rc4 2025-12-04T09:23:32.9596857Z * [new tag] v2.8.0-rc5 -> v2.8.0-rc5 2025-12-04T09:23:32.9597859Z * [new tag] v2.8.0-rc6 -> v2.8.0-rc6 2025-12-04T09:23:32.9598173Z * [new tag] v2.8.0-rc7 -> v2.8.0-rc7 2025-12-04T09:23:32.9599505Z * [new tag] v2.8.0-rc8 -> v2.8.0-rc8 2025-12-04T09:23:32.9599755Z * [new tag] v2.9.0 -> v2.9.0 2025-12-04T09:23:32.9600754Z * [new tag] v2.9.0-rc1 -> v2.9.0-rc1 2025-12-04T09:23:32.9601135Z * [new tag] v2.9.0-rc10 -> v2.9.0-rc10 2025-12-04T09:23:32.9601973Z * [new tag] v2.9.0-rc11 -> v2.9.0-rc11 2025-12-04T09:23:32.9604703Z * [new tag] v2.9.0-rc2 -> v2.9.0-rc2 2025-12-04T09:23:32.9604905Z * [new tag] v2.9.0-rc3 -> v2.9.0-rc3 2025-12-04T09:23:32.9605004Z * [new tag] v2.9.0-rc4 -> v2.9.0-rc4 2025-12-04T09:23:32.9609572Z * [new tag] v2.9.0-rc5 -> v2.9.0-rc5 2025-12-04T09:23:32.9609814Z * [new tag] v2.9.0-rc6 -> v2.9.0-rc6 2025-12-04T09:23:32.9609925Z * [new tag] v2.9.0-rc7 -> v2.9.0-rc7 2025-12-04T09:23:32.9610025Z * [new tag] v2.9.0-rc8 -> v2.9.0-rc8 2025-12-04T09:23:32.9610120Z * [new tag] v2.9.0-rc9 -> v2.9.0-rc9 2025-12-04T09:23:32.9610214Z * [new tag] v2.9.1 -> v2.9.1 2025-12-04T09:23:32.9613886Z * [new tag] v2.9.1-rc1 -> v2.9.1-rc1 2025-12-04T09:23:32.9614153Z * [new tag] v2.9.1-rc2 -> v2.9.1-rc2 2025-12-04T09:23:32.9614326Z * [new tag] viable/strict/1759343184 -> viable/strict/1759343184 2025-12-04T09:23:32.9614461Z * [new tag] viable/strict/1759346540 -> viable/strict/1759346540 2025-12-04T09:23:32.9614585Z * [new tag] viable/strict/1759348181 -> viable/strict/1759348181 2025-12-04T09:23:32.9614719Z * [new tag] viable/strict/1759350324 -> viable/strict/1759350324 2025-12-04T09:23:32.9614864Z * [new tag] viable/strict/1759351793 -> viable/strict/1759351793 2025-12-04T09:23:32.9617445Z * [new tag] viable/strict/1759353844 -> viable/strict/1759353844 2025-12-04T09:23:32.9617591Z * [new tag] viable/strict/1759355374 -> viable/strict/1759355374 2025-12-04T09:23:32.9617742Z * [new tag] viable/strict/1759357472 -> viable/strict/1759357472 2025-12-04T09:23:32.9617859Z * [new tag] viable/strict/1759361002 -> viable/strict/1759361002 2025-12-04T09:23:32.9618052Z * [new tag] viable/strict/1759362585 -> viable/strict/1759362585 2025-12-04T09:23:32.9623711Z * [new tag] viable/strict/1759365359 -> viable/strict/1759365359 2025-12-04T09:23:32.9627942Z * [new tag] viable/strict/1759370089 -> viable/strict/1759370089 2025-12-04T09:23:32.9628124Z * [new tag] viable/strict/1759377554 -> viable/strict/1759377554 2025-12-04T09:23:32.9628272Z * [new tag] viable/strict/1759379133 -> viable/strict/1759379133 2025-12-04T09:23:32.9628390Z * [new tag] viable/strict/1759389871 -> viable/strict/1759389871 2025-12-04T09:23:32.9628511Z * [new tag] viable/strict/1759393562 -> viable/strict/1759393562 2025-12-04T09:23:32.9628662Z * [new tag] viable/strict/1759395076 -> viable/strict/1759395076 2025-12-04T09:23:32.9628784Z * [new tag] viable/strict/1759398579 -> viable/strict/1759398579 2025-12-04T09:23:32.9628915Z * [new tag] viable/strict/1759404142 -> viable/strict/1759404142 2025-12-04T09:23:32.9629033Z * [new tag] viable/strict/1759405773 -> viable/strict/1759405773 2025-12-04T09:23:32.9629151Z * [new tag] viable/strict/1759408041 -> viable/strict/1759408041 2025-12-04T09:23:32.9629275Z * [new tag] viable/strict/1759411593 -> viable/strict/1759411593 2025-12-04T09:23:32.9629622Z * [new tag] viable/strict/1759427395 -> viable/strict/1759427395 2025-12-04T09:23:32.9629747Z * [new tag] viable/strict/1759434582 -> viable/strict/1759434582 2025-12-04T09:23:32.9629877Z * [new tag] viable/strict/1759436720 -> viable/strict/1759436720 2025-12-04T09:23:32.9630097Z * [new tag] viable/strict/1759440219 -> viable/strict/1759440219 2025-12-04T09:23:32.9630243Z * [new tag] viable/strict/1759441948 -> viable/strict/1759441948 2025-12-04T09:23:32.9630383Z * [new tag] viable/strict/1759443860 -> viable/strict/1759443860 2025-12-04T09:23:32.9630519Z * [new tag] viable/strict/1759445377 -> viable/strict/1759445377 2025-12-04T09:23:32.9630650Z * [new tag] viable/strict/1759447415 -> viable/strict/1759447415 2025-12-04T09:23:32.9630790Z * [new tag] viable/strict/1759451750 -> viable/strict/1759451750 2025-12-04T09:23:32.9630930Z * [new tag] viable/strict/1759453910 -> viable/strict/1759453910 2025-12-04T09:23:32.9631069Z * [new tag] viable/strict/1759456483 -> viable/strict/1759456483 2025-12-04T09:23:32.9631205Z * [new tag] viable/strict/1759459279 -> viable/strict/1759459279 2025-12-04T09:23:32.9631352Z * [new tag] viable/strict/1759460742 -> viable/strict/1759460742 2025-12-04T09:23:32.9631478Z * [new tag] viable/strict/1759462025 -> viable/strict/1759462025 2025-12-04T09:23:32.9631621Z * [new tag] viable/strict/1759469086 -> viable/strict/1759469086 2025-12-04T09:23:32.9631763Z * [new tag] viable/strict/1759470581 -> viable/strict/1759470581 2025-12-04T09:23:32.9631906Z * [new tag] viable/strict/1759472786 -> viable/strict/1759472786 2025-12-04T09:23:32.9632049Z * [new tag] viable/strict/1759476294 -> viable/strict/1759476294 2025-12-04T09:23:32.9632184Z * [new tag] viable/strict/1759479963 -> viable/strict/1759479963 2025-12-04T09:23:32.9632541Z * [new tag] viable/strict/1759492177 -> viable/strict/1759492177 2025-12-04T09:23:32.9633271Z * [new tag] viable/strict/1759519278 -> viable/strict/1759519278 2025-12-04T09:23:32.9634137Z * [new tag] viable/strict/1759524580 -> viable/strict/1759524580 2025-12-04T09:23:32.9634450Z * [new tag] viable/strict/1759528193 -> viable/strict/1759528193 2025-12-04T09:23:32.9637094Z * [new tag] viable/strict/1759533797 -> viable/strict/1759533797 2025-12-04T09:23:32.9637410Z * [new tag] viable/strict/1759542780 -> viable/strict/1759542780 2025-12-04T09:23:32.9637581Z * [new tag] viable/strict/1759549779 -> viable/strict/1759549779 2025-12-04T09:23:32.9637713Z * [new tag] viable/strict/1759555455 -> viable/strict/1759555455 2025-12-04T09:23:32.9637996Z * [new tag] viable/strict/1759559176 -> viable/strict/1759559176 2025-12-04T09:23:32.9638276Z * [new tag] viable/strict/1759560629 -> viable/strict/1759560629 2025-12-04T09:23:32.9639672Z * [new tag] viable/strict/1759569848 -> viable/strict/1759569848 2025-12-04T09:23:32.9639888Z * [new tag] viable/strict/1759571382 -> viable/strict/1759571382 2025-12-04T09:23:32.9640280Z * [new tag] viable/strict/1759573474 -> viable/strict/1759573474 2025-12-04T09:23:32.9641844Z * [new tag] viable/strict/1759618187 -> viable/strict/1759618187 2025-12-04T09:23:32.9642165Z * [new tag] viable/strict/1759626742 -> viable/strict/1759626742 2025-12-04T09:23:32.9642452Z * [new tag] viable/strict/1759632427 -> viable/strict/1759632427 2025-12-04T09:23:32.9642815Z * [new tag] viable/strict/1759634971 -> viable/strict/1759634971 2025-12-04T09:23:32.9643350Z * [new tag] viable/strict/1759661382 -> viable/strict/1759661382 2025-12-04T09:23:32.9644260Z * [new tag] viable/strict/1759663294 -> viable/strict/1759663294 2025-12-04T09:23:32.9644455Z * [new tag] viable/strict/1759708178 -> viable/strict/1759708178 2025-12-04T09:23:32.9648713Z * [new tag] viable/strict/1759715695 -> viable/strict/1759715695 2025-12-04T09:23:32.9648889Z * [new tag] viable/strict/1759728293 -> viable/strict/1759728293 2025-12-04T09:23:32.9649023Z * [new tag] viable/strict/1759735513 -> viable/strict/1759735513 2025-12-04T09:23:32.9649145Z * [new tag] viable/strict/1759739177 -> viable/strict/1759739177 2025-12-04T09:23:32.9649264Z * [new tag] viable/strict/1759758635 -> viable/strict/1759758635 2025-12-04T09:23:32.9649398Z * [new tag] viable/strict/1759765784 -> viable/strict/1759765784 2025-12-04T09:23:32.9649711Z * [new tag] viable/strict/1759767948 -> viable/strict/1759767948 2025-12-04T09:23:32.9649898Z * [new tag] viable/strict/1759771461 -> viable/strict/1759771461 2025-12-04T09:23:32.9650240Z * [new tag] viable/strict/1759776706 -> viable/strict/1759776706 2025-12-04T09:23:32.9651996Z * [new tag] viable/strict/1759782317 -> viable/strict/1759782317 2025-12-04T09:23:32.9652324Z * [new tag] viable/strict/1759783777 -> viable/strict/1759783777 2025-12-04T09:23:32.9652482Z * [new tag] viable/strict/1759785815 -> viable/strict/1759785815 2025-12-04T09:23:32.9653029Z * [new tag] viable/strict/1759789459 -> viable/strict/1759789459 2025-12-04T09:23:32.9653782Z * [new tag] viable/strict/1759790974 -> viable/strict/1759790974 2025-12-04T09:23:32.9654064Z * [new tag] viable/strict/1759794583 -> viable/strict/1759794583 2025-12-04T09:23:32.9657685Z * [new tag] viable/strict/1759797408 -> viable/strict/1759797408 2025-12-04T09:23:32.9657904Z * [new tag] viable/strict/1759799518 -> viable/strict/1759799518 2025-12-04T09:23:32.9658035Z * [new tag] viable/strict/1759804909 -> viable/strict/1759804909 2025-12-04T09:23:32.9658186Z * [new tag] viable/strict/1759807643 -> viable/strict/1759807643 2025-12-04T09:23:32.9658316Z * [new tag] viable/strict/1759809089 -> viable/strict/1759809089 2025-12-04T09:23:32.9658442Z * [new tag] viable/strict/1759811145 -> viable/strict/1759811145 2025-12-04T09:23:32.9658566Z * [new tag] viable/strict/1759812581 -> viable/strict/1759812581 2025-12-04T09:23:32.9658730Z * [new tag] viable/strict/1759814683 -> viable/strict/1759814683 2025-12-04T09:23:32.9659376Z * [new tag] viable/strict/1759821889 -> viable/strict/1759821889 2025-12-04T09:23:32.9659934Z * [new tag] viable/strict/1759823376 -> viable/strict/1759823376 2025-12-04T09:23:32.9660564Z * [new tag] viable/strict/1759827107 -> viable/strict/1759827107 2025-12-04T09:23:32.9661030Z * [new tag] viable/strict/1759830577 -> viable/strict/1759830577 2025-12-04T09:23:32.9661921Z * [new tag] viable/strict/1759832720 -> viable/strict/1759832720 2025-12-04T09:23:32.9662272Z * [new tag] viable/strict/1759842063 -> viable/strict/1759842063 2025-12-04T09:23:32.9663504Z * [new tag] viable/strict/1759847121 -> viable/strict/1759847121 2025-12-04T09:23:32.9663760Z * [new tag] viable/strict/1759850721 -> viable/strict/1759850721 2025-12-04T09:23:32.9664317Z * [new tag] viable/strict/1759857870 -> viable/strict/1759857870 2025-12-04T09:23:32.9667947Z * [new tag] viable/strict/1759863143 -> viable/strict/1759863143 2025-12-04T09:23:32.9668297Z * [new tag] viable/strict/1759875874 -> viable/strict/1759875874 2025-12-04T09:23:32.9668445Z * [new tag] viable/strict/1759877385 -> viable/strict/1759877385 2025-12-04T09:23:32.9668576Z * [new tag] viable/strict/1759883801 -> viable/strict/1759883801 2025-12-04T09:23:32.9668759Z * [new tag] viable/strict/1759885922 -> viable/strict/1759885922 2025-12-04T09:23:32.9668896Z * [new tag] viable/strict/1759888488 -> viable/strict/1759888488 2025-12-04T09:23:32.9669020Z * [new tag] viable/strict/1759895471 -> viable/strict/1759895471 2025-12-04T09:23:32.9669151Z * [new tag] viable/strict/1759904803 -> viable/strict/1759904803 2025-12-04T09:23:32.9669657Z * [new tag] viable/strict/1759908300 -> viable/strict/1759908300 2025-12-04T09:23:32.9670253Z * [new tag] viable/strict/1759915520 -> viable/strict/1759915520 2025-12-04T09:23:32.9670776Z * [new tag] viable/strict/1759916978 -> viable/strict/1759916978 2025-12-04T09:23:32.9671324Z * [new tag] viable/strict/1759930024 -> viable/strict/1759930024 2025-12-04T09:23:32.9671835Z * [new tag] viable/strict/1759948122 -> viable/strict/1759948122 2025-12-04T09:23:32.9672837Z * [new tag] viable/strict/1759952983 -> viable/strict/1759952983 2025-12-04T09:23:32.9673182Z * [new tag] viable/strict/1759955121 -> viable/strict/1759955121 2025-12-04T09:23:32.9674070Z * [new tag] viable/strict/1759962298 -> viable/strict/1759962298 2025-12-04T09:23:32.9674460Z * [new tag] viable/strict/1759965837 -> viable/strict/1759965837 2025-12-04T09:23:32.9677565Z * [new tag] viable/strict/1759970213 -> viable/strict/1759970213 2025-12-04T09:23:32.9677735Z * [new tag] viable/strict/1759974894 -> viable/strict/1759974894 2025-12-04T09:23:32.9677891Z * [new tag] viable/strict/1759977763 -> viable/strict/1759977763 2025-12-04T09:23:32.9678023Z * [new tag] viable/strict/1759979241 -> viable/strict/1759979241 2025-12-04T09:23:32.9678155Z * [new tag] viable/strict/1759985417 -> viable/strict/1759985417 2025-12-04T09:23:32.9678808Z * [new tag] viable/strict/1759987490 -> viable/strict/1759987490 2025-12-04T09:23:32.9678945Z * [new tag] viable/strict/1759996180 -> viable/strict/1759996180 2025-12-04T09:23:32.9681550Z * [new tag] viable/strict/1760065682 -> viable/strict/1760065682 2025-12-04T09:23:32.9681865Z * [new tag] viable/strict/1760066894 -> viable/strict/1760066894 2025-12-04T09:23:32.9682045Z * [new tag] viable/strict/1760070345 -> viable/strict/1760070345 2025-12-04T09:23:32.9682250Z * [new tag] viable/strict/1760089782 -> viable/strict/1760089782 2025-12-04T09:23:32.9682418Z * [new tag] viable/strict/1760091921 -> viable/strict/1760091921 2025-12-04T09:23:32.9682729Z * [new tag] viable/strict/1760127924 -> viable/strict/1760127924 2025-12-04T09:23:32.9688396Z * [new tag] viable/strict/1760129489 -> viable/strict/1760129489 2025-12-04T09:23:32.9694496Z * [new tag] viable/strict/1760132980 -> viable/strict/1760132980 2025-12-04T09:23:32.9696842Z * [new tag] viable/strict/1760135060 -> viable/strict/1760135060 2025-12-04T09:23:32.9702259Z * [new tag] viable/strict/1760215782 -> viable/strict/1760215782 2025-12-04T09:23:32.9704531Z * [new tag] viable/strict/1760273849 -> viable/strict/1760273849 2025-12-04T09:23:32.9708095Z * [new tag] viable/strict/1760275517 -> viable/strict/1760275517 2025-12-04T09:23:32.9708249Z * [new tag] viable/strict/1760276979 -> viable/strict/1760276979 2025-12-04T09:23:32.9708620Z * [new tag] viable/strict/1760279007 -> viable/strict/1760279007 2025-12-04T09:23:32.9708738Z * [new tag] viable/strict/1760286328 -> viable/strict/1760286328 2025-12-04T09:23:32.9708863Z * [new tag] viable/strict/1760493304 -> viable/strict/1760493304 2025-12-04T09:23:32.9709050Z * [new tag] viable/strict/1760496298 -> viable/strict/1760496298 2025-12-04T09:23:32.9709173Z * [new tag] viable/strict/1760518396 -> viable/strict/1760518396 2025-12-04T09:23:32.9709299Z * [new tag] viable/strict/1760534864 -> viable/strict/1760534864 2025-12-04T09:23:32.9709419Z * [new tag] viable/strict/1760549062 -> viable/strict/1760549062 2025-12-04T09:23:32.9709554Z * [new tag] viable/strict/1760552799 -> viable/strict/1760552799 2025-12-04T09:23:32.9709702Z * [new tag] viable/strict/1760554355 -> viable/strict/1760554355 2025-12-04T09:23:32.9709828Z * [new tag] viable/strict/1760556275 -> viable/strict/1760556275 2025-12-04T09:23:32.9709971Z * [new tag] viable/strict/1760564979 -> viable/strict/1760564979 2025-12-04T09:23:32.9710111Z * [new tag] viable/strict/1760567049 -> viable/strict/1760567049 2025-12-04T09:23:32.9710249Z * [new tag] viable/strict/1760568585 -> viable/strict/1760568585 2025-12-04T09:23:32.9710382Z * [new tag] viable/strict/1760570630 -> viable/strict/1760570630 2025-12-04T09:23:32.9710519Z * [new tag] viable/strict/1760572180 -> viable/strict/1760572180 2025-12-04T09:23:32.9710658Z * [new tag] viable/strict/1760575094 -> viable/strict/1760575094 2025-12-04T09:23:32.9710795Z * [new tag] viable/strict/1760579709 -> viable/strict/1760579709 2025-12-04T09:23:32.9710920Z * [new tag] viable/strict/1760582614 -> viable/strict/1760582614 2025-12-04T09:23:32.9711051Z * [new tag] viable/strict/1760586815 -> viable/strict/1760586815 2025-12-04T09:23:32.9711190Z * [new tag] viable/strict/1760588829 -> viable/strict/1760588829 2025-12-04T09:23:32.9711574Z * [new tag] viable/strict/1760590200 -> viable/strict/1760590200 2025-12-04T09:23:32.9711925Z * [new tag] viable/strict/1760592311 -> viable/strict/1760592311 2025-12-04T09:23:32.9712279Z * [new tag] viable/strict/1760619733 -> viable/strict/1760619733 2025-12-04T09:23:32.9712615Z * [new tag] viable/strict/1760628335 -> viable/strict/1760628335 2025-12-04T09:23:32.9713204Z * [new tag] viable/strict/1760635490 -> viable/strict/1760635490 2025-12-04T09:23:32.9713542Z * [new tag] viable/strict/1760640743 -> viable/strict/1760640743 2025-12-04T09:23:32.9713869Z * [new tag] viable/strict/1760642528 -> viable/strict/1760642528 2025-12-04T09:23:32.9714185Z * [new tag] viable/strict/1760646330 -> viable/strict/1760646330 2025-12-04T09:23:32.9714502Z * [new tag] viable/strict/1760666101 -> viable/strict/1760666101 2025-12-04T09:23:32.9714804Z * [new tag] viable/strict/1760668990 -> viable/strict/1760668990 2025-12-04T09:23:32.9715127Z * [new tag] viable/strict/1760670600 -> viable/strict/1760670600 2025-12-04T09:23:32.9715437Z * [new tag] viable/strict/1760671704 -> viable/strict/1760671704 2025-12-04T09:23:32.9715754Z * [new tag] viable/strict/1760673121 -> viable/strict/1760673121 2025-12-04T09:23:32.9716061Z * [new tag] viable/strict/1760675352 -> viable/strict/1760675352 2025-12-04T09:23:32.9716394Z * [new tag] viable/strict/1760696731 -> viable/strict/1760696731 2025-12-04T09:23:32.9716757Z * [new tag] viable/strict/1760723515 -> viable/strict/1760723515 2025-12-04T09:23:32.9717090Z * [new tag] viable/strict/1760727234 -> viable/strict/1760727234 2025-12-04T09:23:32.9717404Z * [new tag] viable/strict/1760730578 -> viable/strict/1760730578 2025-12-04T09:23:32.9717711Z * [new tag] viable/strict/1760732726 -> viable/strict/1760732726 2025-12-04T09:23:32.9718064Z * [new tag] viable/strict/1760734180 -> viable/strict/1760734180 2025-12-04T09:23:32.9718357Z * [new tag] viable/strict/1760736251 -> viable/strict/1760736251 2025-12-04T09:23:32.9718649Z * [new tag] viable/strict/1760737772 -> viable/strict/1760737772 2025-12-04T09:23:32.9718954Z * [new tag] viable/strict/1760758005 -> viable/strict/1760758005 2025-12-04T09:23:32.9719251Z * [new tag] viable/strict/1760761532 -> viable/strict/1760761532 2025-12-04T09:23:32.9719636Z * [new tag] viable/strict/1760802581 -> viable/strict/1760802581 2025-12-04T09:23:32.9720069Z * [new tag] viable/strict/1760827772 -> viable/strict/1760827772 2025-12-04T09:23:32.9720374Z * [new tag] viable/strict/1760834524 -> viable/strict/1760834524 2025-12-04T09:23:32.9720672Z * [new tag] viable/strict/1760845009 -> viable/strict/1760845009 2025-12-04T09:23:32.9720999Z * [new tag] viable/strict/1760876836 -> viable/strict/1760876836 2025-12-04T09:23:32.9721306Z * [new tag] viable/strict/1760880329 -> viable/strict/1760880329 2025-12-04T09:23:32.9721602Z * [new tag] viable/strict/1760888987 -> viable/strict/1760888987 2025-12-04T09:23:32.9721893Z * [new tag] viable/strict/1760912664 -> viable/strict/1760912664 2025-12-04T09:23:32.9722187Z * [new tag] viable/strict/1760925321 -> viable/strict/1760925321 2025-12-04T09:23:32.9722480Z * [new tag] viable/strict/1760931488 -> viable/strict/1760931488 2025-12-04T09:23:32.9722915Z * [new tag] viable/strict/1760932693 -> viable/strict/1760932693 2025-12-04T09:23:32.9723248Z * [new tag] viable/strict/1761004184 -> viable/strict/1761004184 2025-12-04T09:23:32.9723946Z * [new tag] viable/strict/1761014748 -> viable/strict/1761014748 2025-12-04T09:23:32.9724281Z * [new tag] viable/strict/1761017491 -> viable/strict/1761017491 2025-12-04T09:23:32.9724804Z * [new tag] viable/strict/1761018806 -> viable/strict/1761018806 2025-12-04T09:23:32.9725425Z * [new tag] viable/strict/1761020754 -> viable/strict/1761020754 2025-12-04T09:23:32.9726020Z * [new tag] viable/strict/1761024303 -> viable/strict/1761024303 2025-12-04T09:23:32.9726543Z * [new tag] viable/strict/1761029582 -> viable/strict/1761029582 2025-12-04T09:23:32.9727236Z * [new tag] viable/strict/1761031535 -> viable/strict/1761031535 2025-12-04T09:23:32.9728117Z * [new tag] viable/strict/1761035196 -> viable/strict/1761035196 2025-12-04T09:23:32.9728513Z * [new tag] viable/strict/1761045825 -> viable/strict/1761045825 2025-12-04T09:23:32.9729123Z * [new tag] viable/strict/1761054796 -> viable/strict/1761054796 2025-12-04T09:23:32.9733093Z * [new tag] viable/strict/1761060314 -> viable/strict/1761060314 2025-12-04T09:23:32.9737185Z * [new tag] viable/strict/1761071198 -> viable/strict/1761071198 2025-12-04T09:23:32.9737567Z * [new tag] viable/strict/1761074628 -> viable/strict/1761074628 2025-12-04T09:23:32.9737878Z * [new tag] viable/strict/1761078351 -> viable/strict/1761078351 2025-12-04T09:23:32.9738169Z * [new tag] viable/strict/1761079822 -> viable/strict/1761079822 2025-12-04T09:23:32.9738621Z * [new tag] viable/strict/1761081873 -> viable/strict/1761081873 2025-12-04T09:23:32.9738918Z * [new tag] viable/strict/1761083392 -> viable/strict/1761083392 2025-12-04T09:23:32.9739202Z * [new tag] viable/strict/1761085465 -> viable/strict/1761085465 2025-12-04T09:23:32.9739497Z * [new tag] viable/strict/1761089099 -> viable/strict/1761089099 2025-12-04T09:23:32.9740212Z * [new tag] viable/strict/1761095535 -> viable/strict/1761095535 2025-12-04T09:23:32.9740515Z * [new tag] viable/strict/1761098119 -> viable/strict/1761098119 2025-12-04T09:23:32.9740802Z * [new tag] viable/strict/1761101330 -> viable/strict/1761101330 2025-12-04T09:23:32.9741101Z * [new tag] viable/strict/1761114425 -> viable/strict/1761114425 2025-12-04T09:23:32.9741407Z * [new tag] viable/strict/1761116036 -> viable/strict/1761116036 2025-12-04T09:23:32.9741703Z * [new tag] viable/strict/1761119379 -> viable/strict/1761119379 2025-12-04T09:23:32.9741995Z * [new tag] viable/strict/1761121601 -> viable/strict/1761121601 2025-12-04T09:23:32.9742287Z * [new tag] viable/strict/1761123234 -> viable/strict/1761123234 2025-12-04T09:23:32.9742586Z * [new tag] viable/strict/1761126621 -> viable/strict/1761126621 2025-12-04T09:23:32.9742878Z * [new tag] viable/strict/1761132259 -> viable/strict/1761132259 2025-12-04T09:23:32.9743340Z * [new tag] viable/strict/1761146746 -> viable/strict/1761146746 2025-12-04T09:23:32.9743649Z * [new tag] viable/strict/1761164752 -> viable/strict/1761164752 2025-12-04T09:23:32.9743949Z * [new tag] viable/strict/1761166198 -> viable/strict/1761166198 2025-12-04T09:23:32.9744245Z * [new tag] viable/strict/1761175424 -> viable/strict/1761175424 2025-12-04T09:23:32.9749904Z * [new tag] viable/strict/1761176983 -> viable/strict/1761176983 2025-12-04T09:23:32.9755520Z * [new tag] viable/strict/1761179891 -> viable/strict/1761179891 2025-12-04T09:23:32.9759911Z * [new tag] viable/strict/1761181930 -> viable/strict/1761181930 2025-12-04T09:23:32.9764433Z * [new tag] viable/strict/1761184516 -> viable/strict/1761184516 2025-12-04T09:23:32.9769807Z * [new tag] viable/strict/1761190179 -> viable/strict/1761190179 2025-12-04T09:23:32.9772180Z * [new tag] viable/strict/1761193558 -> viable/strict/1761193558 2025-12-04T09:23:32.9777695Z * [new tag] viable/strict/1761207990 -> viable/strict/1761207990 2025-12-04T09:23:32.9780086Z * [new tag] viable/strict/1761229539 -> viable/strict/1761229539 2025-12-04T09:23:32.9785654Z * [new tag] viable/strict/1761244031 -> viable/strict/1761244031 2025-12-04T09:23:32.9786198Z * [new tag] viable/strict/1761248986 -> viable/strict/1761248986 2025-12-04T09:23:32.9786641Z * [new tag] viable/strict/1761259791 -> viable/strict/1761259791 2025-12-04T09:23:32.9787392Z * [new tag] viable/strict/1761266139 -> viable/strict/1761266139 2025-12-04T09:23:32.9787808Z * [new tag] viable/strict/1761268316 -> viable/strict/1761268316 2025-12-04T09:23:32.9788122Z * [new tag] viable/strict/1761273805 -> viable/strict/1761273805 2025-12-04T09:23:32.9788433Z * [new tag] viable/strict/1761275261 -> viable/strict/1761275261 2025-12-04T09:23:32.9788733Z * [new tag] viable/strict/1761277913 -> viable/strict/1761277913 2025-12-04T09:23:32.9789048Z * [new tag] viable/strict/1761290701 -> viable/strict/1761290701 2025-12-04T09:23:32.9789384Z * [new tag] viable/strict/1761294396 -> viable/strict/1761294396 2025-12-04T09:23:32.9789882Z * [new tag] viable/strict/1761303047 -> viable/strict/1761303047 2025-12-04T09:23:32.9790218Z * [new tag] viable/strict/1761335388 -> viable/strict/1761335388 2025-12-04T09:23:32.9790552Z * [new tag] viable/strict/1761337551 -> viable/strict/1761337551 2025-12-04T09:23:32.9790934Z * [new tag] viable/strict/1761339007 -> viable/strict/1761339007 2025-12-04T09:23:32.9791263Z * [new tag] viable/strict/1761341050 -> viable/strict/1761341050 2025-12-04T09:23:32.9791597Z * [new tag] viable/strict/1761346188 -> viable/strict/1761346188 2025-12-04T09:23:32.9791930Z * [new tag] viable/strict/1761349792 -> viable/strict/1761349792 2025-12-04T09:23:32.9792261Z * [new tag] viable/strict/1761352620 -> viable/strict/1761352620 2025-12-04T09:23:32.9792582Z * [new tag] viable/strict/1761354730 -> viable/strict/1761354730 2025-12-04T09:23:32.9793128Z * [new tag] viable/strict/1761357298 -> viable/strict/1761357298 2025-12-04T09:23:32.9793464Z * [new tag] viable/strict/1761360201 -> viable/strict/1761360201 2025-12-04T09:23:32.9793825Z * [new tag] viable/strict/1761361753 -> viable/strict/1761361753 2025-12-04T09:23:32.9794162Z * [new tag] viable/strict/1761364351 -> viable/strict/1761364351 2025-12-04T09:23:32.9794499Z * [new tag] viable/strict/1761366338 -> viable/strict/1761366338 2025-12-04T09:23:32.9794824Z * [new tag] viable/strict/1761367802 -> viable/strict/1761367802 2025-12-04T09:23:32.9795147Z * [new tag] viable/strict/1761369889 -> viable/strict/1761369889 2025-12-04T09:23:32.9795468Z * [new tag] viable/strict/1761371385 -> viable/strict/1761371385 2025-12-04T09:23:32.9795784Z * [new tag] viable/strict/1761373581 -> viable/strict/1761373581 2025-12-04T09:23:32.9796097Z * [new tag] viable/strict/1761375054 -> viable/strict/1761375054 2025-12-04T09:23:32.9796399Z * [new tag] viable/strict/1761421785 -> viable/strict/1761421785 2025-12-04T09:23:32.9796710Z * [new tag] viable/strict/1761434614 -> viable/strict/1761434614 2025-12-04T09:23:32.9797030Z * [new tag] viable/strict/1761439254 -> viable/strict/1761439254 2025-12-04T09:23:32.9797338Z * [new tag] viable/strict/1761454187 -> viable/strict/1761454187 2025-12-04T09:23:32.9797651Z * [new tag] viable/strict/1761459991 -> viable/strict/1761459991 2025-12-04T09:23:32.9797966Z * [new tag] viable/strict/1761470668 -> viable/strict/1761470668 2025-12-04T09:23:32.9798286Z * [new tag] viable/strict/1761472188 -> viable/strict/1761472188 2025-12-04T09:23:32.9798600Z * [new tag] viable/strict/1761503178 -> viable/strict/1761503178 2025-12-04T09:23:32.9798910Z * [new tag] viable/strict/1761517492 -> viable/strict/1761517492 2025-12-04T09:23:32.9799234Z * [new tag] viable/strict/1761518981 -> viable/strict/1761518981 2025-12-04T09:23:32.9799552Z * [new tag] viable/strict/1761533609 -> viable/strict/1761533609 2025-12-04T09:23:32.9799867Z * [new tag] viable/strict/1761546438 -> viable/strict/1761546438 2025-12-04T09:23:32.9800178Z * [new tag] viable/strict/1761548133 -> viable/strict/1761548133 2025-12-04T09:23:32.9800497Z * [new tag] viable/strict/1761555186 -> viable/strict/1761555186 2025-12-04T09:23:32.9800815Z * [new tag] viable/strict/1761557178 -> viable/strict/1761557178 2025-12-04T09:23:32.9801124Z * [new tag] viable/strict/1761560772 -> viable/strict/1761560772 2025-12-04T09:23:32.9801436Z * [new tag] viable/strict/1761562266 -> viable/strict/1761562266 2025-12-04T09:23:32.9801806Z * [new tag] viable/strict/1761564260 -> viable/strict/1761564260 2025-12-04T09:23:32.9802119Z * [new tag] viable/strict/1761568072 -> viable/strict/1761568072 2025-12-04T09:23:32.9802447Z * [new tag] viable/strict/1761571683 -> viable/strict/1761571683 2025-12-04T09:23:32.9802826Z * [new tag] viable/strict/1761580199 -> viable/strict/1761580199 2025-12-04T09:23:32.9803146Z * [new tag] viable/strict/1761587383 -> viable/strict/1761587383 2025-12-04T09:23:32.9803454Z * [new tag] viable/strict/1761591165 -> viable/strict/1761591165 2025-12-04T09:23:32.9803776Z * [new tag] viable/strict/1761594575 -> viable/strict/1761594575 2025-12-04T09:23:32.9804101Z * [new tag] viable/strict/1761596710 -> viable/strict/1761596710 2025-12-04T09:23:32.9804425Z * [new tag] viable/strict/1761598189 -> viable/strict/1761598189 2025-12-04T09:23:32.9804735Z * [new tag] viable/strict/1761600254 -> viable/strict/1761600254 2025-12-04T09:23:32.9805049Z * [new tag] viable/strict/1761603879 -> viable/strict/1761603879 2025-12-04T09:23:32.9805359Z * [new tag] viable/strict/1761605429 -> viable/strict/1761605429 2025-12-04T09:23:32.9805846Z * [new tag] viable/strict/1761607468 -> viable/strict/1761607468 2025-12-04T09:23:32.9806179Z * [new tag] viable/strict/1761608983 -> viable/strict/1761608983 2025-12-04T09:23:32.9806498Z * [new tag] viable/strict/1761611846 -> viable/strict/1761611846 2025-12-04T09:23:32.9806834Z * [new tag] viable/strict/1761613922 -> viable/strict/1761613922 2025-12-04T09:23:32.9807141Z * [new tag] viable/strict/1761616504 -> viable/strict/1761616504 2025-12-04T09:23:32.9807456Z * [new tag] viable/strict/1761619599 -> viable/strict/1761619599 2025-12-04T09:23:32.9807776Z * [new tag] viable/strict/1761686693 -> viable/strict/1761686693 2025-12-04T09:23:32.9808089Z * [new tag] viable/strict/1761688179 -> viable/strict/1761688179 2025-12-04T09:23:32.9808394Z * [new tag] viable/strict/1761691973 -> viable/strict/1761691973 2025-12-04T09:23:32.9808712Z * [new tag] viable/strict/1761693884 -> viable/strict/1761693884 2025-12-04T09:23:32.9809022Z * [new tag] viable/strict/1761695389 -> viable/strict/1761695389 2025-12-04T09:23:32.9809335Z * [new tag] viable/strict/1761698408 -> viable/strict/1761698408 2025-12-04T09:23:32.9809676Z * [new tag] viable/strict/1761702931 -> viable/strict/1761702931 2025-12-04T09:23:32.9809985Z * [new tag] viable/strict/1761706307 -> viable/strict/1761706307 2025-12-04T09:23:32.9810300Z * [new tag] viable/strict/1761709065 -> viable/strict/1761709065 2025-12-04T09:23:32.9810606Z * [new tag] viable/strict/1761710285 -> viable/strict/1761710285 2025-12-04T09:23:32.9810921Z * [new tag] viable/strict/1761711983 -> viable/strict/1761711983 2025-12-04T09:23:32.9811236Z * [new tag] viable/strict/1761713514 -> viable/strict/1761713514 2025-12-04T09:23:32.9811544Z * [new tag] viable/strict/1761715523 -> viable/strict/1761715523 2025-12-04T09:23:32.9811862Z * [new tag] viable/strict/1761727973 -> viable/strict/1761727973 2025-12-04T09:23:32.9812173Z * [new tag] viable/strict/1761751558 -> viable/strict/1761751558 2025-12-04T09:23:32.9812487Z * [new tag] viable/strict/1761755187 -> viable/strict/1761755187 2025-12-04T09:23:32.9812788Z * [new tag] viable/strict/1761756826 -> viable/strict/1761756826 2025-12-04T09:23:32.9813104Z * [new tag] viable/strict/1761769551 -> viable/strict/1761769551 2025-12-04T09:23:32.9813555Z * [new tag] viable/strict/1761771032 -> viable/strict/1761771032 2025-12-04T09:23:32.9813873Z * [new tag] viable/strict/1761773101 -> viable/strict/1761773101 2025-12-04T09:23:32.9814181Z * [new tag] viable/strict/1761781792 -> viable/strict/1761781792 2025-12-04T09:23:32.9814551Z * [new tag] viable/strict/1761784788 -> viable/strict/1761784788 2025-12-04T09:23:32.9814869Z * [new tag] viable/strict/1761786740 -> viable/strict/1761786740 2025-12-04T09:23:32.9815177Z * [new tag] viable/strict/1761789332 -> viable/strict/1761789332 2025-12-04T09:23:32.9815499Z * [new tag] viable/strict/1761792569 -> viable/strict/1761792569 2025-12-04T09:23:32.9815808Z * [new tag] viable/strict/1761795289 -> viable/strict/1761795289 2025-12-04T09:23:32.9816116Z * [new tag] viable/strict/1761798345 -> viable/strict/1761798345 2025-12-04T09:23:32.9816424Z * [new tag] viable/strict/1761799827 -> viable/strict/1761799827 2025-12-04T09:23:32.9816730Z * [new tag] viable/strict/1761805604 -> viable/strict/1761805604 2025-12-04T09:23:32.9817043Z * [new tag] viable/strict/1761807202 -> viable/strict/1761807202 2025-12-04T09:23:32.9817356Z * [new tag] viable/strict/1761809094 -> viable/strict/1761809094 2025-12-04T09:23:32.9817657Z * [new tag] viable/strict/1761810576 -> viable/strict/1761810576 2025-12-04T09:23:32.9817972Z * [new tag] viable/strict/1761812771 -> viable/strict/1761812771 2025-12-04T09:23:32.9818285Z * [new tag] viable/strict/1761814363 -> viable/strict/1761814363 2025-12-04T09:23:32.9818597Z * [new tag] viable/strict/1761857410 -> viable/strict/1761857410 2025-12-04T09:23:32.9819097Z * [new tag] viable/strict/1761860985 -> viable/strict/1761860985 2025-12-04T09:23:32.9819513Z * [new tag] viable/strict/1761863094 -> viable/strict/1761863094 2025-12-04T09:23:32.9820123Z * [new tag] viable/strict/1761864590 -> viable/strict/1761864590 2025-12-04T09:23:32.9820885Z * [new tag] viable/strict/1761866675 -> viable/strict/1761866675 2025-12-04T09:23:32.9821721Z * [new tag] viable/strict/1761868178 -> viable/strict/1761868178 2025-12-04T09:23:32.9822191Z * [new tag] viable/strict/1761871111 -> viable/strict/1761871111 2025-12-04T09:23:32.9822837Z * [new tag] viable/strict/1761873126 -> viable/strict/1761873126 2025-12-04T09:23:32.9824742Z * [new tag] viable/strict/1761875714 -> viable/strict/1761875714 2025-12-04T09:23:32.9825271Z * [new tag] viable/strict/1761878924 -> viable/strict/1761878924 2025-12-04T09:23:32.9825724Z * [new tag] viable/strict/1761881727 -> viable/strict/1761881727 2025-12-04T09:23:32.9826186Z * [new tag] viable/strict/1761882959 -> viable/strict/1761882959 2025-12-04T09:23:32.9826640Z * [new tag] viable/strict/1761886268 -> viable/strict/1761886268 2025-12-04T09:23:32.9827095Z * [new tag] viable/strict/1761893641 -> viable/strict/1761893641 2025-12-04T09:23:32.9827827Z * [new tag] viable/strict/1761931517 -> viable/strict/1761931517 2025-12-04T09:23:32.9828217Z * [new tag] viable/strict/1761933080 -> viable/strict/1761933080 2025-12-04T09:23:32.9828583Z * [new tag] viable/strict/1761935217 -> viable/strict/1761935217 2025-12-04T09:23:32.9829403Z * [new tag] viable/strict/1761938533 -> viable/strict/1761938533 2025-12-04T09:23:32.9830034Z * [new tag] viable/strict/1761940184 -> viable/strict/1761940184 2025-12-04T09:23:32.9830631Z * [new tag] viable/strict/1761942338 -> viable/strict/1761942338 2025-12-04T09:23:32.9831301Z * [new tag] viable/strict/1761946100 -> viable/strict/1761946100 2025-12-04T09:23:32.9831951Z * [new tag] viable/strict/1761947374 -> viable/strict/1761947374 2025-12-04T09:23:32.9832548Z * [new tag] viable/strict/1761950978 -> viable/strict/1761950978 2025-12-04T09:23:32.9838282Z * [new tag] viable/strict/1761957727 -> viable/strict/1761957727 2025-12-04T09:23:32.9842940Z * [new tag] viable/strict/1761959532 -> viable/strict/1761959532 2025-12-04T09:23:32.9847652Z * [new tag] viable/strict/1761965366 -> viable/strict/1761965366 2025-12-04T09:23:32.9852457Z * [new tag] viable/strict/1761968066 -> viable/strict/1761968066 2025-12-04T09:23:32.9856155Z * [new tag] viable/strict/1761969322 -> viable/strict/1761969322 2025-12-04T09:23:32.9860982Z * [new tag] viable/strict/1761974723 -> viable/strict/1761974723 2025-12-04T09:23:32.9865912Z * [new tag] viable/strict/1761981837 -> viable/strict/1761981837 2025-12-04T09:23:32.9866307Z * [new tag] viable/strict/1761985546 -> viable/strict/1761985546 2025-12-04T09:23:32.9866621Z * [new tag] viable/strict/1761987030 -> viable/strict/1761987030 2025-12-04T09:23:32.9866945Z * [new tag] viable/strict/1762003554 -> viable/strict/1762003554 2025-12-04T09:23:32.9867252Z * [new tag] viable/strict/1762021560 -> viable/strict/1762021560 2025-12-04T09:23:32.9867541Z * [new tag] viable/strict/1762032190 -> viable/strict/1762032190 2025-12-04T09:23:32.9867835Z * [new tag] viable/strict/1762040981 -> viable/strict/1762040981 2025-12-04T09:23:32.9868132Z * [new tag] viable/strict/1762048525 -> viable/strict/1762048525 2025-12-04T09:23:32.9868428Z * [new tag] viable/strict/1762104223 -> viable/strict/1762104223 2025-12-04T09:23:32.9868732Z * [new tag] viable/strict/1762105778 -> viable/strict/1762105778 2025-12-04T09:23:32.9869059Z * [new tag] viable/strict/1762115109 -> viable/strict/1762115109 2025-12-04T09:23:32.9869392Z * [new tag] viable/strict/1762125840 -> viable/strict/1762125840 2025-12-04T09:23:32.9869690Z * [new tag] viable/strict/1762127377 -> viable/strict/1762127377 2025-12-04T09:23:32.9869986Z * [new tag] viable/strict/1762134925 -> viable/strict/1762134925 2025-12-04T09:23:32.9870278Z * [new tag] viable/strict/1762138338 -> viable/strict/1762138338 2025-12-04T09:23:32.9870576Z * [new tag] viable/strict/1762148993 -> viable/strict/1762148993 2025-12-04T09:23:32.9870887Z * [new tag] viable/strict/1762152871 -> viable/strict/1762152871 2025-12-04T09:23:32.9871202Z * [new tag] viable/strict/1762156183 -> viable/strict/1762156183 2025-12-04T09:23:32.9871517Z * [new tag] viable/strict/1762163457 -> viable/strict/1762163457 2025-12-04T09:23:32.9871834Z * [new tag] viable/strict/1762165569 -> viable/strict/1762165569 2025-12-04T09:23:32.9872162Z * [new tag] viable/strict/1762169035 -> viable/strict/1762169035 2025-12-04T09:23:32.9872482Z * [new tag] viable/strict/1762174936 -> viable/strict/1762174936 2025-12-04T09:23:32.9873003Z * [new tag] viable/strict/1762194412 -> viable/strict/1762194412 2025-12-04T09:23:32.9873322Z * [new tag] viable/strict/1762195876 -> viable/strict/1762195876 2025-12-04T09:23:32.9873660Z * [new tag] viable/strict/1762197788 -> viable/strict/1762197788 2025-12-04T09:23:32.9873995Z * [new tag] viable/strict/1762199389 -> viable/strict/1762199389 2025-12-04T09:23:32.9874329Z * [new tag] viable/strict/1762206585 -> viable/strict/1762206585 2025-12-04T09:23:32.9874805Z * [new tag] viable/strict/1762210184 -> viable/strict/1762210184 2025-12-04T09:23:32.9875121Z * [new tag] viable/strict/1762218736 -> viable/strict/1762218736 2025-12-04T09:23:32.9875453Z * [new tag] viable/strict/1762224529 -> viable/strict/1762224529 2025-12-04T09:23:32.9875818Z * [new tag] viable/strict/1762227253 -> viable/strict/1762227253 2025-12-04T09:23:32.9876141Z * [new tag] viable/strict/1762228515 -> viable/strict/1762228515 2025-12-04T09:23:32.9876459Z * [new tag] viable/strict/1762230349 -> viable/strict/1762230349 2025-12-04T09:23:32.9876781Z * [new tag] viable/strict/1762231859 -> viable/strict/1762231859 2025-12-04T09:23:32.9877087Z * [new tag] viable/strict/1762233925 -> viable/strict/1762233925 2025-12-04T09:23:32.9877417Z * [new tag] viable/strict/1762237630 -> viable/strict/1762237630 2025-12-04T09:23:32.9877733Z * [new tag] viable/strict/1762253522 -> viable/strict/1762253522 2025-12-04T09:23:32.9878053Z * [new tag] viable/strict/1762278588 -> viable/strict/1762278588 2025-12-04T09:23:32.9878368Z * [new tag] viable/strict/1762284203 -> viable/strict/1762284203 2025-12-04T09:23:32.9878685Z * [new tag] viable/strict/1762289446 -> viable/strict/1762289446 2025-12-04T09:23:32.9879005Z * [new tag] viable/strict/1762291515 -> viable/strict/1762291515 2025-12-04T09:23:32.9879330Z * [new tag] viable/strict/1762295100 -> viable/strict/1762295100 2025-12-04T09:23:32.9879645Z * [new tag] viable/strict/1762296590 -> viable/strict/1762296590 2025-12-04T09:23:32.9879977Z * [new tag] viable/strict/1762300179 -> viable/strict/1762300179 2025-12-04T09:23:32.9880300Z * [new tag] viable/strict/1762303207 -> viable/strict/1762303207 2025-12-04T09:23:32.9880610Z * [new tag] viable/strict/1762386584 -> viable/strict/1762386584 2025-12-04T09:23:32.9880893Z * [new tag] viable/strict/1762391537 -> viable/strict/1762391537 2025-12-04T09:23:32.9881183Z * [new tag] viable/strict/1762394119 -> viable/strict/1762394119 2025-12-04T09:23:32.9881476Z * [new tag] viable/strict/1762397437 -> viable/strict/1762397437 2025-12-04T09:23:32.9881758Z * [new tag] viable/strict/1762400256 -> viable/strict/1762400256 2025-12-04T09:23:32.9882044Z * [new tag] viable/strict/1762401469 -> viable/strict/1762401469 2025-12-04T09:23:32.9882334Z * [new tag] viable/strict/1762408195 -> viable/strict/1762408195 2025-12-04T09:23:32.9882621Z * [new tag] viable/strict/1762410411 -> viable/strict/1762410411 2025-12-04T09:23:32.9882901Z * [new tag] viable/strict/1762417613 -> viable/strict/1762417613 2025-12-04T09:23:32.9883193Z * [new tag] viable/strict/1762419198 -> viable/strict/1762419198 2025-12-04T09:23:32.9883477Z * [new tag] viable/strict/1762422656 -> viable/strict/1762422656 2025-12-04T09:23:32.9883761Z * [new tag] viable/strict/1762424746 -> viable/strict/1762424746 2025-12-04T09:23:32.9884045Z * [new tag] viable/strict/1762446386 -> viable/strict/1762446386 2025-12-04T09:23:32.9884331Z * [new tag] viable/strict/1762449912 -> viable/strict/1762449912 2025-12-04T09:23:32.9884616Z * [new tag] viable/strict/1762457031 -> viable/strict/1762457031 2025-12-04T09:23:32.9884892Z * [new tag] viable/strict/1762462441 -> viable/strict/1762462441 2025-12-04T09:23:32.9885179Z * [new tag] viable/strict/1762467909 -> viable/strict/1762467909 2025-12-04T09:23:32.9885465Z * [new tag] viable/strict/1762471493 -> viable/strict/1762471493 2025-12-04T09:23:32.9885804Z * [new tag] viable/strict/1762475990 -> viable/strict/1762475990 2025-12-04T09:23:32.9886084Z * [new tag] viable/strict/1762477933 -> viable/strict/1762477933 2025-12-04T09:23:32.9886370Z * [new tag] viable/strict/1762491053 -> viable/strict/1762491053 2025-12-04T09:23:32.9886689Z * [new tag] viable/strict/1762493118 -> viable/strict/1762493118 2025-12-04T09:23:32.9886990Z * [new tag] viable/strict/1762498442 -> viable/strict/1762498442 2025-12-04T09:23:32.9887272Z * [new tag] viable/strict/1762501778 -> viable/strict/1762501778 2025-12-04T09:23:32.9887561Z * [new tag] viable/strict/1762504001 -> viable/strict/1762504001 2025-12-04T09:23:32.9887850Z * [new tag] viable/strict/1762505583 -> viable/strict/1762505583 2025-12-04T09:23:32.9888137Z * [new tag] viable/strict/1762507523 -> viable/strict/1762507523 2025-12-04T09:23:32.9888419Z * [new tag] viable/strict/1762511140 -> viable/strict/1762511140 2025-12-04T09:23:32.9888707Z * [new tag] viable/strict/1762512632 -> viable/strict/1762512632 2025-12-04T09:23:32.9888994Z * [new tag] viable/strict/1762520467 -> viable/strict/1762520467 2025-12-04T09:23:32.9889276Z * [new tag] viable/strict/1762522016 -> viable/strict/1762522016 2025-12-04T09:23:32.9889564Z * [new tag] viable/strict/1762530591 -> viable/strict/1762530591 2025-12-04T09:23:32.9889852Z * [new tag] viable/strict/1762543405 -> viable/strict/1762543405 2025-12-04T09:23:32.9890139Z * [new tag] viable/strict/1762544998 -> viable/strict/1762544998 2025-12-04T09:23:32.9890420Z * [new tag] viable/strict/1762552182 -> viable/strict/1762552182 2025-12-04T09:23:32.9890706Z * [new tag] viable/strict/1762554297 -> viable/strict/1762554297 2025-12-04T09:23:32.9891004Z * [new tag] viable/strict/1762559381 -> viable/strict/1762559381 2025-12-04T09:23:32.9891307Z * [new tag] viable/strict/1762562222 -> viable/strict/1762562222 2025-12-04T09:23:32.9891596Z * [new tag] viable/strict/1762564319 -> viable/strict/1762564319 2025-12-04T09:23:32.9891902Z * [new tag] viable/strict/1762566904 -> viable/strict/1762566904 2025-12-04T09:23:32.9892195Z * [new tag] viable/strict/1762569781 -> viable/strict/1762569781 2025-12-04T09:23:32.9892474Z * [new tag] viable/strict/1762575940 -> viable/strict/1762575940 2025-12-04T09:23:32.9892761Z * [new tag] viable/strict/1762580974 -> viable/strict/1762580974 2025-12-04T09:23:32.9893054Z * [new tag] viable/strict/1762583185 -> viable/strict/1762583185 2025-12-04T09:23:32.9893346Z * [new tag] viable/strict/1762586647 -> viable/strict/1762586647 2025-12-04T09:23:32.9893853Z * [new tag] viable/strict/1762588183 -> viable/strict/1762588183 2025-12-04T09:23:32.9894165Z * [new tag] viable/strict/1762593886 -> viable/strict/1762593886 2025-12-04T09:23:32.9894488Z * [new tag] viable/strict/1762650743 -> viable/strict/1762650743 2025-12-04T09:23:32.9899045Z * [new tag] viable/strict/1762653328 -> viable/strict/1762653328 2025-12-04T09:23:32.9904469Z * [new tag] viable/strict/1762659342 -> viable/strict/1762659342 2025-12-04T09:23:32.9906067Z * [new tag] viable/strict/1762662360 -> viable/strict/1762662360 2025-12-04T09:23:32.9906393Z * [new tag] viable/strict/1762667377 -> viable/strict/1762667377 2025-12-04T09:23:32.9906696Z * [new tag] viable/strict/1762671090 -> viable/strict/1762671090 2025-12-04T09:23:32.9906995Z * [new tag] viable/strict/1762680284 -> viable/strict/1762680284 2025-12-04T09:23:32.9907563Z * [new tag] viable/strict/1762683900 -> viable/strict/1762683900 2025-12-04T09:23:32.9907896Z * [new tag] viable/strict/1762705541 -> viable/strict/1762705541 2025-12-04T09:23:32.9908226Z * [new tag] viable/strict/1762709004 -> viable/strict/1762709004 2025-12-04T09:23:32.9908630Z * [new tag] viable/strict/1762746004 -> viable/strict/1762746004 2025-12-04T09:23:32.9908968Z * [new tag] viable/strict/1762748799 -> viable/strict/1762748799 2025-12-04T09:23:32.9909301Z * [new tag] viable/strict/1762759504 -> viable/strict/1762759504 2025-12-04T09:23:32.9909622Z * [new tag] viable/strict/1762760973 -> viable/strict/1762760973 2025-12-04T09:23:32.9909950Z * [new tag] viable/strict/1762775374 -> viable/strict/1762775374 2025-12-04T09:23:32.9910277Z * [new tag] viable/strict/1762777661 -> viable/strict/1762777661 2025-12-04T09:23:32.9910614Z * [new tag] viable/strict/1762779774 -> viable/strict/1762779774 2025-12-04T09:23:32.9910946Z * [new tag] viable/strict/1762781259 -> viable/strict/1762781259 2025-12-04T09:23:32.9911272Z * [new tag] viable/strict/1762793628 -> viable/strict/1762793628 2025-12-04T09:23:32.9911629Z * [new tag] viable/strict/1762800711 -> viable/strict/1762800711 2025-12-04T09:23:32.9911955Z * [new tag] viable/strict/1762809894 -> viable/strict/1762809894 2025-12-04T09:23:32.9912280Z * [new tag] viable/strict/1762811384 -> viable/strict/1762811384 2025-12-04T09:23:32.9912614Z * [new tag] viable/strict/1762813841 -> viable/strict/1762813841 2025-12-04T09:23:32.9913221Z * [new tag] viable/strict/1762815047 -> viable/strict/1762815047 2025-12-04T09:23:32.9913583Z * [new tag] viable/strict/1762817094 -> viable/strict/1762817094 2025-12-04T09:23:32.9913967Z * [new tag] viable/strict/1762818582 -> viable/strict/1762818582 2025-12-04T09:23:32.9914310Z * [new tag] viable/strict/1762821623 -> viable/strict/1762821623 2025-12-04T09:23:32.9914645Z * [new tag] viable/strict/1762823531 -> viable/strict/1762823531 2025-12-04T09:23:32.9914981Z * [new tag] viable/strict/1762849583 -> viable/strict/1762849583 2025-12-04T09:23:32.9915310Z * [new tag] viable/strict/1762851200 -> viable/strict/1762851200 2025-12-04T09:23:32.9915619Z * [new tag] viable/strict/1762854603 -> viable/strict/1762854603 2025-12-04T09:23:32.9915940Z * [new tag] viable/strict/1762858276 -> viable/strict/1762858276 2025-12-04T09:23:32.9916263Z * [new tag] viable/strict/1762860891 -> viable/strict/1762860891 2025-12-04T09:23:32.9916605Z * [new tag] viable/strict/1762866174 -> viable/strict/1762866174 2025-12-04T09:23:32.9916940Z * [new tag] viable/strict/1762867653 -> viable/strict/1762867653 2025-12-04T09:23:32.9917288Z * [new tag] viable/strict/1762872669 -> viable/strict/1762872669 2025-12-04T09:23:32.9917611Z * [new tag] viable/strict/1762878380 -> viable/strict/1762878380 2025-12-04T09:23:32.9917953Z * [new tag] viable/strict/1762889003 -> viable/strict/1762889003 2025-12-04T09:23:32.9918421Z * [new tag] viable/strict/1762890589 -> viable/strict/1762890589 2025-12-04T09:23:32.9919080Z * [new tag] viable/strict/1762892743 -> viable/strict/1762892743 2025-12-04T09:23:32.9924677Z * [new tag] viable/strict/1762894271 -> viable/strict/1762894271 2025-12-04T09:23:32.9925197Z * [new tag] viable/strict/1762896287 -> viable/strict/1762896287 2025-12-04T09:23:32.9925601Z * [new tag] viable/strict/1762915871 -> viable/strict/1762915871 2025-12-04T09:23:32.9926075Z * [new tag] viable/strict/1762918569 -> viable/strict/1762918569 2025-12-04T09:23:32.9926402Z * [new tag] viable/strict/1762919776 -> viable/strict/1762919776 2025-12-04T09:23:32.9926708Z * [new tag] viable/strict/1762923072 -> viable/strict/1762923072 2025-12-04T09:23:32.9927055Z * [new tag] viable/strict/1762928826 -> viable/strict/1762928826 2025-12-04T09:23:32.9927365Z * [new tag] viable/strict/1762930451 -> viable/strict/1762930451 2025-12-04T09:23:32.9927655Z * [new tag] viable/strict/1762933780 -> viable/strict/1762933780 2025-12-04T09:23:32.9927953Z * [new tag] viable/strict/1762937638 -> viable/strict/1762937638 2025-12-04T09:23:32.9928309Z * [new tag] viable/strict/1762939545 -> viable/strict/1762939545 2025-12-04T09:23:32.9928600Z * [new tag] viable/strict/1762962692 -> viable/strict/1762962692 2025-12-04T09:23:32.9933105Z * [new tag] viable/strict/1762979143 -> viable/strict/1762979143 2025-12-04T09:23:32.9933475Z * [new tag] viable/strict/1762984188 -> viable/strict/1762984188 2025-12-04T09:23:32.9933790Z * [new tag] viable/strict/1762986306 -> viable/strict/1762986306 2025-12-04T09:23:32.9934121Z * [new tag] viable/strict/1762989903 -> viable/strict/1762989903 2025-12-04T09:23:32.9934419Z * [new tag] viable/strict/1762991377 -> viable/strict/1762991377 2025-12-04T09:23:32.9934716Z * [new tag] viable/strict/1762998921 -> viable/strict/1762998921 2025-12-04T09:23:32.9935011Z * [new tag] viable/strict/1763002287 -> viable/strict/1763002287 2025-12-04T09:23:32.9935298Z * [new tag] viable/strict/1763016840 -> viable/strict/1763016840 2025-12-04T09:23:32.9935593Z * [new tag] viable/strict/1763020180 -> viable/strict/1763020180 2025-12-04T09:23:32.9935897Z * [new tag] viable/strict/1763027421 -> viable/strict/1763027421 2025-12-04T09:23:32.9936192Z * [new tag] viable/strict/1763031120 -> viable/strict/1763031120 2025-12-04T09:23:32.9936481Z * [new tag] viable/strict/1763036861 -> viable/strict/1763036861 2025-12-04T09:23:32.9936785Z * [new tag] viable/strict/1763038993 -> viable/strict/1763038993 2025-12-04T09:23:32.9937080Z * [new tag] viable/strict/1763054703 -> viable/strict/1763054703 2025-12-04T09:23:32.9937382Z * [new tag] viable/strict/1763067061 -> viable/strict/1763067061 2025-12-04T09:23:32.9937669Z * [new tag] viable/strict/1763070847 -> viable/strict/1763070847 2025-12-04T09:23:32.9937963Z * [new tag] viable/strict/1763072706 -> viable/strict/1763072706 2025-12-04T09:23:32.9938455Z * [new tag] viable/strict/1763076302 -> viable/strict/1763076302 2025-12-04T09:23:32.9938796Z * [new tag] viable/strict/1763080816 -> viable/strict/1763080816 2025-12-04T09:23:32.9939222Z * [new tag] viable/strict/1763082732 -> viable/strict/1763082732 2025-12-04T09:23:32.9940040Z * [new tag] viable/strict/1763085329 -> viable/strict/1763085329 2025-12-04T09:23:32.9940612Z * [new tag] viable/strict/1763088623 -> viable/strict/1763088623 2025-12-04T09:23:32.9942840Z * [new tag] viable/strict/1763091402 -> viable/strict/1763091402 2025-12-04T09:23:32.9943357Z * [new tag] viable/strict/1763092602 -> viable/strict/1763092602 2025-12-04T09:23:32.9943753Z * [new tag] viable/strict/1763094355 -> viable/strict/1763094355 2025-12-04T09:23:32.9944091Z * [new tag] viable/strict/1763099390 -> viable/strict/1763099390 2025-12-04T09:23:32.9944522Z * [new tag] viable/strict/1763101608 -> viable/strict/1763101608 2025-12-04T09:23:32.9945099Z * [new tag] viable/strict/1763105102 -> viable/strict/1763105102 2025-12-04T09:23:32.9945405Z * [new tag] viable/strict/1763112347 -> viable/strict/1763112347 2025-12-04T09:23:32.9946093Z * [new tag] viable/strict/1763119471 -> viable/strict/1763119471 2025-12-04T09:23:32.9946678Z * [new tag] viable/strict/1763126835 -> viable/strict/1763126835 2025-12-04T09:23:32.9947180Z * [new tag] viable/strict/1763149779 -> viable/strict/1763149779 2025-12-04T09:23:32.9947839Z * [new tag] viable/strict/1763164178 -> viable/strict/1763164178 2025-12-04T09:23:32.9948451Z * [new tag] viable/strict/1763167104 -> viable/strict/1763167104 2025-12-04T09:23:32.9949051Z * [new tag] viable/strict/1763169132 -> viable/strict/1763169132 2025-12-04T09:23:32.9949707Z * [new tag] viable/strict/1763171708 -> viable/strict/1763171708 2025-12-04T09:23:32.9950265Z * [new tag] viable/strict/1763174759 -> viable/strict/1763174759 2025-12-04T09:23:32.9952534Z * [new tag] viable/strict/1763180744 -> viable/strict/1763180744 2025-12-04T09:23:32.9953141Z * [new tag] viable/strict/1763182227 -> viable/strict/1763182227 2025-12-04T09:23:32.9953507Z * [new tag] viable/strict/1763184309 -> viable/strict/1763184309 2025-12-04T09:23:32.9953854Z * [new tag] viable/strict/1763187991 -> viable/strict/1763187991 2025-12-04T09:23:32.9954233Z * [new tag] viable/strict/1763191445 -> viable/strict/1763191445 2025-12-04T09:23:32.9954918Z * [new tag] viable/strict/1763195152 -> viable/strict/1763195152 2025-12-04T09:23:32.9955472Z * [new tag] viable/strict/1763205769 -> viable/strict/1763205769 2025-12-04T09:23:32.9956514Z * [new tag] viable/strict/1763246990 -> viable/strict/1763246990 2025-12-04T09:23:32.9957242Z * [new tag] viable/strict/1763261578 -> viable/strict/1763261578 2025-12-04T09:23:32.9957830Z * [new tag] viable/strict/1763286573 -> viable/strict/1763286573 2025-12-04T09:23:32.9958308Z * [new tag] viable/strict/1763292167 -> viable/strict/1763292167 2025-12-04T09:23:32.9958668Z * [new tag] viable/strict/1763333386 -> viable/strict/1763333386 2025-12-04T09:23:32.9959253Z * [new tag] viable/strict/1763340082 -> viable/strict/1763340082 2025-12-04T09:23:32.9963490Z * [new tag] viable/strict/1763364324 -> viable/strict/1763364324 2025-12-04T09:23:32.9964016Z * [new tag] viable/strict/1763371569 -> viable/strict/1763371569 2025-12-04T09:23:32.9964453Z * [new tag] viable/strict/1763373067 -> viable/strict/1763373067 2025-12-04T09:23:32.9964804Z * [new tag] viable/strict/1763375157 -> viable/strict/1763375157 2025-12-04T09:23:32.9965249Z * [new tag] viable/strict/1763382462 -> viable/strict/1763382462 2025-12-04T09:23:32.9965959Z * [new tag] viable/strict/1763394661 -> viable/strict/1763394661 2025-12-04T09:23:32.9966330Z * [new tag] viable/strict/1763396797 -> viable/strict/1763396797 2025-12-04T09:23:32.9966668Z * [new tag] viable/strict/1763398542 -> viable/strict/1763398542 2025-12-04T09:23:32.9966975Z * [new tag] viable/strict/1763401807 -> viable/strict/1763401807 2025-12-04T09:23:32.9967272Z * [new tag] viable/strict/1763414698 -> viable/strict/1763414698 2025-12-04T09:23:32.9967570Z * [new tag] viable/strict/1763419807 -> viable/strict/1763419807 2025-12-04T09:23:32.9967883Z * [new tag] viable/strict/1763426369 -> viable/strict/1763426369 2025-12-04T09:23:32.9968436Z * [new tag] viable/strict/1763428331 -> viable/strict/1763428331 2025-12-04T09:23:32.9969000Z * [new tag] viable/strict/1763430922 -> viable/strict/1763430922 2025-12-04T09:23:32.9969539Z * [new tag] viable/strict/1763434184 -> viable/strict/1763434184 2025-12-04T09:23:32.9970204Z * [new tag] viable/strict/1763439973 -> viable/strict/1763439973 2025-12-04T09:23:32.9970987Z * [new tag] viable/strict/1763444995 -> viable/strict/1763444995 2025-12-04T09:23:32.9971475Z * [new tag] viable/strict/1763447206 -> viable/strict/1763447206 2025-12-04T09:23:32.9977219Z * [new tag] viable/strict/1763448826 -> viable/strict/1763448826 2025-12-04T09:23:32.9977742Z * [new tag] viable/strict/1763450717 -> viable/strict/1763450717 2025-12-04T09:23:32.9978146Z * [new tag] viable/strict/1763452183 -> viable/strict/1763452183 2025-12-04T09:23:32.9978476Z * [new tag] viable/strict/1763457945 -> viable/strict/1763457945 2025-12-04T09:23:32.9978913Z * [new tag] viable/strict/1763459439 -> viable/strict/1763459439 2025-12-04T09:23:32.9979711Z * [new tag] viable/strict/1763461556 -> viable/strict/1763461556 2025-12-04T09:23:32.9980092Z * [new tag] viable/strict/1763463103 -> viable/strict/1763463103 2025-12-04T09:23:32.9980412Z * [new tag] viable/strict/1763465100 -> viable/strict/1763465100 2025-12-04T09:23:32.9980714Z * [new tag] viable/strict/1763468866 -> viable/strict/1763468866 2025-12-04T09:23:32.9980996Z * [new tag] viable/strict/1763493823 -> viable/strict/1763493823 2025-12-04T09:23:32.9981291Z * [new tag] viable/strict/1763496249 -> viable/strict/1763496249 2025-12-04T09:23:32.9981583Z * [new tag] viable/strict/1763502620 -> viable/strict/1763502620 2025-12-04T09:23:32.9981875Z * [new tag] viable/strict/1763504715 -> viable/strict/1763504715 2025-12-04T09:23:32.9982164Z * [new tag] viable/strict/1763506208 -> viable/strict/1763506208 2025-12-04T09:23:32.9982450Z * [new tag] viable/strict/1763520590 -> viable/strict/1763520590 2025-12-04T09:23:32.9982735Z * [new tag] viable/strict/1763523357 -> viable/strict/1763523357 2025-12-04T09:23:32.9983183Z * [new tag] viable/strict/1763529922 -> viable/strict/1763529922 2025-12-04T09:23:32.9988164Z * [new tag] viable/strict/1763531408 -> viable/strict/1763531408 2025-12-04T09:23:32.9988675Z * [new tag] viable/strict/1763533622 -> viable/strict/1763533622 2025-12-04T09:23:32.9989112Z * [new tag] viable/strict/1763538576 -> viable/strict/1763538576 2025-12-04T09:23:32.9989545Z * [new tag] viable/strict/1763545823 -> viable/strict/1763545823 2025-12-04T09:23:32.9989855Z * [new tag] viable/strict/1763547951 -> viable/strict/1763547951 2025-12-04T09:23:32.9990180Z * [new tag] viable/strict/1763551477 -> viable/strict/1763551477 2025-12-04T09:23:32.9990481Z * [new tag] viable/strict/1763552982 -> viable/strict/1763552982 2025-12-04T09:23:32.9990779Z * [new tag] viable/strict/1763594698 -> viable/strict/1763594698 2025-12-04T09:23:32.9991075Z * [new tag] viable/strict/1763596178 -> viable/strict/1763596178 2025-12-04T09:23:32.9991388Z * [new tag] viable/strict/1763599155 -> viable/strict/1763599155 2025-12-04T09:23:32.9991686Z * [new tag] viable/strict/1763603717 -> viable/strict/1763603717 2025-12-04T09:23:32.9991976Z * [new tag] viable/strict/1763606923 -> viable/strict/1763606923 2025-12-04T09:23:32.9992282Z * [new tag] viable/strict/1763609715 -> viable/strict/1763609715 2025-12-04T09:23:32.9992593Z * [new tag] viable/strict/1763612757 -> viable/strict/1763612757 2025-12-04T09:23:32.9993287Z * [new tag] viable/strict/1763616325 -> viable/strict/1763616325 2025-12-04T09:23:32.9993597Z * [new tag] viable/strict/1763623509 -> viable/strict/1763623509 2025-12-04T09:23:32.9993923Z * [new tag] viable/strict/1763624984 -> viable/strict/1763624984 2025-12-04T09:23:32.9994278Z * [new tag] viable/strict/1763628796 -> viable/strict/1763628796 2025-12-04T09:23:32.9994601Z * [new tag] viable/strict/1763634343 -> viable/strict/1763634343 2025-12-04T09:23:32.9999947Z * [new tag] viable/strict/1763635867 -> viable/strict/1763635867 2025-12-04T09:23:33.0000469Z * [new tag] viable/strict/1763639382 -> viable/strict/1763639382 2025-12-04T09:23:33.0000921Z * [new tag] viable/strict/1763646626 -> viable/strict/1763646626 2025-12-04T09:23:33.0001357Z * [new tag] viable/strict/1763655997 -> viable/strict/1763655997 2025-12-04T09:23:33.0001689Z * [new tag] viable/strict/1763659444 -> viable/strict/1763659444 2025-12-04T09:23:33.0002014Z * [new tag] viable/strict/1763660992 -> viable/strict/1763660992 2025-12-04T09:23:33.0002352Z * [new tag] viable/strict/1763663201 -> viable/strict/1763663201 2025-12-04T09:23:33.0002691Z * [new tag] viable/strict/1763670362 -> viable/strict/1763670362 2025-12-04T09:23:33.0003018Z * [new tag] viable/strict/1763675378 -> viable/strict/1763675378 2025-12-04T09:23:33.0003321Z * [new tag] viable/strict/1763693343 -> viable/strict/1763693343 2025-12-04T09:23:33.0003620Z * [new tag] viable/strict/1763696088 -> viable/strict/1763696088 2025-12-04T09:23:33.0003915Z * [new tag] viable/strict/1763697343 -> viable/strict/1763697343 2025-12-04T09:23:33.0004219Z * [new tag] viable/strict/1763699165 -> viable/strict/1763699165 2025-12-04T09:23:33.0008445Z * [new tag] viable/strict/1763700660 -> viable/strict/1763700660 2025-12-04T09:23:33.0008977Z * [new tag] viable/strict/1763704209 -> viable/strict/1763704209 2025-12-04T09:23:33.0009419Z * [new tag] viable/strict/1763706411 -> viable/strict/1763706411 2025-12-04T09:23:33.0009867Z * [new tag] viable/strict/1763708082 -> viable/strict/1763708082 2025-12-04T09:23:33.0010182Z * [new tag] viable/strict/1763711381 -> viable/strict/1763711381 2025-12-04T09:23:33.0010506Z * [new tag] viable/strict/1763713593 -> viable/strict/1763713593 2025-12-04T09:23:33.0010813Z * [new tag] viable/strict/1763715201 -> viable/strict/1763715201 2025-12-04T09:23:33.0011145Z * [new tag] viable/strict/1763733017 -> viable/strict/1763733017 2025-12-04T09:23:33.0011474Z * [new tag] viable/strict/1763735108 -> viable/strict/1763735108 2025-12-04T09:23:33.0011802Z * [new tag] viable/strict/1763749579 -> viable/strict/1763749579 2025-12-04T09:23:33.0012122Z * [new tag] viable/strict/1763751113 -> viable/strict/1763751113 2025-12-04T09:23:33.0012421Z * [new tag] viable/strict/1763753035 -> viable/strict/1763753035 2025-12-04T09:23:33.0012720Z * [new tag] viable/strict/1763754578 -> viable/strict/1763754578 2025-12-04T09:23:33.0013011Z * [new tag] viable/strict/1763756748 -> viable/strict/1763756748 2025-12-04T09:23:33.0013333Z * [new tag] viable/strict/1763758205 -> viable/strict/1763758205 2025-12-04T09:23:33.0013632Z * [new tag] viable/strict/1763764050 -> viable/strict/1763764050 2025-12-04T09:23:33.0013932Z * [new tag] viable/strict/1763771887 -> viable/strict/1763771887 2025-12-04T09:23:33.0014230Z * [new tag] viable/strict/1763773920 -> viable/strict/1763773920 2025-12-04T09:23:33.0015369Z * [new tag] viable/strict/1763776501 -> viable/strict/1763776501 2025-12-04T09:23:33.0015812Z * [new tag] viable/strict/1763779437 -> viable/strict/1763779437 2025-12-04T09:23:33.0016250Z * [new tag] viable/strict/1763781038 -> viable/strict/1763781038 2025-12-04T09:23:33.0016752Z * [new tag] viable/strict/1763782245 -> viable/strict/1763782245 2025-12-04T09:23:33.0017058Z * [new tag] viable/strict/1763785568 -> viable/strict/1763785568 2025-12-04T09:23:33.0017374Z * [new tag] viable/strict/1763787006 -> viable/strict/1763787006 2025-12-04T09:23:33.0017676Z * [new tag] viable/strict/1763789103 -> viable/strict/1763789103 2025-12-04T09:23:33.0017978Z * [new tag] viable/strict/1763790578 -> viable/strict/1763790578 2025-12-04T09:23:33.0022628Z * [new tag] viable/strict/1763796275 -> viable/strict/1763796275 2025-12-04T09:23:33.0023184Z * [new tag] viable/strict/1763801465 -> viable/strict/1763801465 2025-12-04T09:23:33.0023659Z * [new tag] viable/strict/1763803522 -> viable/strict/1763803522 2025-12-04T09:23:33.0024493Z * [new tag] viable/strict/1763808581 -> viable/strict/1763808581 2025-12-04T09:23:33.0024924Z * [new tag] viable/strict/1763840977 -> viable/strict/1763840977 2025-12-04T09:23:33.0025287Z * [new tag] viable/strict/1763846659 -> viable/strict/1763846659 2025-12-04T09:23:33.0025611Z * [new tag] viable/strict/1763872065 -> viable/strict/1763872065 2025-12-04T09:23:33.0025952Z * [new tag] viable/strict/1763873648 -> viable/strict/1763873648 2025-12-04T09:23:33.0026298Z * [new tag] viable/strict/1763875506 -> viable/strict/1763875506 2025-12-04T09:23:33.0026653Z * [new tag] viable/strict/1763889904 -> viable/strict/1763889904 2025-12-04T09:23:33.0026794Z * [new tag] viable/strict/1763930999 -> viable/strict/1763930999 2025-12-04T09:23:33.0026913Z * [new tag] viable/strict/1763944964 -> viable/strict/1763944964 2025-12-04T09:23:33.0027045Z * [new tag] viable/strict/1763958474 -> viable/strict/1763958474 2025-12-04T09:23:33.0027167Z * [new tag] viable/strict/1763967263 -> viable/strict/1763967263 2025-12-04T09:23:33.0027521Z * [new tag] viable/strict/1763972803 -> viable/strict/1763972803 2025-12-04T09:23:33.0027649Z * [new tag] viable/strict/1763976376 -> viable/strict/1763976376 2025-12-04T09:23:33.0027771Z * [new tag] viable/strict/1763989404 -> viable/strict/1763989404 2025-12-04T09:23:33.0027901Z * [new tag] viable/strict/1763990887 -> viable/strict/1763990887 2025-12-04T09:23:33.0028028Z * [new tag] viable/strict/1764019919 -> viable/strict/1764019919 2025-12-04T09:23:33.0028151Z * [new tag] viable/strict/1764023134 -> viable/strict/1764023134 2025-12-04T09:23:33.0028436Z * [new tag] viable/strict/1764024593 -> viable/strict/1764024593 2025-12-04T09:23:33.0028645Z * [new tag] viable/strict/1764026706 -> viable/strict/1764026706 2025-12-04T09:23:33.0033666Z * [new tag] viable/strict/1764031139 -> viable/strict/1764031139 2025-12-04T09:23:33.0033992Z * [new tag] viable/strict/1764033131 -> viable/strict/1764033131 2025-12-04T09:23:33.0034583Z * [new tag] viable/strict/1764035725 -> viable/strict/1764035725 2025-12-04T09:23:33.0034762Z * [new tag] viable/strict/1764624265 -> viable/strict/1764624265 2025-12-04T09:23:33.0034900Z * [new tag] viable/strict/1764631514 -> viable/strict/1764631514 2025-12-04T09:23:33.0035186Z * [new tag] viable/strict/1764632987 -> viable/strict/1764632987 2025-12-04T09:23:33.0035318Z * [new tag] viable/strict/1764636063 -> viable/strict/1764636063 2025-12-04T09:23:33.0035468Z * [new tag] viable/strict/1764643975 -> viable/strict/1764643975 2025-12-04T09:23:33.0035594Z * [new tag] viable/strict/1764646859 -> viable/strict/1764646859 2025-12-04T09:23:33.0035779Z * [new tag] viable/strict/1764653120 -> viable/strict/1764653120 2025-12-04T09:23:33.0035917Z * [new tag] viable/strict/1764654632 -> viable/strict/1764654632 2025-12-04T09:23:33.0036043Z * [new tag] viable/strict/1764656821 -> viable/strict/1764656821 2025-12-04T09:23:33.0036178Z * [new tag] viable/strict/1764658557 -> viable/strict/1764658557 2025-12-04T09:23:33.0036339Z * [new tag] viable/strict/1764660333 -> viable/strict/1764660333 2025-12-04T09:23:33.0036481Z * [new tag] viable/strict/1764661812 -> viable/strict/1764661812 2025-12-04T09:23:33.0036649Z * [new tag] viable/strict/1764664023 -> viable/strict/1764664023 2025-12-04T09:23:33.0037237Z * [new tag] viable/strict/1764669150 -> viable/strict/1764669150 2025-12-04T09:23:33.0037697Z * [new tag] viable/strict/1764680709 -> viable/strict/1764680709 2025-12-04T09:23:33.0040321Z * [new tag] viable/strict/1764687619 -> viable/strict/1764687619 2025-12-04T09:23:33.0040629Z * [new tag] viable/strict/1764696355 -> viable/strict/1764696355 2025-12-04T09:23:33.0040765Z * [new tag] viable/strict/1764701767 -> viable/strict/1764701767 2025-12-04T09:23:33.0040905Z * [new tag] viable/strict/1764710768 -> viable/strict/1764710768 2025-12-04T09:23:33.0041105Z * [new tag] viable/strict/1764716202 -> viable/strict/1764716202 2025-12-04T09:23:33.0041246Z * [new tag] viable/strict/1764793566 -> viable/strict/1764793566 2025-12-04T09:23:33.0041570Z * [new tag] viable/strict/1764797093 -> viable/strict/1764797093 2025-12-04T09:23:33.0042705Z * [new tag] viable/strict/1764800729 -> viable/strict/1764800729 2025-12-04T09:23:33.0042913Z * [new tag] whc_flight_1 -> whc_flight_1 2025-12-04T09:23:33.0045785Z * [new tag] whc_flight_2 -> whc_flight_2 2025-12-04T09:23:33.0047619Z * [new tag] whc_flight_4 -> whc_flight_4 2025-12-04T09:23:33.0552343Z [command]/usr/bin/git rev-parse --verify --quiet ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32^{object} 2025-12-04T09:23:33.0588311Z ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:23:33.0589382Z ##[endgroup] 2025-12-04T09:23:33.0589580Z ##[group]Determining the checkout info 2025-12-04T09:23:33.0589983Z ##[endgroup] 2025-12-04T09:23:33.0599555Z [command]/usr/bin/git sparse-checkout disable 2025-12-04T09:23:33.0635368Z [command]/usr/bin/git config --local --unset-all extensions.worktreeConfig 2025-12-04T09:23:33.0672468Z ##[group]Checking out the ref 2025-12-04T09:23:33.0672973Z [command]/usr/bin/git checkout --progress --force ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:23:34.0981369Z Updating files: 97% (19541/20121) 2025-12-04T09:23:34.1210184Z Updating files: 98% (19719/20121) 2025-12-04T09:23:34.1345409Z Updating files: 99% (19920/20121) 2025-12-04T09:23:34.1347418Z Updating files: 100% (20121/20121) 2025-12-04T09:23:34.1347694Z Updating files: 100% (20121/20121), done. 2025-12-04T09:23:34.1569055Z Note: switching to 'ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32'. 2025-12-04T09:23:34.1569323Z 2025-12-04T09:23:34.1569492Z You are in 'detached HEAD' state. You can look around, make experimental 2025-12-04T09:23:34.1569845Z changes and commit them, and you can discard any commits you make in this 2025-12-04T09:23:34.1570278Z state without impacting any branches by switching back to a branch. 2025-12-04T09:23:34.1570981Z 2025-12-04T09:23:34.1571125Z If you want to create a new branch to retain commits you create, you may 2025-12-04T09:23:34.1571481Z do so (now or later) by using -c with the switch command. Example: 2025-12-04T09:23:34.1571653Z 2025-12-04T09:23:34.1571739Z git switch -c 2025-12-04T09:23:34.1571874Z 2025-12-04T09:23:34.1572047Z Or undo this operation with: 2025-12-04T09:23:34.1572166Z 2025-12-04T09:23:34.1572239Z git switch - 2025-12-04T09:23:34.1572330Z 2025-12-04T09:23:34.1572491Z Turn off this advice by setting config variable advice.detachedHead to false 2025-12-04T09:23:34.1572693Z 2025-12-04T09:23:34.1578554Z HEAD is now at ffd9b0fb435 Resolve collective autotuning test failure on arm (#168919) 2025-12-04T09:23:34.1641568Z ##[endgroup] 2025-12-04T09:23:34.1642086Z ##[group]Setting up auth for fetching submodules 2025-12-04T09:23:34.1642650Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2025-12-04T09:23:34.1707380Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2025-12-04T09:23:34.1741208Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2025-12-04T09:23:34.1770635Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2025-12-04T09:23:34.1797847Z ##[endgroup] 2025-12-04T09:23:34.1798229Z ##[group]Fetching submodules 2025-12-04T09:23:34.1799503Z [command]/usr/bin/git submodule sync --recursive 2025-12-04T09:23:34.2111984Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2025-12-04T09:23:34.2436022Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2025-12-04T09:23:34.2438111Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2025-12-04T09:23:34.2446335Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2025-12-04T09:23:34.2448632Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2025-12-04T09:23:34.2449192Z Submodule 'third_party/NVTX' (https://github.com/NVIDIA/NVTX.git) registered for path 'third_party/NVTX' 2025-12-04T09:23:34.2450031Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2025-12-04T09:23:34.2455088Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2025-12-04T09:23:34.2457015Z Submodule 'third_party/aiter' (https://github.com/ROCm/aiter.git) registered for path 'third_party/aiter' 2025-12-04T09:23:34.2457749Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2025-12-04T09:23:34.2458520Z Submodule 'third_party/composable_kernel' (https://github.com/ROCm/composable_kernel.git) registered for path 'third_party/composable_kernel' 2025-12-04T09:23:34.2459280Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2025-12-04T09:23:34.2459871Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2025-12-04T09:23:34.2460460Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2025-12-04T09:23:34.2461041Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2025-12-04T09:23:34.2461546Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2025-12-04T09:23:34.2462131Z Submodule 'third_party/flash-attention' (https://github.com/Dao-AILab/flash-attention.git) registered for path 'third_party/flash-attention' 2025-12-04T09:23:34.2463133Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2025-12-04T09:23:34.2463774Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2025-12-04T09:23:34.2464569Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2025-12-04T09:23:34.2466451Z Submodule 'third_party/gloo' (https://github.com/pytorch/gloo) registered for path 'third_party/gloo' 2025-12-04T09:23:34.2472565Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2025-12-04T09:23:34.2473342Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2025-12-04T09:23:34.2473863Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2025-12-04T09:23:34.2480444Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2025-12-04T09:23:34.2481180Z Submodule 'third_party/kleidiai' (https://github.com/ARM-software/kleidiai.git) registered for path 'third_party/kleidiai' 2025-12-04T09:23:34.2482351Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2025-12-04T09:23:34.2482963Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2025-12-04T09:23:34.2486164Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2025-12-04T09:23:34.2486790Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2025-12-04T09:23:34.2491857Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2025-12-04T09:23:34.2492560Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2025-12-04T09:23:34.2494555Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2025-12-04T09:23:34.2499158Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2025-12-04T09:23:34.2500776Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2025-12-04T09:23:34.2506087Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2025-12-04T09:23:34.2506765Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2025-12-04T09:23:34.2509970Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2025-12-04T09:23:34.2546697Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2025-12-04T09:23:34.4859913Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2025-12-04T09:23:34.4861401Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2025-12-04T09:23:34.4861941Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2025-12-04T09:23:34.4862422Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2025-12-04T09:23:34.4885071Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2025-12-04T09:23:34.7701795Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2025-12-04T09:23:34.7702594Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2025-12-04T09:23:34.7703609Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2025-12-04T09:23:34.7704458Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2025-12-04T09:23:34.7705348Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2025-12-04T09:23:34.7706462Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2025-12-04T09:23:34.7707291Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2025-12-04T09:23:34.7990719Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NVTX'... 2025-12-04T09:23:34.8587249Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2025-12-04T09:23:36.1051461Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kleidiai'... 2025-12-04T09:23:36.1052425Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flash-attention'... 2025-12-04T09:23:36.1053259Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2025-12-04T09:23:36.1054066Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2025-12-04T09:23:36.1054915Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2025-12-04T09:23:36.1055703Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2025-12-04T09:23:36.1056493Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2025-12-04T09:23:36.1057290Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2025-12-04T09:23:36.1058076Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2025-12-04T09:23:36.1058894Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2025-12-04T09:23:36.1059700Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2025-12-04T09:23:36.1060449Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2025-12-04T09:23:36.1677272Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2025-12-04T09:23:50.5733094Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2025-12-04T09:23:50.5737758Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2025-12-04T09:23:50.5739829Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2025-12-04T09:23:50.5740319Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2025-12-04T09:23:50.5740800Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/composable_kernel'... 2025-12-04T09:23:50.5741382Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2025-12-04T09:23:50.5741857Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2025-12-04T09:23:50.5742316Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2025-12-04T09:23:50.5742759Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/aiter'... 2025-12-04T09:23:50.5864876Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2025-12-04T09:23:50.5974181Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2025-12-04T09:23:50.6067493Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2025-12-04T09:23:50.6273361Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2025-12-04T09:23:50.7003698Z Submodule path 'third_party/NVTX': checked out '3ebbc93ded7285963bff932c678fa367eb393ba6' 2025-12-04T09:23:50.7431275Z Submodule path 'third_party/VulkanMemoryAllocator': checked out '1d8f600fd424278486eade7ed3e877c99f0846b1' 2025-12-04T09:23:51.2746599Z Submodule path 'third_party/XNNPACK': checked out '51a0103656eff6fc9bfd39a4597923c4b542c883' 2025-12-04T09:23:51.4330250Z Submodule path 'third_party/aiter': checked out '01aae101b9e5e94d6c16a9514c9fb8df99c93150' 2025-12-04T09:23:51.4348363Z Submodule '3rdparty/composable_kernel' (https://github.com/ROCm/composable_kernel.git) registered for path 'third_party/aiter/3rdparty/composable_kernel' 2025-12-04T09:23:51.4373607Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/aiter/3rdparty/composable_kernel'... 2025-12-04T09:23:56.4110742Z Submodule path 'third_party/aiter/3rdparty/composable_kernel': checked out 'cffe8fa2a442ac8e80dd236a1a5d24fe3d7e0cbf' 2025-12-04T09:23:56.4315366Z Submodule path 'third_party/benchmark': checked out '299e5928955cc62af9968370293b916f5130916f' 2025-12-04T09:23:56.7027383Z Submodule path 'third_party/composable_kernel': checked out '7fe50dc3da2069d6645d9deb8c017a876472a977' 2025-12-04T09:23:56.7429396Z Submodule path 'third_party/cpp-httplib': checked out '89c932f313c6437c38f2982869beacc89c2f2246' 2025-12-04T09:23:56.8274801Z Submodule path 'third_party/cpuinfo': checked out 'f858c30bcb16f8effd5ff46996f0514539e17abc' 2025-12-04T09:23:56.8703549Z Submodule path 'third_party/cudnn_frontend': checked out '0b1577c8c83401237d601d0d0db5210506705396' 2025-12-04T09:23:57.3940767Z Submodule path 'third_party/cutlass': checked out 'f88806b1e31dfa579842638740216dd41fc6c588' 2025-12-04T09:23:57.5166300Z Submodule path 'third_party/fbgemm': checked out 'c0b988d39a9e47c794d699f29930ed4d7c7e13a4' 2025-12-04T09:23:57.5185651Z Submodule 'external/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/external/asmjit' 2025-12-04T09:23:57.5188597Z Submodule 'external/composable_kernel' (https://github.com/ROCm/composable_kernel.git) registered for path 'third_party/fbgemm/external/composable_kernel' 2025-12-04T09:23:57.5189327Z Submodule 'external/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/external/cpuinfo' 2025-12-04T09:23:57.5189882Z Submodule 'external/cutlass' (https://github.com/jwfromm/cutlass) registered for path 'third_party/fbgemm/external/cutlass' 2025-12-04T09:23:57.5190472Z Submodule 'external/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/external/googletest' 2025-12-04T09:23:57.5191125Z Submodule 'external/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/external/hipify_torch' 2025-12-04T09:23:57.5191775Z Submodule 'external/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/fbgemm/external/json' 2025-12-04T09:23:57.5225266Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/external/asmjit'... 2025-12-04T09:23:58.6340795Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/external/hipify_torch'... 2025-12-04T09:23:58.6341432Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/external/cpuinfo'... 2025-12-04T09:23:58.6341988Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/external/googletest'... 2025-12-04T09:23:58.7342147Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/external/composable_kernel'... 2025-12-04T09:24:02.2780348Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/external/cutlass'... 2025-12-04T09:24:02.3783121Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/external/json'... 2025-12-04T09:24:04.3724807Z Submodule path 'third_party/fbgemm/external/asmjit': checked out 'a3199e8857792cd10b7589ff5d58343d2c9008ea' 2025-12-04T09:24:04.6445388Z Submodule path 'third_party/fbgemm/external/composable_kernel': checked out '7fe50dc3da2069d6645d9deb8c017a876472a977' 2025-12-04T09:24:04.7330234Z Submodule path 'third_party/fbgemm/external/cpuinfo': checked out '6543fec09b2f04ac4a666882998b534afc9c1349' 2025-12-04T09:24:05.2514901Z Submodule path 'third_party/fbgemm/external/cutlass': checked out '98125ce499b0fdf7ffbe0e3052f5b8709f4840f8' 2025-12-04T09:24:05.2929450Z Submodule path 'third_party/fbgemm/external/googletest': checked out '52eb8108c5bdec04579160ae17225d66034bd723' 2025-12-04T09:24:05.3057648Z Submodule path 'third_party/fbgemm/external/hipify_torch': checked out '63b6a7b541fa7f08f8475ca7d74054db36ff2691' 2025-12-04T09:24:05.3977855Z Submodule path 'third_party/fbgemm/external/json': checked out '9cca280a4d0ccf0c08f47a99aa71d1b0e52f8d03' 2025-12-04T09:24:05.4812159Z Submodule path 'third_party/flash-attention': checked out '979702c87a8713a8e0a5e9fee122b90d2ef13be5' 2025-12-04T09:24:05.4826668Z Submodule 'csrc/composable_kernel' (https://github.com/ROCm/composable_kernel.git) registered for path 'third_party/flash-attention/csrc/composable_kernel' 2025-12-04T09:24:05.4828263Z Submodule 'csrc/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/flash-attention/csrc/cutlass' 2025-12-04T09:24:05.4854943Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flash-attention/csrc/composable_kernel'... 2025-12-04T09:24:10.2376701Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flash-attention/csrc/cutlass'... 2025-12-04T09:24:10.4148424Z Submodule path 'third_party/flash-attention/csrc/composable_kernel': checked out '888317e698e9803c62bd38568abc9e05d7709f33' 2025-12-04T09:24:10.8614882Z Submodule path 'third_party/flash-attention/csrc/cutlass': checked out 'c506e16788cb08416a4a57e11a9067beeee29420' 2025-12-04T09:24:10.9661988Z Submodule path 'third_party/flatbuffers': checked out 'a2cd1ea3b6d3fee220106b5fed3f7ce8da9eb757' 2025-12-04T09:24:10.9923195Z Submodule path 'third_party/fmt': checked out '407c905e45ad75fc29bf0f9bb7c5c2fd3475976f' 2025-12-04T09:24:11.0250602Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2025-12-04T09:24:11.0457714Z Submodule path 'third_party/gloo': checked out '54cbae0d3a67fa890b4c3d9ee162b7860315e341' 2025-12-04T09:24:11.0850983Z Submodule path 'third_party/googletest': checked out '52eb8108c5bdec04579160ae17225d66034bd723' 2025-12-04T09:24:11.0975975Z Submodule path 'third_party/ideep': checked out '719d8e6cd7f7a0e01b155657526d693acf97c2b3' 2025-12-04T09:24:11.0990705Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2025-12-04T09:24:11.1018471Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2025-12-04T09:24:26.2851435Z Submodule path 'third_party/ideep/mkl-dnn': checked out '8d263e693366ef8db40acc569cc7d8edf644556d' 2025-12-04T09:24:26.3033585Z Submodule path 'third_party/ittapi': checked out 'dec1d23ca65ab069d225dfe40dea14f455170959' 2025-12-04T09:24:26.3907250Z Submodule path 'third_party/kineto': checked out '31f85df8fbd89c188f14ef10f1ec65379786b943' 2025-12-04T09:24:26.3921159Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2025-12-04T09:24:26.3924103Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2025-12-04T09:24:26.3927205Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2025-12-04T09:24:26.3950034Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2025-12-04T09:24:27.0402761Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2025-12-04T09:24:28.0339652Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2025-12-04T09:24:28.1126383Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out 'd2ffe0a4e3acace628db49974246b66fc3e85fb1' 2025-12-04T09:24:28.1147667Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-12-04T09:24:28.1149003Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-12-04T09:24:28.1149771Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-12-04T09:24:28.1150555Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-12-04T09:24:28.1151581Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-12-04T09:24:28.1152458Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-12-04T09:24:28.1157926Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-12-04T09:24:28.1158799Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-12-04T09:24:28.1163517Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp' 2025-12-04T09:24:28.1188791Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2025-12-04T09:24:29.7701618Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2025-12-04T09:24:29.7702393Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2025-12-04T09:24:29.7703182Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp'... 2025-12-04T09:24:29.7703888Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2025-12-04T09:24:29.7704587Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2025-12-04T09:24:29.7705304Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2025-12-04T09:24:29.7706210Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2025-12-04T09:24:29.8706463Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2025-12-04T09:24:35.0218821Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2025-12-04T09:24:35.0384352Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2025-12-04T09:24:35.0706572Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2025-12-04T09:24:35.0836940Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2025-12-04T09:24:35.0847871Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-12-04T09:24:35.0880647Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2025-12-04T09:24:35.3305343Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2025-12-04T09:24:35.3474623Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2025-12-04T09:24:35.3877450Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '52eb8108c5bdec04579160ae17225d66034bd723' 2025-12-04T09:24:35.4727586Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2025-12-04T09:24:35.4875991Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2025-12-04T09:24:35.5023773Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp': checked out 'b1234816facfdda29845c46696a02998a4af115a' 2025-12-04T09:24:35.5040593Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:24:35.5041645Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:24:35.5070727Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb'... 2025-12-04T09:24:37.3069958Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest'... 2025-12-04T09:24:37.5207972Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'd7ba35bbb649209c66e582d5a0244ba988a15159' 2025-12-04T09:24:37.5607037Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2025-12-04T09:24:37.5903312Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '40626af88bd7df9a5fb80be7b25ac85b122d6c21' 2025-12-04T09:24:37.6307974Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '52eb8108c5bdec04579160ae17225d66034bd723' 2025-12-04T09:24:37.6784204Z Submodule path 'third_party/kleidiai': checked out 'd7770c89632329a9914ef1a90289917597639cbe' 2025-12-04T09:24:37.7138407Z Submodule path 'third_party/mimalloc': checked out 'fbd8b99c2b828428947d70fdc046bb55609be93e' 2025-12-04T09:24:37.8051377Z Submodule path 'third_party/nlohmann': checked out '55f93686c01528224f448c19128836e7df245f72' 2025-12-04T09:24:38.0955642Z Submodule path 'third_party/onnx': checked out 'e709452ef2bbc1d113faf678c24e6d3467696e83' 2025-12-04T09:24:38.0988887Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2025-12-04T09:24:38.1016481Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2025-12-04T09:24:38.9620685Z Submodule path 'third_party/onnx/third_party/pybind11': checked out 'a2e59f0e7065404b44dfe92a28aca47ba1378dc4' 2025-12-04T09:24:39.0177334Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2025-12-04T09:24:39.0193803Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-12-04T09:24:39.0194966Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2025-12-04T09:24:39.0195674Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-12-04T09:24:39.0196481Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-12-04T09:24:39.0197341Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-12-04T09:24:39.0198609Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-12-04T09:24:39.0201096Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-12-04T09:24:39.0202252Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-12-04T09:24:39.0235106Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2025-12-04T09:24:39.4691870Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2025-12-04T09:24:39.4692598Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2025-12-04T09:24:39.4693228Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2025-12-04T09:24:39.4693896Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2025-12-04T09:24:39.5692120Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2025-12-04T09:24:40.2672457Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2025-12-04T09:24:45.8131231Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2025-12-04T09:24:46.6600473Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2025-12-04T09:24:46.6952105Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2025-12-04T09:24:46.7116191Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2025-12-04T09:24:46.8025566Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2025-12-04T09:24:46.8153767Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2025-12-04T09:24:46.8295825Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2025-12-04T09:24:46.8430783Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2025-12-04T09:24:46.8443324Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:24:46.8444154Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:24:46.8472864Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2025-12-04T09:24:48.8519029Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2025-12-04T09:24:49.0577623Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2025-12-04T09:24:49.0958201Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2025-12-04T09:24:49.4127018Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2025-12-04T09:24:49.4234748Z Submodule path 'third_party/pocketfft': checked out '0fa0ef591e38c2758e3184c6c23e497b9f732ffa' 2025-12-04T09:24:49.6375854Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2025-12-04T09:24:49.6395876Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2025-12-04T09:24:49.6396653Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2025-12-04T09:24:49.6426564Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2025-12-04T09:24:50.1305419Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2025-12-04T09:24:50.5300589Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2025-12-04T09:24:50.5922395Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2025-12-04T09:24:50.6019710Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2025-12-04T09:24:50.6129603Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2025-12-04T09:24:50.6490396Z Submodule path 'third_party/pybind11': checked out 'f5fbe867d2d26e4a0a9177a51f6e568868ad3dc8' 2025-12-04T09:24:50.6738510Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2025-12-04T09:24:50.7111800Z Submodule path 'third_party/sleef': checked out '5a1d179df9cf652951b59010a2d2075372d67f68' 2025-12-04T09:24:50.7342271Z Submodule path 'third_party/tensorpipe': checked out '2b4cd91092d335a697416b2a3cb398283246849d' 2025-12-04T09:24:50.7358474Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2025-12-04T09:24:50.7359347Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2025-12-04T09:24:50.7360097Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2025-12-04T09:24:50.7367401Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2025-12-04T09:24:50.7386852Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2025-12-04T09:24:51.6109306Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2025-12-04T09:24:51.6109982Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2025-12-04T09:24:51.6884929Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2025-12-04T09:24:51.7389612Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2025-12-04T09:24:51.7530004Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2025-12-04T09:24:51.8152823Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '5152db2cbfeb5582e9c27c5ea1dba2cd9e10759b' 2025-12-04T09:24:51.8406731Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2025-12-04T09:24:51.8424021Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-12-04T09:24:51.8448863Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2025-12-04T09:24:52.0234235Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2025-12-04T09:24:52.0284827Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2025-12-04T09:24:52.0607889Z Entering 'android/libs/fbjni' 2025-12-04T09:24:52.0656101Z Entering 'third_party/FP16' 2025-12-04T09:24:52.0697447Z Entering 'third_party/FXdiv' 2025-12-04T09:24:52.0739444Z Entering 'third_party/NNPACK' 2025-12-04T09:24:52.0783042Z Entering 'third_party/NVTX' 2025-12-04T09:24:52.0822027Z Entering 'third_party/VulkanMemoryAllocator' 2025-12-04T09:24:52.0867533Z Entering 'third_party/XNNPACK' 2025-12-04T09:24:52.0916600Z Entering 'third_party/aiter' 2025-12-04T09:24:52.0955736Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-12-04T09:24:52.1004625Z Entering 'third_party/benchmark' 2025-12-04T09:24:52.1048483Z Entering 'third_party/composable_kernel' 2025-12-04T09:24:52.1096046Z Entering 'third_party/cpp-httplib' 2025-12-04T09:24:52.1139647Z Entering 'third_party/cpuinfo' 2025-12-04T09:24:52.1179821Z Entering 'third_party/cudnn_frontend' 2025-12-04T09:24:52.1222923Z Entering 'third_party/cutlass' 2025-12-04T09:24:52.1265166Z Entering 'third_party/fbgemm' 2025-12-04T09:24:52.1306684Z Entering 'third_party/fbgemm/external/asmjit' 2025-12-04T09:24:52.1350436Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-12-04T09:24:52.1393257Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-12-04T09:24:52.1437205Z Entering 'third_party/fbgemm/external/cutlass' 2025-12-04T09:24:52.1484893Z Entering 'third_party/fbgemm/external/googletest' 2025-12-04T09:24:52.1522984Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-12-04T09:24:52.1562299Z Entering 'third_party/fbgemm/external/json' 2025-12-04T09:24:52.1606058Z Entering 'third_party/flash-attention' 2025-12-04T09:24:52.1651171Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-12-04T09:24:52.1691712Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-12-04T09:24:52.1743688Z Entering 'third_party/flatbuffers' 2025-12-04T09:24:52.1784884Z Entering 'third_party/fmt' 2025-12-04T09:24:52.1831703Z Entering 'third_party/gemmlowp/gemmlowp' 2025-12-04T09:24:52.1870682Z Entering 'third_party/gloo' 2025-12-04T09:24:52.1913460Z Entering 'third_party/googletest' 2025-12-04T09:24:52.1959433Z Entering 'third_party/ideep' 2025-12-04T09:24:52.1995403Z Entering 'third_party/ideep/mkl-dnn' 2025-12-04T09:24:52.2041655Z Entering 'third_party/ittapi' 2025-12-04T09:24:52.2083539Z Entering 'third_party/kineto' 2025-12-04T09:24:52.2127552Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-12-04T09:24:52.2172160Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-12-04T09:24:52.2206603Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-12-04T09:24:52.2248569Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-12-04T09:24:52.2287701Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-12-04T09:24:52.2329671Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-12-04T09:24:52.2374932Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-12-04T09:24:52.2415278Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-12-04T09:24:52.2459918Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-12-04T09:24:52.2501007Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-12-04T09:24:52.2546076Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp' 2025-12-04T09:24:52.2589105Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:24:52.2630110Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:24:52.2681384Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-12-04T09:24:52.2719626Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-12-04T09:24:52.2766216Z Entering 'third_party/kleidiai' 2025-12-04T09:24:52.2804663Z Entering 'third_party/mimalloc' 2025-12-04T09:24:52.2851015Z Entering 'third_party/nlohmann' 2025-12-04T09:24:52.2896309Z Entering 'third_party/onnx' 2025-12-04T09:24:52.2951633Z Entering 'third_party/onnx/third_party/pybind11' 2025-12-04T09:24:52.2998332Z Entering 'third_party/opentelemetry-cpp' 2025-12-04T09:24:52.3043687Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-12-04T09:24:52.3083441Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-12-04T09:24:52.3121311Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-12-04T09:24:52.3163468Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-12-04T09:24:52.3201673Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-12-04T09:24:52.3240356Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-12-04T09:24:52.3284976Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-12-04T09:24:52.3320080Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:24:52.3366234Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:24:52.3403472Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-12-04T09:24:52.3461450Z Entering 'third_party/pocketfft' 2025-12-04T09:24:52.3500755Z Entering 'third_party/protobuf' 2025-12-04T09:24:52.3543948Z Entering 'third_party/protobuf/third_party/benchmark' 2025-12-04T09:24:52.3588632Z Entering 'third_party/protobuf/third_party/googletest' 2025-12-04T09:24:52.3628188Z Entering 'third_party/psimd' 2025-12-04T09:24:52.3671369Z Entering 'third_party/pthreadpool' 2025-12-04T09:24:52.3711864Z Entering 'third_party/pybind11' 2025-12-04T09:24:52.3753066Z Entering 'third_party/python-peachpy' 2025-12-04T09:24:52.3793405Z Entering 'third_party/sleef' 2025-12-04T09:24:52.3833818Z Entering 'third_party/tensorpipe' 2025-12-04T09:24:52.3879698Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-12-04T09:24:52.3912545Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-12-04T09:24:52.3954030Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-12-04T09:24:52.3994116Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-12-04T09:24:52.4036544Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-12-04T09:24:52.4096127Z ##[endgroup] 2025-12-04T09:24:52.4096480Z ##[group]Persisting credentials for submodules 2025-12-04T09:24:52.4099630Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2025-12-04T09:24:52.4421854Z Entering 'android/libs/fbjni' 2025-12-04T09:24:52.4479723Z Entering 'third_party/FP16' 2025-12-04T09:24:52.4540272Z Entering 'third_party/FXdiv' 2025-12-04T09:24:52.4590137Z Entering 'third_party/NNPACK' 2025-12-04T09:24:52.4652393Z Entering 'third_party/NVTX' 2025-12-04T09:24:52.4706508Z Entering 'third_party/VulkanMemoryAllocator' 2025-12-04T09:24:52.4767107Z Entering 'third_party/XNNPACK' 2025-12-04T09:24:52.4829987Z Entering 'third_party/aiter' 2025-12-04T09:24:52.4891069Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-12-04T09:24:52.4949893Z Entering 'third_party/benchmark' 2025-12-04T09:24:52.5017593Z Entering 'third_party/composable_kernel' 2025-12-04T09:24:52.5071963Z Entering 'third_party/cpp-httplib' 2025-12-04T09:24:52.5134512Z Entering 'third_party/cpuinfo' 2025-12-04T09:24:52.5187206Z Entering 'third_party/cudnn_frontend' 2025-12-04T09:24:52.5239264Z Entering 'third_party/cutlass' 2025-12-04T09:24:52.5301254Z Entering 'third_party/fbgemm' 2025-12-04T09:24:52.5362313Z Entering 'third_party/fbgemm/external/asmjit' 2025-12-04T09:24:52.5427913Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-12-04T09:24:52.5487267Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-12-04T09:24:52.5539187Z Entering 'third_party/fbgemm/external/cutlass' 2025-12-04T09:24:52.5600631Z Entering 'third_party/fbgemm/external/googletest' 2025-12-04T09:24:52.5657111Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-12-04T09:24:52.5708892Z Entering 'third_party/fbgemm/external/json' 2025-12-04T09:24:52.5772834Z Entering 'third_party/flash-attention' 2025-12-04T09:24:52.5823399Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-12-04T09:24:52.5880529Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-12-04T09:24:52.5944361Z Entering 'third_party/flatbuffers' 2025-12-04T09:24:52.6002778Z Entering 'third_party/fmt' 2025-12-04T09:24:52.6061337Z Entering 'third_party/gemmlowp/gemmlowp' 2025-12-04T09:24:52.6110918Z Entering 'third_party/gloo' 2025-12-04T09:24:52.6172644Z Entering 'third_party/googletest' 2025-12-04T09:24:52.6224003Z Entering 'third_party/ideep' 2025-12-04T09:24:52.6276226Z Entering 'third_party/ideep/mkl-dnn' 2025-12-04T09:24:52.6335812Z Entering 'third_party/ittapi' 2025-12-04T09:24:52.6387499Z Entering 'third_party/kineto' 2025-12-04T09:24:52.6452410Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-12-04T09:24:52.6502322Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-12-04T09:24:52.6558998Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-12-04T09:24:52.6612926Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-12-04T09:24:52.6669572Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-12-04T09:24:52.6716570Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-12-04T09:24:52.6773209Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-12-04T09:24:52.6826492Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-12-04T09:24:52.6874021Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-12-04T09:24:52.6930060Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-12-04T09:24:52.6984517Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp' 2025-12-04T09:24:52.7038351Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:24:52.7092256Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:24:52.7152194Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-12-04T09:24:52.7205999Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-12-04T09:24:52.7266987Z Entering 'third_party/kleidiai' 2025-12-04T09:24:52.7320202Z Entering 'third_party/mimalloc' 2025-12-04T09:24:52.7375552Z Entering 'third_party/nlohmann' 2025-12-04T09:24:52.7424990Z Entering 'third_party/onnx' 2025-12-04T09:24:52.7490027Z Entering 'third_party/onnx/third_party/pybind11' 2025-12-04T09:24:52.7556858Z Entering 'third_party/opentelemetry-cpp' 2025-12-04T09:24:52.7614976Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-12-04T09:24:52.7668344Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-12-04T09:24:52.7726072Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-12-04T09:24:52.7777653Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-12-04T09:24:52.7836568Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-12-04T09:24:52.7887227Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-12-04T09:24:52.7943120Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-12-04T09:24:52.7992052Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:24:52.8052325Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:24:52.8103021Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-12-04T09:24:52.8176270Z Entering 'third_party/pocketfft' 2025-12-04T09:24:52.8233466Z Entering 'third_party/protobuf' 2025-12-04T09:24:52.8289144Z Entering 'third_party/protobuf/third_party/benchmark' 2025-12-04T09:24:52.8341605Z Entering 'third_party/protobuf/third_party/googletest' 2025-12-04T09:24:52.8396039Z Entering 'third_party/psimd' 2025-12-04T09:24:52.8453752Z Entering 'third_party/pthreadpool' 2025-12-04T09:24:52.8504858Z Entering 'third_party/pybind11' 2025-12-04T09:24:52.8561555Z Entering 'third_party/python-peachpy' 2025-12-04T09:24:52.8615746Z Entering 'third_party/sleef' 2025-12-04T09:24:52.8667948Z Entering 'third_party/tensorpipe' 2025-12-04T09:24:52.8716933Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-12-04T09:24:52.8767981Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-12-04T09:24:52.8819932Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-12-04T09:24:52.8880707Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-12-04T09:24:52.8923173Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-12-04T09:24:52.9002121Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2025-12-04T09:24:52.9318102Z Entering 'android/libs/fbjni' 2025-12-04T09:24:52.9371087Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2025-12-04T09:24:52.9385968Z Entering 'third_party/FP16' 2025-12-04T09:24:52.9434630Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2025-12-04T09:24:52.9457164Z Entering 'third_party/FXdiv' 2025-12-04T09:24:52.9502032Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2025-12-04T09:24:52.9517746Z Entering 'third_party/NNPACK' 2025-12-04T09:24:52.9566877Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2025-12-04T09:24:52.9583330Z Entering 'third_party/NVTX' 2025-12-04T09:24:52.9629034Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NVTX/config remote.origin.url 2025-12-04T09:24:52.9646167Z Entering 'third_party/VulkanMemoryAllocator' 2025-12-04T09:24:52.9694897Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2025-12-04T09:24:52.9709808Z Entering 'third_party/XNNPACK' 2025-12-04T09:24:52.9761055Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2025-12-04T09:24:52.9789302Z Entering 'third_party/aiter' 2025-12-04T09:24:52.9842852Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/aiter/config remote.origin.url 2025-12-04T09:24:52.9860455Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-12-04T09:24:52.9905414Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/aiter/modules/3rdparty/composable_kernel/config remote.origin.url 2025-12-04T09:24:52.9934391Z Entering 'third_party/benchmark' 2025-12-04T09:24:52.9983087Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2025-12-04T09:24:52.9997910Z Entering 'third_party/composable_kernel' 2025-12-04T09:24:53.0051629Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/composable_kernel/config remote.origin.url 2025-12-04T09:24:53.0072746Z Entering 'third_party/cpp-httplib' 2025-12-04T09:24:53.0120983Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2025-12-04T09:24:53.0139460Z Entering 'third_party/cpuinfo' 2025-12-04T09:24:53.0190283Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2025-12-04T09:24:53.0203208Z Entering 'third_party/cudnn_frontend' 2025-12-04T09:24:53.0256826Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2025-12-04T09:24:53.0276088Z Entering 'third_party/cutlass' 2025-12-04T09:24:53.0321406Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2025-12-04T09:24:53.0343943Z Entering 'third_party/fbgemm' 2025-12-04T09:24:53.0394755Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2025-12-04T09:24:53.0412814Z Entering 'third_party/fbgemm/external/asmjit' 2025-12-04T09:24:53.0462192Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/asmjit/config remote.origin.url 2025-12-04T09:24:53.0478237Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-12-04T09:24:53.0523631Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/composable_kernel/config remote.origin.url 2025-12-04T09:24:53.0551292Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-12-04T09:24:53.0599322Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/cpuinfo/config remote.origin.url 2025-12-04T09:24:53.0617744Z Entering 'third_party/fbgemm/external/cutlass' 2025-12-04T09:24:53.0668141Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/cutlass/config remote.origin.url 2025-12-04T09:24:53.0691493Z Entering 'third_party/fbgemm/external/googletest' 2025-12-04T09:24:53.0739886Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/googletest/config remote.origin.url 2025-12-04T09:24:53.0756395Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-12-04T09:24:53.0805507Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/hipify_torch/config remote.origin.url 2025-12-04T09:24:53.0825761Z Entering 'third_party/fbgemm/external/json' 2025-12-04T09:24:53.0870618Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/json/config remote.origin.url 2025-12-04T09:24:53.0892002Z Entering 'third_party/flash-attention' 2025-12-04T09:24:53.0940945Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/config remote.origin.url 2025-12-04T09:24:53.0955009Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-12-04T09:24:53.1003683Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/modules/csrc/composable_kernel/config remote.origin.url 2025-12-04T09:24:53.1029843Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-12-04T09:24:53.1073308Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/modules/csrc/cutlass/config remote.origin.url 2025-12-04T09:24:53.1099668Z Entering 'third_party/flatbuffers' 2025-12-04T09:24:53.1149121Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2025-12-04T09:24:53.1171803Z Entering 'third_party/fmt' 2025-12-04T09:24:53.1217791Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2025-12-04T09:24:53.1237416Z Entering 'third_party/gemmlowp/gemmlowp' 2025-12-04T09:24:53.1286438Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2025-12-04T09:24:53.1301349Z Entering 'third_party/gloo' 2025-12-04T09:24:53.1346989Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2025-12-04T09:24:53.1361888Z Entering 'third_party/googletest' 2025-12-04T09:24:53.1416717Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2025-12-04T09:24:53.1428760Z Entering 'third_party/ideep' 2025-12-04T09:24:53.1480818Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2025-12-04T09:24:53.1494177Z Entering 'third_party/ideep/mkl-dnn' 2025-12-04T09:24:53.1542239Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2025-12-04T09:24:53.1569150Z Entering 'third_party/ittapi' 2025-12-04T09:24:53.1610323Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2025-12-04T09:24:53.1628075Z Entering 'third_party/kineto' 2025-12-04T09:24:53.1682027Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2025-12-04T09:24:53.1698663Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-12-04T09:24:53.1749096Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2025-12-04T09:24:53.1764012Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-12-04T09:24:53.1807951Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2025-12-04T09:24:53.1826770Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-12-04T09:24:53.1872767Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2025-12-04T09:24:53.1890436Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-12-04T09:24:53.1936420Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2025-12-04T09:24:53.1956732Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-12-04T09:24:53.2002879Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2025-12-04T09:24:53.2020001Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-12-04T09:24:53.2065029Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2025-12-04T09:24:53.2087117Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-12-04T09:24:53.2132717Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2025-12-04T09:24:53.2152741Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-12-04T09:24:53.2201342Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2025-12-04T09:24:53.2218256Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-12-04T09:24:53.2268784Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2025-12-04T09:24:53.2289509Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-12-04T09:24:53.2339022Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2025-12-04T09:24:53.2355043Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp' 2025-12-04T09:24:53.2396713Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/prometheus-cpp/config remote.origin.url 2025-12-04T09:24:53.2412523Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:24:53.2458755Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2025-12-04T09:24:53.2477606Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:24:53.2528723Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2025-12-04T09:24:53.2554414Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-12-04T09:24:53.2600246Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2025-12-04T09:24:53.2623354Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-12-04T09:24:53.2668875Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2025-12-04T09:24:53.2689810Z Entering 'third_party/kleidiai' 2025-12-04T09:24:53.2740511Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kleidiai/config remote.origin.url 2025-12-04T09:24:53.2762334Z Entering 'third_party/mimalloc' 2025-12-04T09:24:53.2811739Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2025-12-04T09:24:53.2825756Z Entering 'third_party/nlohmann' 2025-12-04T09:24:53.2871132Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2025-12-04T09:24:53.2889972Z Entering 'third_party/onnx' 2025-12-04T09:24:53.2933453Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2025-12-04T09:24:53.2962299Z Entering 'third_party/onnx/third_party/pybind11' 2025-12-04T09:24:53.3010823Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2025-12-04T09:24:53.3031350Z Entering 'third_party/opentelemetry-cpp' 2025-12-04T09:24:53.3080365Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2025-12-04T09:24:53.3100956Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-12-04T09:24:53.3149663Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2025-12-04T09:24:53.3167999Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-12-04T09:24:53.3214591Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2025-12-04T09:24:53.3231932Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-12-04T09:24:53.3282692Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2025-12-04T09:24:53.3294923Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-12-04T09:24:53.3345708Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2025-12-04T09:24:53.3364791Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-12-04T09:24:53.3407218Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2025-12-04T09:24:53.3423618Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-12-04T09:24:53.3472794Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2025-12-04T09:24:53.3489846Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-12-04T09:24:53.3542753Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2025-12-04T09:24:53.3552809Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:24:53.3600939Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2025-12-04T09:24:53.3622153Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:24:53.3674250Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2025-12-04T09:24:53.3692407Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-12-04T09:24:53.3743748Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2025-12-04T09:24:53.3777335Z Entering 'third_party/pocketfft' 2025-12-04T09:24:53.3826763Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2025-12-04T09:24:53.3845682Z Entering 'third_party/protobuf' 2025-12-04T09:24:53.3887915Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2025-12-04T09:24:53.3905219Z Entering 'third_party/protobuf/third_party/benchmark' 2025-12-04T09:24:53.3954976Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2025-12-04T09:24:53.3974475Z Entering 'third_party/protobuf/third_party/googletest' 2025-12-04T09:24:53.4020290Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2025-12-04T09:24:53.4045545Z Entering 'third_party/psimd' 2025-12-04T09:24:53.4091050Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2025-12-04T09:24:53.4106885Z Entering 'third_party/pthreadpool' 2025-12-04T09:24:53.4156624Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2025-12-04T09:24:53.4175388Z Entering 'third_party/pybind11' 2025-12-04T09:24:53.4226313Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2025-12-04T09:24:53.4251433Z Entering 'third_party/python-peachpy' 2025-12-04T09:24:53.4295514Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2025-12-04T09:24:53.4313412Z Entering 'third_party/sleef' 2025-12-04T09:24:53.4361328Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2025-12-04T09:24:53.4377423Z Entering 'third_party/tensorpipe' 2025-12-04T09:24:53.4429442Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2025-12-04T09:24:53.4444687Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-12-04T09:24:53.4490355Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2025-12-04T09:24:53.4507212Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-12-04T09:24:53.4560975Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2025-12-04T09:24:53.4578816Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-12-04T09:24:53.4627029Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2025-12-04T09:24:53.4648812Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-12-04T09:24:53.4701108Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2025-12-04T09:24:53.4712884Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-12-04T09:24:53.4762145Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2025-12-04T09:24:53.5919307Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2025-12-04T09:24:53.6232169Z Entering 'android/libs/fbjni' 2025-12-04T09:24:53.6274071Z Entering 'third_party/FP16' 2025-12-04T09:24:53.6316200Z Entering 'third_party/FXdiv' 2025-12-04T09:24:53.6357717Z Entering 'third_party/NNPACK' 2025-12-04T09:24:53.6398248Z Entering 'third_party/NVTX' 2025-12-04T09:24:53.6440466Z Entering 'third_party/VulkanMemoryAllocator' 2025-12-04T09:24:53.6486757Z Entering 'third_party/XNNPACK' 2025-12-04T09:24:53.6534113Z Entering 'third_party/aiter' 2025-12-04T09:24:53.6574589Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-12-04T09:24:53.6623408Z Entering 'third_party/benchmark' 2025-12-04T09:24:53.6666296Z Entering 'third_party/composable_kernel' 2025-12-04T09:24:53.6708876Z Entering 'third_party/cpp-httplib' 2025-12-04T09:24:53.6750337Z Entering 'third_party/cpuinfo' 2025-12-04T09:24:53.6790109Z Entering 'third_party/cudnn_frontend' 2025-12-04T09:24:53.6835530Z Entering 'third_party/cutlass' 2025-12-04T09:24:53.6887808Z Entering 'third_party/fbgemm' 2025-12-04T09:24:53.6924019Z Entering 'third_party/fbgemm/external/asmjit' 2025-12-04T09:24:53.6969120Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-12-04T09:24:53.7017778Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-12-04T09:24:53.7059834Z Entering 'third_party/fbgemm/external/cutlass' 2025-12-04T09:24:53.7106145Z Entering 'third_party/fbgemm/external/googletest' 2025-12-04T09:24:53.7150586Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-12-04T09:24:53.7190377Z Entering 'third_party/fbgemm/external/json' 2025-12-04T09:24:53.7235736Z Entering 'third_party/flash-attention' 2025-12-04T09:24:53.7275102Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-12-04T09:24:53.7316555Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-12-04T09:24:53.7366008Z Entering 'third_party/flatbuffers' 2025-12-04T09:24:53.7404957Z Entering 'third_party/fmt' 2025-12-04T09:24:53.7447505Z Entering 'third_party/gemmlowp/gemmlowp' 2025-12-04T09:24:53.7490947Z Entering 'third_party/gloo' 2025-12-04T09:24:53.7532104Z Entering 'third_party/googletest' 2025-12-04T09:24:53.7570341Z Entering 'third_party/ideep' 2025-12-04T09:24:53.7605331Z Entering 'third_party/ideep/mkl-dnn' 2025-12-04T09:24:53.7652106Z Entering 'third_party/ittapi' 2025-12-04T09:24:53.7692942Z Entering 'third_party/kineto' 2025-12-04T09:24:53.7732029Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-12-04T09:24:53.7768870Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-12-04T09:24:53.7807715Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-12-04T09:24:53.7853119Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-12-04T09:24:53.7890994Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-12-04T09:24:53.7927334Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-12-04T09:24:53.7970795Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-12-04T09:24:53.8010407Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-12-04T09:24:53.8053178Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-12-04T09:24:53.8089259Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-12-04T09:24:53.8126657Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp' 2025-12-04T09:24:53.8164695Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:24:53.8209533Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:24:53.8260826Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-12-04T09:24:53.8304086Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-12-04T09:24:53.8345330Z Entering 'third_party/kleidiai' 2025-12-04T09:24:53.8390007Z Entering 'third_party/mimalloc' 2025-12-04T09:24:53.8433328Z Entering 'third_party/nlohmann' 2025-12-04T09:24:53.8474917Z Entering 'third_party/onnx' 2025-12-04T09:24:53.8531900Z Entering 'third_party/onnx/third_party/pybind11' 2025-12-04T09:24:53.8568980Z Entering 'third_party/opentelemetry-cpp' 2025-12-04T09:24:53.8613680Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-12-04T09:24:53.8649767Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-12-04T09:24:53.8688577Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-12-04T09:24:53.8729286Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-12-04T09:24:53.8768826Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-12-04T09:24:53.8806064Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-12-04T09:24:53.8851343Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-12-04T09:24:53.8886663Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:24:53.8927986Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:24:53.8972218Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-12-04T09:24:53.9027610Z Entering 'third_party/pocketfft' 2025-12-04T09:24:53.9073152Z Entering 'third_party/protobuf' 2025-12-04T09:24:53.9113978Z Entering 'third_party/protobuf/third_party/benchmark' 2025-12-04T09:24:53.9151986Z Entering 'third_party/protobuf/third_party/googletest' 2025-12-04T09:24:53.9195795Z Entering 'third_party/psimd' 2025-12-04T09:24:53.9233892Z Entering 'third_party/pthreadpool' 2025-12-04T09:24:53.9279208Z Entering 'third_party/pybind11' 2025-12-04T09:24:53.9315152Z Entering 'third_party/python-peachpy' 2025-12-04T09:24:53.9363952Z Entering 'third_party/sleef' 2025-12-04T09:24:53.9398041Z Entering 'third_party/tensorpipe' 2025-12-04T09:24:53.9440752Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-12-04T09:24:53.9481869Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-12-04T09:24:53.9521852Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-12-04T09:24:53.9566790Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-12-04T09:24:53.9605622Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-12-04T09:24:53.9664464Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2025-12-04T09:24:53.9984110Z Entering 'android/libs/fbjni' 2025-12-04T09:24:54.0026675Z Entering 'third_party/FP16' 2025-12-04T09:24:54.0066824Z Entering 'third_party/FXdiv' 2025-12-04T09:24:54.0109597Z Entering 'third_party/NNPACK' 2025-12-04T09:24:54.0149457Z Entering 'third_party/NVTX' 2025-12-04T09:24:54.0189189Z Entering 'third_party/VulkanMemoryAllocator' 2025-12-04T09:24:54.0233450Z Entering 'third_party/XNNPACK' 2025-12-04T09:24:54.0291200Z Entering 'third_party/aiter' 2025-12-04T09:24:54.0331317Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-12-04T09:24:54.0379699Z Entering 'third_party/benchmark' 2025-12-04T09:24:54.0419275Z Entering 'third_party/composable_kernel' 2025-12-04T09:24:54.0466826Z Entering 'third_party/cpp-httplib' 2025-12-04T09:24:54.0507142Z Entering 'third_party/cpuinfo' 2025-12-04T09:24:54.0549218Z Entering 'third_party/cudnn_frontend' 2025-12-04T09:24:54.0592050Z Entering 'third_party/cutlass' 2025-12-04T09:24:54.0639445Z Entering 'third_party/fbgemm' 2025-12-04T09:24:54.0683913Z Entering 'third_party/fbgemm/external/asmjit' 2025-12-04T09:24:54.0721354Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-12-04T09:24:54.0772546Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-12-04T09:24:54.0815624Z Entering 'third_party/fbgemm/external/cutlass' 2025-12-04T09:24:54.0863383Z Entering 'third_party/fbgemm/external/googletest' 2025-12-04T09:24:54.0900491Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-12-04T09:24:54.0944825Z Entering 'third_party/fbgemm/external/json' 2025-12-04T09:24:54.0990193Z Entering 'third_party/flash-attention' 2025-12-04T09:24:54.1031503Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-12-04T09:24:54.1078714Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-12-04T09:24:54.1130059Z Entering 'third_party/flatbuffers' 2025-12-04T09:24:54.1174949Z Entering 'third_party/fmt' 2025-12-04T09:24:54.1211797Z Entering 'third_party/gemmlowp/gemmlowp' 2025-12-04T09:24:54.1252205Z Entering 'third_party/gloo' 2025-12-04T09:24:54.1293205Z Entering 'third_party/googletest' 2025-12-04T09:24:54.1329893Z Entering 'third_party/ideep' 2025-12-04T09:24:54.1367213Z Entering 'third_party/ideep/mkl-dnn' 2025-12-04T09:24:54.1410253Z Entering 'third_party/ittapi' 2025-12-04T09:24:54.1457547Z Entering 'third_party/kineto' 2025-12-04T09:24:54.1497640Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-12-04T09:24:54.1537415Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-12-04T09:24:54.1578848Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-12-04T09:24:54.1618546Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-12-04T09:24:54.1660837Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-12-04T09:24:54.1697117Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-12-04T09:24:54.1740512Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-12-04T09:24:54.1781909Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-12-04T09:24:54.1822400Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-12-04T09:24:54.1859009Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-12-04T09:24:54.1894732Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp' 2025-12-04T09:24:54.1937150Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:24:54.1982250Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:24:54.2026678Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-12-04T09:24:54.2068674Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-12-04T09:24:54.2112868Z Entering 'third_party/kleidiai' 2025-12-04T09:24:54.2158481Z Entering 'third_party/mimalloc' 2025-12-04T09:24:54.2196393Z Entering 'third_party/nlohmann' 2025-12-04T09:24:54.2234951Z Entering 'third_party/onnx' 2025-12-04T09:24:54.2285550Z Entering 'third_party/onnx/third_party/pybind11' 2025-12-04T09:24:54.2322849Z Entering 'third_party/opentelemetry-cpp' 2025-12-04T09:24:54.2366340Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-12-04T09:24:54.2402877Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-12-04T09:24:54.2442679Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-12-04T09:24:54.2483737Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-12-04T09:24:54.2524303Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-12-04T09:24:54.2566643Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-12-04T09:24:54.2605280Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-12-04T09:24:54.2646963Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:24:54.2686410Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:24:54.2732329Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-12-04T09:24:54.2790499Z Entering 'third_party/pocketfft' 2025-12-04T09:24:54.2831855Z Entering 'third_party/protobuf' 2025-12-04T09:24:54.2874705Z Entering 'third_party/protobuf/third_party/benchmark' 2025-12-04T09:24:54.2913219Z Entering 'third_party/protobuf/third_party/googletest' 2025-12-04T09:24:54.2958081Z Entering 'third_party/psimd' 2025-12-04T09:24:54.2993253Z Entering 'third_party/pthreadpool' 2025-12-04T09:24:54.3034886Z Entering 'third_party/pybind11' 2025-12-04T09:24:54.3077703Z Entering 'third_party/python-peachpy' 2025-12-04T09:24:54.3117530Z Entering 'third_party/sleef' 2025-12-04T09:24:54.3157856Z Entering 'third_party/tensorpipe' 2025-12-04T09:24:54.3192044Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-12-04T09:24:54.3235387Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-12-04T09:24:54.3273948Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-12-04T09:24:54.3310358Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-12-04T09:24:54.3350343Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-12-04T09:24:54.3401622Z ##[endgroup] 2025-12-04T09:24:54.3436605Z [command]/usr/bin/git log -1 --format=%H 2025-12-04T09:24:54.3464181Z ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:24:54.3589530Z ##[group]Run cd "${GITHUB_WORKSPACE}" 2025-12-04T09:24:54.3589795Z cd "${GITHUB_WORKSPACE}" 2025-12-04T09:24:54.3590023Z # Clean stale submodule dirs 2025-12-04T09:24:54.3590242Z if [ -z "${NO_SUDO}" ]; then 2025-12-04T09:24:54.3590528Z  sudo git submodule foreach --recursive git clean -ffdx 2025-12-04T09:24:54.3590771Z else 2025-12-04T09:24:54.3590973Z  git submodule foreach --recursive git clean -ffdx 2025-12-04T09:24:54.3591204Z fi 2025-12-04T09:24:54.3600010Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:24:54.3600261Z env: 2025-12-04T09:24:54.3600423Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:24:54.3600598Z NO_SUDO: true 2025-12-04T09:24:54.3600758Z ##[endgroup] 2025-12-04T09:24:54.3930999Z Entering 'android/libs/fbjni' 2025-12-04T09:24:54.3959205Z Entering 'third_party/FP16' 2025-12-04T09:24:54.3991272Z Entering 'third_party/FXdiv' 2025-12-04T09:24:54.4023586Z Entering 'third_party/NNPACK' 2025-12-04T09:24:54.4059992Z Entering 'third_party/NVTX' 2025-12-04T09:24:54.4095530Z Entering 'third_party/VulkanMemoryAllocator' 2025-12-04T09:24:54.4127626Z Entering 'third_party/XNNPACK' 2025-12-04T09:24:54.4229950Z Entering 'third_party/aiter' 2025-12-04T09:24:54.4268381Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-12-04T09:24:54.4354986Z Entering 'third_party/benchmark' 2025-12-04T09:24:54.4386223Z Entering 'third_party/composable_kernel' 2025-12-04T09:24:54.4487696Z Entering 'third_party/cpp-httplib' 2025-12-04T09:24:54.4522792Z Entering 'third_party/cpuinfo' 2025-12-04T09:24:54.4554475Z Entering 'third_party/cudnn_frontend' 2025-12-04T09:24:54.4589272Z Entering 'third_party/cutlass' 2025-12-04T09:24:54.4672502Z Entering 'third_party/fbgemm' 2025-12-04T09:24:54.4721987Z Entering 'third_party/fbgemm/external/asmjit' 2025-12-04T09:24:54.4751617Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-12-04T09:24:54.4847330Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-12-04T09:24:54.4882454Z Entering 'third_party/fbgemm/external/cutlass' 2025-12-04T09:24:54.4966871Z Entering 'third_party/fbgemm/external/googletest' 2025-12-04T09:24:54.4997596Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-12-04T09:24:54.5028308Z Entering 'third_party/fbgemm/external/json' 2025-12-04T09:24:54.5070051Z Entering 'third_party/flash-attention' 2025-12-04T09:24:54.5103674Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-12-04T09:24:54.5185872Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-12-04T09:24:54.5258984Z Entering 'third_party/flatbuffers' 2025-12-04T09:24:54.5319551Z Entering 'third_party/fmt' 2025-12-04T09:24:54.5353591Z Entering 'third_party/gemmlowp/gemmlowp' 2025-12-04T09:24:54.5386926Z Entering 'third_party/gloo' 2025-12-04T09:24:54.5419827Z Entering 'third_party/googletest' 2025-12-04T09:24:54.5453158Z Entering 'third_party/ideep' 2025-12-04T09:24:54.5481129Z Entering 'third_party/ideep/mkl-dnn' 2025-12-04T09:24:54.5551225Z Entering 'third_party/ittapi' 2025-12-04T09:24:54.5583785Z Entering 'third_party/kineto' 2025-12-04T09:24:54.5616589Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-12-04T09:24:54.5653309Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-12-04T09:24:54.5692095Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-12-04T09:24:54.5723892Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-12-04T09:24:54.5763405Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-12-04T09:24:54.5792117Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-12-04T09:24:54.5826020Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-12-04T09:24:54.5857455Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-12-04T09:24:54.5894109Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-12-04T09:24:54.5927555Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-12-04T09:24:54.5956652Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp' 2025-12-04T09:24:54.5986575Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:24:54.6032759Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:24:54.6073732Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-12-04T09:24:54.6102459Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-12-04T09:24:54.6139172Z Entering 'third_party/kleidiai' 2025-12-04T09:24:54.6176650Z Entering 'third_party/mimalloc' 2025-12-04T09:24:54.6208040Z Entering 'third_party/nlohmann' 2025-12-04T09:24:54.6250058Z Entering 'third_party/onnx' 2025-12-04T09:24:54.6475511Z Entering 'third_party/onnx/third_party/pybind11' 2025-12-04T09:24:54.6514207Z Entering 'third_party/opentelemetry-cpp' 2025-12-04T09:24:54.6563483Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-12-04T09:24:54.6591178Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-12-04T09:24:54.6631110Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-12-04T09:24:54.6659738Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-12-04T09:24:54.6699362Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-12-04T09:24:54.6727169Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-12-04T09:24:54.6762599Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-12-04T09:24:54.6789306Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:24:54.6834278Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:24:54.6871436Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-12-04T09:24:54.7063854Z Entering 'third_party/pocketfft' 2025-12-04T09:24:54.7092306Z Entering 'third_party/protobuf' 2025-12-04T09:24:54.7156546Z Entering 'third_party/protobuf/third_party/benchmark' 2025-12-04T09:24:54.7191357Z Entering 'third_party/protobuf/third_party/googletest' 2025-12-04T09:24:54.7230911Z Entering 'third_party/psimd' 2025-12-04T09:24:54.7259537Z Entering 'third_party/pthreadpool' 2025-12-04T09:24:54.7292478Z Entering 'third_party/pybind11' 2025-12-04T09:24:54.7329585Z Entering 'third_party/python-peachpy' 2025-12-04T09:24:54.7360110Z Entering 'third_party/sleef' 2025-12-04T09:24:54.7391307Z Entering 'third_party/tensorpipe' 2025-12-04T09:24:54.7427283Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-12-04T09:24:54.7461857Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-12-04T09:24:54.7494909Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-12-04T09:24:54.7527040Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-12-04T09:24:54.7554479Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-12-04T09:24:54.7683953Z Prepare all required actions 2025-12-04T09:24:54.7684467Z Getting action download info 2025-12-04T09:24:54.9297266Z ##[group]Run ./.github/actions/setup-linux 2025-12-04T09:24:54.9297506Z env: 2025-12-04T09:24:54.9297679Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:24:54.9297873Z ##[endgroup] 2025-12-04T09:24:54.9331358Z ##[group]Run set -euo pipefail 2025-12-04T09:24:54.9331626Z set -euo pipefail 2025-12-04T09:24:54.9331833Z function get_ec2_metadata() { 2025-12-04T09:24:54.9332092Z  # Pulled from instance metadata endpoint for EC2 2025-12-04T09:24:54.9332504Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2025-12-04T09:24:54.9332869Z  category=$1 2025-12-04T09:24:54.9333117Z  # If it is GCP runner (runner name contains gcp), do not run this 2025-12-04T09:24:54.9333395Z  runner_name_str=i-04ab61807b125ed1d 2025-12-04T09:24:54.9333681Z  if [[ -f /.inarc ]]; then 2025-12-04T09:24:54.9333913Z  echo "ARC Runner, no info on ec2 metadata" 2025-12-04T09:24:54.9334185Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2025-12-04T09:24:54.9334487Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2025-12-04T09:24:54.9334764Z  else 2025-12-04T09:24:54.9335304Z  curl -H "X-aws-ec2-metadata-token: $(curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 30")" -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2025-12-04T09:24:54.9335853Z  fi 2025-12-04T09:24:54.9336012Z } 2025-12-04T09:24:54.9336202Z echo "ami-id: $(get_ec2_metadata ami-id)" 2025-12-04T09:24:54.9336480Z echo "instance-id: $(get_ec2_metadata instance-id)" 2025-12-04T09:24:54.9336795Z echo "instance-type: $(get_ec2_metadata instance-type)" 2025-12-04T09:24:54.9337067Z echo "system info $(uname -a)" 2025-12-04T09:24:54.9342369Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:24:54.9342778Z env: 2025-12-04T09:24:54.9342939Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:24:54.9343115Z ##[endgroup] 2025-12-04T09:24:54.9482182Z ami-id: ami-08982f1c5bf93d976 2025-12-04T09:24:54.9589268Z instance-id: i-04ab61807b125ed1d 2025-12-04T09:24:54.9680538Z instance-type: m7i-flex.8xlarge 2025-12-04T09:24:54.9690202Z system info Linux ip-10-0-71-55.ec2.internal 6.1.150-174.273.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Sep 9 12:21:26 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux 2025-12-04T09:24:54.9710611Z ##[group]Run if [ -f /usr/bin/nvidia-smi ]; then nvidia-smi; fi 2025-12-04T09:24:54.9710974Z if [ -f /usr/bin/nvidia-smi ]; then nvidia-smi; fi 2025-12-04T09:24:54.9716125Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:24:54.9716383Z env: 2025-12-04T09:24:54.9716552Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:24:54.9716776Z ##[endgroup] 2025-12-04T09:24:54.9778247Z ##[group]Run echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-12-04T09:24:54.9778866Z echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-12-04T09:24:54.9783345Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:24:54.9783587Z env: 2025-12-04T09:24:54.9783746Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:24:54.9783930Z ##[endgroup] 2025-12-04T09:24:54.9834703Z ##[group]Run if systemctl is-active --quiet docker; then 2025-12-04T09:24:54.9834999Z if systemctl is-active --quiet docker; then 2025-12-04T09:24:54.9835251Z  echo "Docker daemon is running..."; 2025-12-04T09:24:54.9835464Z else 2025-12-04T09:24:54.9835691Z  echo "Starting docker daemon..." && sudo systemctl start docker; 2025-12-04T09:24:54.9835951Z fi 2025-12-04T09:24:54.9840040Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:24:54.9840295Z env: 2025-12-04T09:24:54.9840443Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:24:54.9840618Z ##[endgroup] 2025-12-04T09:24:54.9919450Z Docker daemon is running... 2025-12-04T09:24:54.9949875Z ##[group]Run nick-fields/retry@v3.0.0 2025-12-04T09:24:54.9950123Z with: 2025-12-04T09:24:54.9950275Z shell: bash 2025-12-04T09:24:54.9950448Z timeout_minutes: 5 2025-12-04T09:24:54.9950629Z max_attempts: 3 2025-12-04T09:24:54.9950802Z retry_wait_seconds: 30 2025-12-04T09:24:54.9952359Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" # For LF Runners we need to make sure we also login to Meta's ECR docker registry too. META_AWS_ACCOUNT_ID=308535385114 if [ "$AWS_ACCOUNT_ID" != "$META_AWS_ACCOUNT_ID" ] ; then aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$META_AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" fi 2025-12-04T09:24:54.9953943Z polling_interval_seconds: 1 2025-12-04T09:24:54.9954155Z warning_on_retry: true 2025-12-04T09:24:54.9954350Z continue_on_error: false 2025-12-04T09:24:54.9954536Z env: 2025-12-04T09:24:54.9954713Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:24:54.9954906Z AWS_RETRY_MODE: standard 2025-12-04T09:24:54.9955087Z AWS_MAX_ATTEMPTS: 5 2025-12-04T09:24:54.9955283Z AWS_DEFAULT_REGION: us-east-1 2025-12-04T09:24:54.9955487Z ##[endgroup] 2025-12-04T09:24:55.9686223Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2025-12-04T09:24:55.9688785Z Configure a credential helper to remove this warning. See 2025-12-04T09:24:55.9689252Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2025-12-04T09:24:55.9689521Z 2025-12-04T09:24:55.9689622Z Login Succeeded 2025-12-04T09:24:56.0625358Z Command completed after 1 attempt(s). 2025-12-04T09:24:56.0674265Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2025-12-04T09:24:56.0674651Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2025-12-04T09:24:56.0674975Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2025-12-04T09:24:56.0680067Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:24:56.0680299Z env: 2025-12-04T09:24:56.0680456Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:24:56.0680633Z ##[endgroup] 2025-12-04T09:24:56.0755100Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2025-12-04T09:24:56.0755479Z # ignore expansion of "docker ps -q" since it could be empty 2025-12-04T09:24:56.0755740Z # shellcheck disable=SC2046 2025-12-04T09:24:56.0755965Z docker stop $(docker ps -q) || true 2025-12-04T09:24:56.0756187Z # Prune all of the docker images 2025-12-04T09:24:56.0756408Z docker system prune -af 2025-12-04T09:24:56.0760968Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:24:56.0761219Z env: 2025-12-04T09:24:56.0761368Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:24:56.0761542Z ##[endgroup] 2025-12-04T09:24:56.1151490Z "docker stop" requires at least 1 argument. 2025-12-04T09:24:56.1151815Z See 'docker stop --help'. 2025-12-04T09:24:56.1151937Z 2025-12-04T09:24:56.1152058Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2025-12-04T09:24:56.1152234Z 2025-12-04T09:24:56.1152316Z Stop one or more running containers 2025-12-04T09:24:56.1369046Z Total reclaimed space: 0B 2025-12-04T09:24:56.1489617Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2025-12-04T09:24:56.1489951Z with: 2025-12-04T09:24:56.1490524Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:24:56.1491120Z use-custom-docker-registry: true 2025-12-04T09:24:56.1491344Z docker-build-dir: .ci/docker 2025-12-04T09:24:56.1491571Z docker-build-script: ./build.sh 2025-12-04T09:24:56.1491774Z working-directory: . 2025-12-04T09:24:56.1492014Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-12-04T09:24:56.1492279Z force-push: false 2025-12-04T09:24:56.1492446Z env: 2025-12-04T09:24:56.1492595Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:24:56.1492777Z ##[endgroup] 2025-12-04T09:24:56.1506353Z ##[group]Run set -ex 2025-12-04T09:24:56.1506598Z set -ex 2025-12-04T09:24:56.1506767Z  2025-12-04T09:24:56.1507085Z # If the docker build directory or the build script doesn't exist, the action will 2025-12-04T09:24:56.1507533Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2025-12-04T09:24:56.1507919Z # job could then download the pre-built image as usual 2025-12-04T09:24:56.1508360Z if [[ -d "${DOCKER_BUILD_DIR}" ]] && [[ -f "${DOCKER_BUILD_DIR}/${DOCKER_BUILD_SCRIPT}" ]] && [[ "${USE_CUSTOM_DOCKER_REGISTRY}" == "true" ]]; then 2025-12-04T09:24:56.1508752Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2025-12-04T09:24:56.1508974Z else 2025-12-04T09:24:56.1509153Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2025-12-04T09:24:56.1509438Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2025-12-04T09:24:56.1509689Z  2025-12-04T09:24:56.1510034Z  echo "Not using custom ECR registry. Either it was not requested or there is no Docker build script in the ${REPO_NAME} repo..." 2025-12-04T09:24:56.1510413Z  exit 0 2025-12-04T09:24:56.1510558Z fi 2025-12-04T09:24:56.1510701Z  2025-12-04T09:24:56.1510926Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2025-12-04T09:24:56.1511309Z  # The docker image name already includes the ECR prefix and tag, so we can just 2025-12-04T09:24:56.1511650Z  # use it as it is, but first let's extract the tag 2025-12-04T09:24:56.1512076Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2025-12-04T09:24:56.1512414Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2025-12-04T09:24:56.1512730Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2025-12-04T09:24:56.1513125Z else 2025-12-04T09:24:56.1513326Z  if [[ "${DOCKER_IMAGE_NAME}" == *:* ]]; then 2025-12-04T09:24:56.1513590Z  CUSTOM_TAG_PREFIX=${DOCKER_IMAGE_NAME#*:} 2025-12-04T09:24:56.1513862Z  DOCKER_IMAGE_NAME=${DOCKER_IMAGE_NAME%%:*} 2025-12-04T09:24:56.1514099Z  fi 2025-12-04T09:24:56.1514416Z  DOCKER_TAG=${CUSTOM_TAG_PREFIX:+${CUSTOM_TAG_PREFIX}-}$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2025-12-04T09:24:56.1514825Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2025-12-04T09:24:56.1515213Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2025-12-04T09:24:56.1515653Z  echo "custom-tag-prefix=${CUSTOM_TAG_PREFIX}" >> "${GITHUB_OUTPUT}" 2025-12-04T09:24:56.1515915Z fi 2025-12-04T09:24:56.1520615Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:24:56.1520858Z env: 2025-12-04T09:24:56.1521016Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:24:56.1521189Z REPO_NAME: pytorch 2025-12-04T09:24:56.1521851Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:24:56.1522412Z DOCKER_BUILD_DIR: .ci/docker 2025-12-04T09:24:56.1522607Z DOCKER_BUILD_SCRIPT: ./build.sh 2025-12-04T09:24:56.1522855Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-12-04T09:24:56.1523121Z USE_CUSTOM_DOCKER_REGISTRY: true 2025-12-04T09:24:56.1523321Z CUSTOM_TAG_PREFIX: 2025-12-04T09:24:56.1523483Z ##[endgroup] 2025-12-04T09:24:56.1547724Z + [[ -d .ci/docker ]] 2025-12-04T09:24:56.1548123Z + [[ -f .ci/docker/./build.sh ]] 2025-12-04T09:24:56.1548425Z + [[ true == \t\r\u\e ]] 2025-12-04T09:24:56.1548625Z + echo skip=false 2025-12-04T09:24:56.1550014Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2025-12-04T09:24:56.1556897Z ++ awk -F '[:,]' '{print $2}' 2025-12-04T09:24:56.1557677Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:24:56.1576350Z + DOCKER_TAG=pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:24:56.1579093Z + echo docker-tag=pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:24:56.1579947Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:24:56.1596787Z ##[group]Run set +e 2025-12-04T09:24:56.1597008Z set +e 2025-12-04T09:24:56.1597171Z set -x 2025-12-04T09:24:56.1597319Z  2025-12-04T09:24:56.1597468Z login() { 2025-12-04T09:24:56.1597792Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2025-12-04T09:24:56.1598113Z } 2025-12-04T09:24:56.1598249Z  2025-12-04T09:24:56.1598385Z retry () { 2025-12-04T09:24:56.1598560Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2025-12-04T09:24:56.1598748Z } 2025-12-04T09:24:56.1598885Z  2025-12-04T09:24:56.1599040Z retry login "${DOCKER_REGISTRY}" 2025-12-04T09:24:56.1599227Z  2025-12-04T09:24:56.1599377Z START_TIME=$(date +%s) 2025-12-04T09:24:56.1599648Z # Wait up to 120 minutes 2025-12-04T09:24:56.1599881Z while [[ $(( $(date +%s) - 7200 )) -lt $START_TIME ]]; do 2025-12-04T09:24:56.1600181Z  # Check if image already exists, if it does then skip building it 2025-12-04T09:24:56.1600480Z  if docker manifest inspect "${DOCKER_IMAGE}"; then 2025-12-04T09:24:56.1600707Z  exit 0 2025-12-04T09:24:56.1600855Z  fi 2025-12-04T09:24:56.1601000Z  2025-12-04T09:24:56.1601247Z  # NB: This flag is used by Docker build workflow to push the image to ECR, so we can 2025-12-04T09:24:56.1601632Z  # use this to differentiate between the Docker build and regular build jobs. For the 2025-12-04T09:24:56.1602022Z  # latter, it will wait for the Docker images to become available before continuing 2025-12-04T09:24:56.1602344Z  if [ "${DOCKER_PUSH:-false}" == "true" ]; then 2025-12-04T09:24:56.1602599Z  # It's a Docker build job, let's build the image 2025-12-04T09:24:56.1602813Z  break 2025-12-04T09:24:56.1602967Z  else 2025-12-04T09:24:56.1603186Z  # It's a regular build job, wait for the image to become available 2025-12-04T09:24:56.1603432Z  sleep 300 2025-12-04T09:24:56.1603597Z  fi 2025-12-04T09:24:56.1603745Z done 2025-12-04T09:24:56.1603888Z  2025-12-04T09:24:56.1604105Z # NB: This part requires a full checkout. Otherwise, the merge base will 2025-12-04T09:24:56.1604577Z # be empty. The default action would be to continue rebuild the image 2025-12-04T09:24:56.1604889Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2025-12-04T09:24:56.1605164Z  # if we're on the base branch then use the parent commit 2025-12-04T09:24:56.1605415Z  MERGE_BASE=$(git rev-parse HEAD~) 2025-12-04T09:24:56.1605617Z else 2025-12-04T09:24:56.1606104Z  # otherwise we're on a PR, so use the most recent base commit 2025-12-04T09:24:56.1606400Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2025-12-04T09:24:56.1606632Z fi 2025-12-04T09:24:56.1606785Z  2025-12-04T09:24:56.1606944Z if [[ -z "${MERGE_BASE}" ]]; then 2025-12-04T09:24:56.1607195Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2025-12-04T09:24:56.1607403Z  2025-12-04T09:24:56.1607681Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2025-12-04T09:24:56.1608005Z  exit 0 2025-12-04T09:24:56.1608158Z fi 2025-12-04T09:24:56.1608301Z  2025-12-04T09:24:56.1608490Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2025-12-04T09:24:56.1608884Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2025-12-04T09:24:56.1609215Z  exit 1 2025-12-04T09:24:56.1609355Z fi 2025-12-04T09:24:56.1609493Z  2025-12-04T09:24:56.1609717Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2025-12-04T09:24:56.1610097Z # If no image exists but the hash is the same as the previous hash then we should error out here 2025-12-04T09:24:56.1610428Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2025-12-04T09:24:56.1610827Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2025-12-04T09:24:56.1611257Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2025-12-04T09:24:56.1611523Z fi 2025-12-04T09:24:56.1611655Z  2025-12-04T09:24:56.1611820Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2025-12-04T09:24:56.1616124Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:24:56.1616339Z env: 2025-12-04T09:24:56.1616566Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:24:56.1616743Z DOCKER_BUILD_DIR: .ci/docker 2025-12-04T09:24:56.1616957Z BASE_REVISION: ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:24:56.1617505Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:24:56.1618182Z DOCKER_TAG: pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:24:56.1618602Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-12-04T09:24:56.1618828Z DOCKER_PUSH: 2025-12-04T09:24:56.1618972Z ##[endgroup] 2025-12-04T09:24:56.1641837Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-12-04T09:24:56.1643935Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-12-04T09:24:56.1644390Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-12-04T09:24:56.1650825Z + aws ecr get-login-password --region us-east-1 2025-12-04T09:24:56.5872163Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2025-12-04T09:24:56.5872534Z Login Succeeded 2025-12-04T09:24:56.5873114Z Configure a credential helper to remove this warning. See 2025-12-04T09:24:56.5873558Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2025-12-04T09:24:56.5873825Z 2025-12-04T09:24:56.5894235Z ++ date +%s 2025-12-04T09:24:56.5904419Z + START_TIME=1764840296 2025-12-04T09:24:56.5905370Z ++ date +%s 2025-12-04T09:24:56.5922175Z + [[ 1764833096 -lt 1764840296 ]] 2025-12-04T09:24:56.5927106Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:24:56.8024163Z { 2025-12-04T09:24:56.8025544Z "schemaVersion": 2, 2025-12-04T09:24:56.8025942Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2025-12-04T09:24:56.8026340Z "config": { 2025-12-04T09:24:56.8026595Z "mediaType": "application/vnd.docker.container.image.v1+json", 2025-12-04T09:24:56.8026885Z "size": 33581, 2025-12-04T09:24:56.8027184Z "digest": "sha256:40bd34edc3497e05e2150804a15313235b6426bcc4b8c19c03c3daf5524be913" 2025-12-04T09:24:56.8027474Z }, 2025-12-04T09:24:56.8030083Z "layers": [ 2025-12-04T09:24:56.8030309Z { 2025-12-04T09:24:56.8030590Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8030881Z "size": 30447951, 2025-12-04T09:24:56.8031234Z "digest": "sha256:63e5bc7682b85ae57a1221210f64d62e7a90b0a30f19af4ca734b8242ae49d63" 2025-12-04T09:24:56.8031557Z }, 2025-12-04T09:24:56.8031706Z { 2025-12-04T09:24:56.8031941Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8032299Z "size": 1554, 2025-12-04T09:24:56.8032613Z "digest": "sha256:6dc15eca51381c13be16385052dd2378ab1dce5fb77f7e7bceab34ed72e6e0e5" 2025-12-04T09:24:56.8033096Z }, 2025-12-04T09:24:56.8033258Z { 2025-12-04T09:24:56.8033488Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8033782Z "size": 313274856, 2025-12-04T09:24:56.8034091Z "digest": "sha256:459f4df18f07c473cb662e8033f9dab9f68c9ac71608767ad055999fae6b31e2" 2025-12-04T09:24:56.8034408Z }, 2025-12-04T09:24:56.8034556Z { 2025-12-04T09:24:56.8034794Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8035063Z "size": 787, 2025-12-04T09:24:56.8035348Z "digest": "sha256:82108541691974f1cc469d0ec89a048191b8de4dc36bf9a0184a4bd50120b69d" 2025-12-04T09:24:56.8035654Z }, 2025-12-04T09:24:56.8035786Z { 2025-12-04T09:24:56.8036003Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8036274Z "size": 106, 2025-12-04T09:24:56.8036548Z "digest": "sha256:3738646b3d92a3f1b6529ba588c00216c306c997ad77730f207c7107c2e5aff3" 2025-12-04T09:24:56.8036853Z }, 2025-12-04T09:24:56.8036988Z { 2025-12-04T09:24:56.8037583Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8037856Z "size": 704, 2025-12-04T09:24:56.8038134Z "digest": "sha256:aa9a10a37b3945f69568695fa4a9321333654a69dd44c55eff4e26bf9bffcad1" 2025-12-04T09:24:56.8038436Z }, 2025-12-04T09:24:56.8038566Z { 2025-12-04T09:24:56.8038785Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8039057Z "size": 1218, 2025-12-04T09:24:56.8039326Z "digest": "sha256:55bac08f3e18897f61d3e7f28ada6547b05b3c36f559a882656834d43de61478" 2025-12-04T09:24:56.8039635Z }, 2025-12-04T09:24:56.8039772Z { 2025-12-04T09:24:56.8039984Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8040308Z "size": 485, 2025-12-04T09:24:56.8040590Z "digest": "sha256:acf6468f6aadc3613e3b8fe73c016ce8d6f47f01e0cfee3a1015d0335c24d691" 2025-12-04T09:24:56.8040915Z }, 2025-12-04T09:24:56.8041043Z { 2025-12-04T09:24:56.8041257Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8041530Z "size": 110362453, 2025-12-04T09:24:56.8041820Z "digest": "sha256:c0c31a0e69b798875a8d2c05d01a6861b2fe623e1970bf7faf822e7df6691d2c" 2025-12-04T09:24:56.8042130Z }, 2025-12-04T09:24:56.8042358Z { 2025-12-04T09:24:56.8042609Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8042875Z "size": 4961, 2025-12-04T09:24:56.8043155Z "digest": "sha256:0f7d0dc70d2d944cc1c8a592f051d3f28d2b062c79166713fb7cc9225b4c97c0" 2025-12-04T09:24:56.8043472Z }, 2025-12-04T09:24:56.8043732Z { 2025-12-04T09:24:56.8043943Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8044198Z "size": 1756, 2025-12-04T09:24:56.8044466Z "digest": "sha256:58c4d15d3bc18a789eb0d1a13f68df0b1d6cea66697fe63adf11b53fec74ba18" 2025-12-04T09:24:56.8044758Z }, 2025-12-04T09:24:56.8044891Z { 2025-12-04T09:24:56.8045139Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8045403Z "size": 724, 2025-12-04T09:24:56.8045658Z "digest": "sha256:2f1cc47b61e1ebefa16f8ee12c5a1df77b109a84a8e8f425e7d65ec51cafdb93" 2025-12-04T09:24:56.8045952Z }, 2025-12-04T09:24:56.8046081Z { 2025-12-04T09:24:56.8046286Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8046534Z "size": 543, 2025-12-04T09:24:56.8046792Z "digest": "sha256:d2472a74103d877e6532d6409a2ac9d54db6672b5d3002be011fdbbd8d360c49" 2025-12-04T09:24:56.8047078Z }, 2025-12-04T09:24:56.8047203Z { 2025-12-04T09:24:56.8047415Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8047676Z "size": 3353192867, 2025-12-04T09:24:56.8047945Z "digest": "sha256:d1fba72936888ea8f8e4c7ad015bfcf3285017b95f3d72795d5b6a0b0ded36fc" 2025-12-04T09:24:56.8048239Z }, 2025-12-04T09:24:56.8048373Z { 2025-12-04T09:24:56.8048574Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8048829Z "size": 32, 2025-12-04T09:24:56.8049098Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:24:56.8049390Z }, 2025-12-04T09:24:56.8049513Z { 2025-12-04T09:24:56.8049719Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8049973Z "size": 397, 2025-12-04T09:24:56.8050228Z "digest": "sha256:6a9b03ce41a77be16366fba2c6ca0aabefe42ee4879ab5d088b312970e830770" 2025-12-04T09:24:56.8050519Z }, 2025-12-04T09:24:56.8050646Z { 2025-12-04T09:24:56.8071106Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8071409Z "size": 237420, 2025-12-04T09:24:56.8071684Z "digest": "sha256:3519ac15be7978a14201f5d431b6472ed30382cde8628c6401c9db98ccd1a871" 2025-12-04T09:24:56.8071976Z }, 2025-12-04T09:24:56.8072115Z { 2025-12-04T09:24:56.8072337Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8072592Z "size": 231, 2025-12-04T09:24:56.8072867Z "digest": "sha256:dabb51b819a06823e5a1c8d4f9e29b689f38b19ee1e90da40075bd1593b8d0fb" 2025-12-04T09:24:56.8073390Z }, 2025-12-04T09:24:56.8073532Z { 2025-12-04T09:24:56.8073771Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8074059Z "size": 2967944, 2025-12-04T09:24:56.8074358Z "digest": "sha256:8f36833a24d02d47bf18bdc573adbb45afb8f5f06886da9bd671a1a33e3007bd" 2025-12-04T09:24:56.8074678Z }, 2025-12-04T09:24:56.8074830Z { 2025-12-04T09:24:56.8075052Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8075315Z "size": 1472, 2025-12-04T09:24:56.8075593Z "digest": "sha256:ab53c5a853a52de13f11fbb026aaf469754adbe52b02e96f4fbdc05916585244" 2025-12-04T09:24:56.8075888Z }, 2025-12-04T09:24:56.8076013Z { 2025-12-04T09:24:56.8076220Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8076474Z "size": 482, 2025-12-04T09:24:56.8076740Z "digest": "sha256:025a0e5e6ac19cb2d27c9423a5d019a34ea7d4004d426d6b890693ccfe9f54d3" 2025-12-04T09:24:56.8077020Z }, 2025-12-04T09:24:56.8077150Z { 2025-12-04T09:24:56.8077346Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8077604Z "size": 200, 2025-12-04T09:24:56.8077871Z "digest": "sha256:ce3394c8f2109f4538f6e7f2c39f685d0ba028aacac33ddacbb8ef9fd349702e" 2025-12-04T09:24:56.8078156Z }, 2025-12-04T09:24:56.8078284Z { 2025-12-04T09:24:56.8078490Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8078734Z "size": 608, 2025-12-04T09:24:56.8079057Z "digest": "sha256:a5c3888c3a0c5812efbfe307d18de826363900fa8d8097c1ee84a72630aa067b" 2025-12-04T09:24:56.8079341Z }, 2025-12-04T09:24:56.8079470Z { 2025-12-04T09:24:56.8079666Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8079915Z "size": 225, 2025-12-04T09:24:56.8080169Z "digest": "sha256:ed902a3a4e3b916aa869b612baf9cd36745b671b6781d19e3b4c3fadc2f513f5" 2025-12-04T09:24:56.8080449Z }, 2025-12-04T09:24:56.8080578Z { 2025-12-04T09:24:56.8080784Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8081025Z "size": 829, 2025-12-04T09:24:56.8081284Z "digest": "sha256:b4e1efca22beb475459a14f4a571a2512c2b1ed4b24e66bd3f82f1681dc5b1bc" 2025-12-04T09:24:56.8081570Z }, 2025-12-04T09:24:56.8081693Z { 2025-12-04T09:24:56.8081898Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8082145Z "size": 32, 2025-12-04T09:24:56.8082393Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:24:56.8082675Z }, 2025-12-04T09:24:56.8082799Z { 2025-12-04T09:24:56.8083003Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8083243Z "size": 104, 2025-12-04T09:24:56.8083496Z "digest": "sha256:79fcfd297d9fcfe2810595fc2c6ba503293b59046bc192bb0620a64d9bdff778" 2025-12-04T09:24:56.8083783Z }, 2025-12-04T09:24:56.8083903Z { 2025-12-04T09:24:56.8084103Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8084348Z "size": 1495, 2025-12-04T09:24:56.8084598Z "digest": "sha256:7bd89c134b49f83cfb0725df3615556b60af380b932186c5abd001232b4597cf" 2025-12-04T09:24:56.8084877Z }, 2025-12-04T09:24:56.8085006Z { 2025-12-04T09:24:56.8085197Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8085447Z "size": 458790783, 2025-12-04T09:24:56.8085724Z "digest": "sha256:b7d40b4fd1b9375cfd45d5c593ec0cccc21c80d667ea79f4effb09f04b0705b0" 2025-12-04T09:24:56.8086009Z }, 2025-12-04T09:24:56.8086129Z { 2025-12-04T09:24:56.8086326Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8086573Z "size": 163, 2025-12-04T09:24:56.8086822Z "digest": "sha256:dd1cdd87320d29d4ca09686ab00b76a396efcc6f3ea6d0bfd1f7922e46336ca6" 2025-12-04T09:24:56.8087103Z }, 2025-12-04T09:24:56.8087230Z { 2025-12-04T09:24:56.8087502Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8087752Z "size": 347, 2025-12-04T09:24:56.8088013Z "digest": "sha256:c21a6e1cd03a718ebfb927ab114ebbbee0b57a93908d0265d24b02cf34c0fee3" 2025-12-04T09:24:56.8088292Z }, 2025-12-04T09:24:56.8088422Z { 2025-12-04T09:24:56.8088627Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8088872Z "size": 32, 2025-12-04T09:24:56.8089131Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:24:56.8089421Z }, 2025-12-04T09:24:56.8089550Z { 2025-12-04T09:24:56.8089750Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8090004Z "size": 106, 2025-12-04T09:24:56.8090259Z "digest": "sha256:0aa3ea0b275426eac8eff6b26ba17c168ae70d1baa5a7845e61d41bd01a9ff43" 2025-12-04T09:24:56.8090538Z }, 2025-12-04T09:24:56.8090668Z { 2025-12-04T09:24:56.8090873Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8091118Z "size": 426, 2025-12-04T09:24:56.8091374Z "digest": "sha256:74e0bdbb05d31425d7a2f40ff0cb2423abaf90f1ed56c4b69f43e8d92c569379" 2025-12-04T09:24:56.8091662Z }, 2025-12-04T09:24:56.8091784Z { 2025-12-04T09:24:56.8091987Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8092240Z "size": 19309400, 2025-12-04T09:24:56.8092506Z "digest": "sha256:ba3aee7dcf03efbd1e7b1611c8f65f385c94b2572d15c099a1535f27b72509f7" 2025-12-04T09:24:56.8092805Z }, 2025-12-04T09:24:56.8092993Z { 2025-12-04T09:24:56.8093207Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8093455Z "size": 108, 2025-12-04T09:24:56.8093718Z "digest": "sha256:8c8d59b8759d40399b07b019a46fc05b9ccabe2b9276f49eea203fd6908ff334" 2025-12-04T09:24:56.8094007Z }, 2025-12-04T09:24:56.8094132Z { 2025-12-04T09:24:56.8094352Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8094608Z "size": 827, 2025-12-04T09:24:56.8094854Z "digest": "sha256:3c960865867b66327e316bf682adabd1867dd9a41cc0aee1507ee8e35fa614a9" 2025-12-04T09:24:56.8095133Z }, 2025-12-04T09:24:56.8095260Z { 2025-12-04T09:24:56.8095456Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8095716Z "size": 724, 2025-12-04T09:24:56.8095982Z "digest": "sha256:2f1cc47b61e1ebefa16f8ee12c5a1df77b109a84a8e8f425e7d65ec51cafdb93" 2025-12-04T09:24:56.8096278Z }, 2025-12-04T09:24:56.8096404Z { 2025-12-04T09:24:56.8096617Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8096872Z "size": 149, 2025-12-04T09:24:56.8097123Z "digest": "sha256:074db79e3832af094ddad6e5c5649347b62e8ba0365229feced3ed0263a0c611" 2025-12-04T09:24:56.8097418Z }, 2025-12-04T09:24:56.8097546Z { 2025-12-04T09:24:56.8097749Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8098004Z "size": 137, 2025-12-04T09:24:56.8098264Z "digest": "sha256:226748f4ff23ba2aef33619c3a639fb44865d5f4515c4b45d4196f4e0e77b187" 2025-12-04T09:24:56.8098556Z }, 2025-12-04T09:24:56.8098682Z { 2025-12-04T09:24:56.8098884Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8099213Z + exit 0 2025-12-04T09:24:56.8099358Z "size": 140, 2025-12-04T09:24:56.8099611Z "digest": "sha256:75d817336f1df7579f5b4be29625c53d594cb78c776f5454cc69438bb1bb0dc9" 2025-12-04T09:24:56.8099897Z }, 2025-12-04T09:24:56.8100029Z { 2025-12-04T09:24:56.8100238Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8100506Z "size": 18890247472, 2025-12-04T09:24:56.8100782Z "digest": "sha256:850870a17b94a4d7c5e55b287cc0b1ddd0982845e11f055765dc358277cdfc54" 2025-12-04T09:24:56.8101070Z }, 2025-12-04T09:24:56.8101196Z { 2025-12-04T09:24:56.8101409Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8101673Z "size": 222, 2025-12-04T09:24:56.8101966Z "digest": "sha256:e6225129924f0a66acaf35efd792901351da58116629962669d9c3acaadbab6e" 2025-12-04T09:24:56.8102255Z }, 2025-12-04T09:24:56.8102384Z { 2025-12-04T09:24:56.8102579Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8102826Z "size": 255, 2025-12-04T09:24:56.8103084Z "digest": "sha256:5cd6b9ec5c3c4a1cf2e2e7a1794aaae173eb5a291b9899bb733e5f36d866fab0" 2025-12-04T09:24:56.8103359Z }, 2025-12-04T09:24:56.8103558Z { 2025-12-04T09:24:56.8103772Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8104025Z "size": 32, 2025-12-04T09:24:56.8104295Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:24:56.8104582Z }, 2025-12-04T09:24:56.8104714Z { 2025-12-04T09:24:56.8104912Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8105182Z "size": 106, 2025-12-04T09:24:56.8105458Z "digest": "sha256:613d2b1f3db875d1d695081b44aefbf94eeb7815d794b28dd17ec623d3f9dad5" 2025-12-04T09:24:56.8106034Z }, 2025-12-04T09:24:56.8106182Z { 2025-12-04T09:24:56.8106403Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8106668Z "size": 312293443, 2025-12-04T09:24:56.8106958Z "digest": "sha256:388169fffe8a06396f4e8d6fc51d07a83e08f04836995ae28c2e13960cb89fab" 2025-12-04T09:24:56.8107261Z }, 2025-12-04T09:24:56.8107395Z { 2025-12-04T09:24:56.8107619Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8108052Z "size": 6628443345, 2025-12-04T09:24:56.8108345Z "digest": "sha256:6ca498e78b7b821a815bc7c14f057d52f7d9ce8dda8bca0945e6f522c7d208e1" 2025-12-04T09:24:56.8108653Z }, 2025-12-04T09:24:56.8108792Z { 2025-12-04T09:24:56.8109012Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8109274Z "size": 129, 2025-12-04T09:24:56.8109557Z "digest": "sha256:952b753ec7be39e1967f95afb87a880712b6425610adcda2afe3b2211cc8de12" 2025-12-04T09:24:56.8109874Z }, 2025-12-04T09:24:56.8110019Z { 2025-12-04T09:24:56.8110238Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8110507Z "size": 880, 2025-12-04T09:24:56.8110781Z "digest": "sha256:5b6f6de4bdd1c96aa8704c326fb27fa7465ebf388f0748ee56d9f521bb6b6697" 2025-12-04T09:24:56.8111101Z }, 2025-12-04T09:24:56.8111243Z { 2025-12-04T09:24:56.8111460Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8111738Z "size": 724, 2025-12-04T09:24:56.8112034Z "digest": "sha256:2f1cc47b61e1ebefa16f8ee12c5a1df77b109a84a8e8f425e7d65ec51cafdb93" 2025-12-04T09:24:56.8112357Z }, 2025-12-04T09:24:56.8112491Z { 2025-12-04T09:24:56.8112727Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8113061Z "size": 139, 2025-12-04T09:24:56.8113339Z "digest": "sha256:fa23d9952f3cea91af7b5ffce93e64d90d98aefb838aa8a1fe4a6b40cd0eab91" 2025-12-04T09:24:56.8113657Z }, 2025-12-04T09:24:56.8113800Z { 2025-12-04T09:24:56.8114013Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8114286Z "size": 32, 2025-12-04T09:24:56.8114568Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:24:56.8114868Z }, 2025-12-04T09:24:56.8115012Z { 2025-12-04T09:24:56.8115234Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8115497Z "size": 161, 2025-12-04T09:24:56.8115797Z "digest": "sha256:c2aa9ea4d09ac3edb41e48cdc892fafe72a5cddc98ebdf67d978c6f8d63cd7d2" 2025-12-04T09:24:56.8116127Z }, 2025-12-04T09:24:56.8116266Z { 2025-12-04T09:24:56.8116477Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8116755Z "size": 1011, 2025-12-04T09:24:56.8117038Z "digest": "sha256:a9bbc9e426d367e15c7f0c1faae6417526e219fa8188ae17b9e6c2c3b8083bdc" 2025-12-04T09:24:56.8117339Z }, 2025-12-04T09:24:56.8117478Z { 2025-12-04T09:24:56.8117773Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8118043Z "size": 724, 2025-12-04T09:24:56.8118329Z "digest": "sha256:2f1cc47b61e1ebefa16f8ee12c5a1df77b109a84a8e8f425e7d65ec51cafdb93" 2025-12-04T09:24:56.8118640Z }, 2025-12-04T09:24:56.8118772Z { 2025-12-04T09:24:56.8119000Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8119276Z "size": 135, 2025-12-04T09:24:56.8119555Z "digest": "sha256:82f2371f6dc2c2797baa60e672fd78bb04909cd0248c52cae058a7fd8f215a4f" 2025-12-04T09:24:56.8119858Z }, 2025-12-04T09:24:56.8119998Z { 2025-12-04T09:24:56.8120222Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8120487Z "size": 32, 2025-12-04T09:24:56.8120765Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:24:56.8121072Z }, 2025-12-04T09:24:56.8121201Z { 2025-12-04T09:24:56.8121423Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8121697Z "size": 158, 2025-12-04T09:24:56.8121968Z "digest": "sha256:c87301921afac5ba5745add04b44f57bf31c551bf48a2574d4424dab64c25ce2" 2025-12-04T09:24:56.8122258Z }, 2025-12-04T09:24:56.8122387Z { 2025-12-04T09:24:56.8122585Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8122841Z "size": 603, 2025-12-04T09:24:56.8123110Z "digest": "sha256:384d566a822f4defa0ecf9c7d37b6d7a5fc5eaab415fe1d3cc02b190ddc19e71" 2025-12-04T09:24:56.8123400Z }, 2025-12-04T09:24:56.8123566Z { 2025-12-04T09:24:56.8123784Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8124041Z "size": 724, 2025-12-04T09:24:56.8124300Z "digest": "sha256:2f1cc47b61e1ebefa16f8ee12c5a1df77b109a84a8e8f425e7d65ec51cafdb93" 2025-12-04T09:24:56.8124593Z }, 2025-12-04T09:24:56.8124724Z { 2025-12-04T09:24:56.8124926Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8125189Z "size": 155, 2025-12-04T09:24:56.8125452Z "digest": "sha256:32ddcdd5dd6920acb12ca95671a164346308ec7743afbe8a333142733c42bafd" 2025-12-04T09:24:56.8125735Z }, 2025-12-04T09:24:56.8125867Z { 2025-12-04T09:24:56.8126084Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8126348Z "size": 32, 2025-12-04T09:24:56.8126601Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:24:56.8126895Z }, 2025-12-04T09:24:56.8127028Z { 2025-12-04T09:24:56.8127235Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8127496Z "size": 188, 2025-12-04T09:24:56.8127759Z "digest": "sha256:a116a32ceaa15bbd55b0f4281fb0040de221c62e29659b23dd71ecd2382a969a" 2025-12-04T09:24:56.8128044Z }, 2025-12-04T09:24:56.8128179Z { 2025-12-04T09:24:56.8128389Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8128643Z "size": 1371, 2025-12-04T09:24:56.8128921Z "digest": "sha256:f6fbbbe6067e383cadb530c3f53b113e9826843fb2348ea1f9a000a4790fa822" 2025-12-04T09:24:56.8129210Z }, 2025-12-04T09:24:56.8129335Z { 2025-12-04T09:24:56.8129547Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8129802Z "size": 32, 2025-12-04T09:24:56.8130066Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:24:56.8130348Z }, 2025-12-04T09:24:56.8130481Z { 2025-12-04T09:24:56.8130699Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8130948Z "size": 137, 2025-12-04T09:24:56.8131204Z "digest": "sha256:648018658875ac3356b277abe3d98df9a8b7a0345f131598c07f5e9f9318b740" 2025-12-04T09:24:56.8131484Z }, 2025-12-04T09:24:56.8131608Z { 2025-12-04T09:24:56.8131818Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8132072Z "size": 528, 2025-12-04T09:24:56.8132340Z "digest": "sha256:5e2f7eac20ad5128eebe0ba7dbd08111d28e7f65c26c7fea7cd6dc7a2c0725b9" 2025-12-04T09:24:56.8132674Z }, 2025-12-04T09:24:56.8132808Z { 2025-12-04T09:24:56.8133008Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8133263Z "size": 32, 2025-12-04T09:24:56.8133526Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:24:56.8133807Z }, 2025-12-04T09:24:56.8133930Z { 2025-12-04T09:24:56.8134136Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8134391Z "size": 104, 2025-12-04T09:24:56.8134639Z "digest": "sha256:0d1243af7593a042ff443492a4cac70f817dc8c56ac45a0c138d102f40c1cc07" 2025-12-04T09:24:56.8134923Z }, 2025-12-04T09:24:56.8135055Z { 2025-12-04T09:24:56.8135256Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8135514Z "size": 435, 2025-12-04T09:24:56.8135776Z "digest": "sha256:1f6d5b941ea1f6f07c190f5ca1d4a5172e4d696ad597ae58dcd37b297f60b556" 2025-12-04T09:24:56.8136062Z }, 2025-12-04T09:24:56.8136192Z { 2025-12-04T09:24:56.8136416Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8136673Z "size": 32, 2025-12-04T09:24:56.8137210Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:24:56.8137499Z }, 2025-12-04T09:24:56.8137633Z { 2025-12-04T09:24:56.8137838Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8138093Z "size": 107, 2025-12-04T09:24:56.8138434Z "digest": "sha256:4444f70d73cefec9f161242e9def76078af5abc7a3ad3e5d6e7d4f1efed07939" 2025-12-04T09:24:56.8138725Z }, 2025-12-04T09:24:56.8138858Z { 2025-12-04T09:24:56.8139066Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8139317Z "size": 1896, 2025-12-04T09:24:56.8139578Z "digest": "sha256:bf45550ddbad2524852930466f963e432429f2b86f7f6ccf0415fc56980004ce" 2025-12-04T09:24:56.8139869Z }, 2025-12-04T09:24:56.8139996Z { 2025-12-04T09:24:56.8140208Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8140471Z "size": 245588924, 2025-12-04T09:24:56.8140747Z "digest": "sha256:30fff4f4bad325a3a2d91463ca6702c411392e5b0646a5cd4f47fcae56c55639" 2025-12-04T09:24:56.8141028Z }, 2025-12-04T09:24:56.8141162Z { 2025-12-04T09:24:56.8141376Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8141619Z "size": 106, 2025-12-04T09:24:56.8141889Z "digest": "sha256:cca4db1ba155a3afd1fd645cd3223e9a4de9c785f8559387f21da187672d9e9e" 2025-12-04T09:24:56.8142374Z }, 2025-12-04T09:24:56.8142504Z { 2025-12-04T09:24:56.8142715Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8142968Z "size": 165, 2025-12-04T09:24:56.8143224Z "digest": "sha256:f0ffa379f4eba2f3ddc323f45f226f39aed5a7e93880884a97bd4d6a91864dc1" 2025-12-04T09:24:56.8143515Z }, 2025-12-04T09:24:56.8143648Z { 2025-12-04T09:24:56.8143855Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8144114Z "size": 7942, 2025-12-04T09:24:56.8144379Z "digest": "sha256:df645e678f95db120f96e4c4c66400d315ed9214d9085216b417348cf682d43a" 2025-12-04T09:24:56.8144666Z }, 2025-12-04T09:24:56.8144794Z { 2025-12-04T09:24:56.8145002Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8145257Z "size": 8076, 2025-12-04T09:24:56.8145515Z "digest": "sha256:52ad6c17d10308710fb2dc855f911246b7a369ccf5c3142cb3ad7fb182f708b8" 2025-12-04T09:24:56.8145801Z }, 2025-12-04T09:24:56.8145929Z { 2025-12-04T09:24:56.8146128Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8146376Z "size": 302, 2025-12-04T09:24:56.8146647Z "digest": "sha256:34408374e32c70dbcfd7e0516edaa8698c25e6e3dbcd920978f2b3a801bbe659" 2025-12-04T09:24:56.8146923Z }, 2025-12-04T09:24:56.8147055Z { 2025-12-04T09:24:56.8147268Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8147575Z "size": 32, 2025-12-04T09:24:56.8147830Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:24:56.8148130Z }, 2025-12-04T09:24:56.8148269Z { 2025-12-04T09:24:56.8148480Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8148745Z "size": 108, 2025-12-04T09:24:56.8149024Z "digest": "sha256:27803b661d9e2cb1ce49c0a100ce824fbcbdd1f3a3ceda4ed5affbcb07ac3be4" 2025-12-04T09:24:56.8149328Z }, 2025-12-04T09:24:56.8149470Z { 2025-12-04T09:24:56.8149694Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8149960Z "size": 54145699, 2025-12-04T09:24:56.8150258Z "digest": "sha256:07dafc893dea952d7677124b42e90b29b167fabe81e8a9d8b3fb4b0aa11b66ec" 2025-12-04T09:24:56.8150564Z }, 2025-12-04T09:24:56.8150696Z { 2025-12-04T09:24:56.8150923Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:24:56.8151201Z "size": 32, 2025-12-04T09:24:56.8151479Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:24:56.8151776Z } 2025-12-04T09:24:56.8151919Z ] 2025-12-04T09:24:56.8152062Z } 2025-12-04T09:24:56.8177325Z ##[group]Run set -eux 2025-12-04T09:24:56.8177550Z set -eux 2025-12-04T09:24:56.8177830Z # It's ok if this steps fails, it would then be an anonymous user like what we used to have 2025-12-04T09:24:56.8178633Z aws secretsmanager get-secret-value --secret-id docker_hub_readonly_token | jq --raw-output '.SecretString' | jq -r .docker_hub_readonly_token | docker login --username pytorchbot --password-stdin || true 2025-12-04T09:24:56.8183629Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:24:56.8183880Z env: 2025-12-04T09:24:56.8184039Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:24:56.8184217Z ##[endgroup] 2025-12-04T09:24:56.8213923Z + aws secretsmanager get-secret-value --secret-id docker_hub_readonly_token 2025-12-04T09:24:56.8218615Z + jq --raw-output .SecretString 2025-12-04T09:24:56.8219102Z + docker login --username pytorchbot --password-stdin 2025-12-04T09:24:56.8219519Z + jq -r .docker_hub_readonly_token 2025-12-04T09:24:57.3104313Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2025-12-04T09:24:57.3104927Z Configure a credential helper to remove this warning. See 2025-12-04T09:24:57.3108132Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2025-12-04T09:24:57.3108419Z 2025-12-04T09:24:57.3108490Z Login Succeeded 2025-12-04T09:24:57.3202873Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*:} 2025-12-04T09:24:57.3203169Z tag=${ECR_DOCKER_IMAGE##*:} 2025-12-04T09:24:57.3203444Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2025-12-04T09:24:57.3208229Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:24:57.3208477Z env: 2025-12-04T09:24:57.3208649Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:24:57.3209203Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:24:57.3209759Z ##[endgroup] 2025-12-04T09:24:57.3233452Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:24:57.3269462Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2025-12-04T09:24:57.3269756Z with: 2025-12-04T09:24:57.3270309Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:24:57.3270967Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-12-04T09:24:57.3271234Z env: 2025-12-04T09:24:57.3271393Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:24:57.3271588Z ##[endgroup] 2025-12-04T09:24:57.3282403Z ##[group]Run set -x 2025-12-04T09:24:57.3282718Z set -x 2025-12-04T09:24:57.3282872Z set +e 2025-12-04T09:24:57.3283025Z  2025-12-04T09:24:57.3283180Z login() { 2025-12-04T09:24:57.3283495Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2025-12-04T09:24:57.3283821Z } 2025-12-04T09:24:57.3283975Z  2025-12-04T09:24:57.3284155Z retry () { 2025-12-04T09:24:57.3284344Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2025-12-04T09:24:57.3284556Z } 2025-12-04T09:24:57.3284703Z  2025-12-04T09:24:57.3284868Z retry login "${DOCKER_REGISTRY}" 2025-12-04T09:24:57.3285068Z  2025-12-04T09:24:57.3285393Z IMAGE_SIZE=$(docker manifest inspect "${DOCKER_IMAGE}" | jq '[.layers[].size, .config.size] | add / 1024 / 1024') 2025-12-04T09:24:57.3285810Z echo "Compressed size of image in MB: ${IMAGE_SIZE}" 2025-12-04T09:24:57.3286048Z  2025-12-04T09:24:57.3286205Z set -e 2025-12-04T09:24:57.3286444Z # ignore output since only exit code is used for conditional 2025-12-04T09:24:57.3286764Z # only pull docker image if it's not available locally 2025-12-04T09:24:57.3287104Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2025-12-04T09:24:57.3287434Z  retry docker pull "${DOCKER_IMAGE}" 2025-12-04T09:24:57.3287653Z fi 2025-12-04T09:24:57.3292397Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:24:57.3292688Z env: 2025-12-04T09:24:57.3292876Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:24:57.3293530Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:24:57.3294271Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-12-04T09:24:57.3294573Z ##[endgroup] 2025-12-04T09:24:57.3318160Z + set +e 2025-12-04T09:24:57.3318598Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-12-04T09:24:57.3318935Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-12-04T09:24:57.3324904Z + aws ecr get-login-password --region us-east-1 2025-12-04T09:24:57.3325438Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-12-04T09:24:57.7657382Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2025-12-04T09:24:57.7657795Z Login Succeeded 2025-12-04T09:24:57.7663406Z Configure a credential helper to remove this warning. See 2025-12-04T09:24:57.7667978Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2025-12-04T09:24:57.7669768Z 2025-12-04T09:24:57.7684147Z ++ docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:24:57.7684914Z ++ jq '[.layers[].size, .config.size] | add / 1024 / 1024' 2025-12-04T09:24:58.0073564Z Compressed size of image in MB: 29010.187264442444 2025-12-04T09:24:58.0073911Z + IMAGE_SIZE=29010.187264442444 2025-12-04T09:24:58.0074170Z + echo 'Compressed size of image in MB: 29010.187264442444' 2025-12-04T09:24:58.0074521Z + set -e 2025-12-04T09:24:58.0075434Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:24:58.0208453Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:24:58.0209385Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:24:58.2261287Z pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a: Pulling from pytorch/ci-image 2025-12-04T09:24:58.2265250Z 63e5bc7682b8: Pulling fs layer 2025-12-04T09:24:58.2266063Z 6dc15eca5138: Pulling fs layer 2025-12-04T09:24:58.2266325Z 459f4df18f07: Pulling fs layer 2025-12-04T09:24:58.2266509Z 821085416919: Pulling fs layer 2025-12-04T09:24:58.2266694Z 3738646b3d92: Pulling fs layer 2025-12-04T09:24:58.2266912Z aa9a10a37b39: Pulling fs layer 2025-12-04T09:24:58.2267120Z 55bac08f3e18: Pulling fs layer 2025-12-04T09:24:58.2267357Z acf6468f6aad: Pulling fs layer 2025-12-04T09:24:58.2267527Z c0c31a0e69b7: Pulling fs layer 2025-12-04T09:24:58.2267710Z 0f7d0dc70d2d: Pulling fs layer 2025-12-04T09:24:58.2267886Z 58c4d15d3bc1: Pulling fs layer 2025-12-04T09:24:58.2268056Z 2f1cc47b61e1: Pulling fs layer 2025-12-04T09:24:58.2268231Z d2472a74103d: Pulling fs layer 2025-12-04T09:24:58.2268407Z d1fba7293688: Pulling fs layer 2025-12-04T09:24:58.2268574Z 4f4fb700ef54: Pulling fs layer 2025-12-04T09:24:58.2268751Z 6a9b03ce41a7: Pulling fs layer 2025-12-04T09:24:58.2268942Z 3519ac15be79: Pulling fs layer 2025-12-04T09:24:58.2269115Z dabb51b819a0: Pulling fs layer 2025-12-04T09:24:58.2269293Z 8f36833a24d0: Pulling fs layer 2025-12-04T09:24:58.2269473Z ab53c5a853a5: Pulling fs layer 2025-12-04T09:24:58.2269646Z 025a0e5e6ac1: Pulling fs layer 2025-12-04T09:24:58.2269825Z ce3394c8f210: Pulling fs layer 2025-12-04T09:24:58.2270007Z a5c3888c3a0c: Pulling fs layer 2025-12-04T09:24:58.2270192Z ed902a3a4e3b: Pulling fs layer 2025-12-04T09:24:58.2270371Z b4e1efca22be: Pulling fs layer 2025-12-04T09:24:58.2270558Z 79fcfd297d9f: Pulling fs layer 2025-12-04T09:24:58.2270739Z 7bd89c134b49: Pulling fs layer 2025-12-04T09:24:58.2270914Z b7d40b4fd1b9: Pulling fs layer 2025-12-04T09:24:58.2271095Z dd1cdd87320d: Pulling fs layer 2025-12-04T09:24:58.2271276Z c21a6e1cd03a: Pulling fs layer 2025-12-04T09:24:58.2271450Z 0aa3ea0b2754: Pulling fs layer 2025-12-04T09:24:58.2271633Z 74e0bdbb05d3: Pulling fs layer 2025-12-04T09:24:58.2271814Z ba3aee7dcf03: Pulling fs layer 2025-12-04T09:24:58.2271992Z 8c8d59b8759d: Pulling fs layer 2025-12-04T09:24:58.2272168Z 3c960865867b: Pulling fs layer 2025-12-04T09:24:58.2272345Z 074db79e3832: Pulling fs layer 2025-12-04T09:24:58.2272516Z 226748f4ff23: Pulling fs layer 2025-12-04T09:24:58.2272697Z 75d817336f1d: Pulling fs layer 2025-12-04T09:24:58.2272889Z 850870a17b94: Pulling fs layer 2025-12-04T09:24:58.2273242Z e6225129924f: Pulling fs layer 2025-12-04T09:24:58.2273444Z 5cd6b9ec5c3c: Pulling fs layer 2025-12-04T09:24:58.2273642Z 613d2b1f3db8: Pulling fs layer 2025-12-04T09:24:58.2273837Z 388169fffe8a: Pulling fs layer 2025-12-04T09:24:58.2274026Z 6ca498e78b7b: Pulling fs layer 2025-12-04T09:24:58.2274234Z 952b753ec7be: Pulling fs layer 2025-12-04T09:24:58.2274418Z 5b6f6de4bdd1: Pulling fs layer 2025-12-04T09:24:58.2274592Z fa23d9952f3c: Pulling fs layer 2025-12-04T09:24:58.2274776Z c2aa9ea4d09a: Pulling fs layer 2025-12-04T09:24:58.2274959Z a9bbc9e426d3: Pulling fs layer 2025-12-04T09:24:58.2275132Z 82f2371f6dc2: Pulling fs layer 2025-12-04T09:24:58.2275315Z c87301921afa: Pulling fs layer 2025-12-04T09:24:58.2275496Z 384d566a822f: Pulling fs layer 2025-12-04T09:24:58.2275669Z 32ddcdd5dd69: Pulling fs layer 2025-12-04T09:24:58.2275849Z a116a32ceaa1: Pulling fs layer 2025-12-04T09:24:58.2276030Z f6fbbbe6067e: Pulling fs layer 2025-12-04T09:24:58.2276203Z 648018658875: Pulling fs layer 2025-12-04T09:24:58.2276381Z 5e2f7eac20ad: Pulling fs layer 2025-12-04T09:24:58.2276806Z 0d1243af7593: Pulling fs layer 2025-12-04T09:24:58.2276991Z 1f6d5b941ea1: Pulling fs layer 2025-12-04T09:24:58.2277165Z 4444f70d73ce: Pulling fs layer 2025-12-04T09:24:58.2277356Z bf45550ddbad: Pulling fs layer 2025-12-04T09:24:58.2277543Z 30fff4f4bad3: Pulling fs layer 2025-12-04T09:24:58.2277714Z 3738646b3d92: Waiting 2025-12-04T09:24:58.2277889Z cca4db1ba155: Pulling fs layer 2025-12-04T09:24:58.2278070Z aa9a10a37b39: Waiting 2025-12-04T09:24:58.2278235Z f0ffa379f4eb: Pulling fs layer 2025-12-04T09:24:58.2278925Z df645e678f95: Pulling fs layer 2025-12-04T09:24:58.2279115Z 52ad6c17d103: Pulling fs layer 2025-12-04T09:24:58.2279355Z 34408374e32c: Pulling fs layer 2025-12-04T09:24:58.2279532Z 55bac08f3e18: Waiting 2025-12-04T09:24:58.2279705Z 27803b661d9e: Pulling fs layer 2025-12-04T09:24:58.2279875Z acf6468f6aad: Waiting 2025-12-04T09:24:58.2280050Z 07dafc893dea: Pulling fs layer 2025-12-04T09:24:58.2280231Z c0c31a0e69b7: Waiting 2025-12-04T09:24:58.2280383Z d2472a74103d: Waiting 2025-12-04T09:24:58.2280556Z 0f7d0dc70d2d: Waiting 2025-12-04T09:24:58.2280715Z 75d817336f1d: Waiting 2025-12-04T09:24:58.2280875Z 58c4d15d3bc1: Waiting 2025-12-04T09:24:58.2281028Z d1fba7293688: Waiting 2025-12-04T09:24:58.2281185Z 850870a17b94: Waiting 2025-12-04T09:24:58.2281347Z 4f4fb700ef54: Waiting 2025-12-04T09:24:58.2281497Z 6a9b03ce41a7: Waiting 2025-12-04T09:24:58.2281656Z 2f1cc47b61e1: Waiting 2025-12-04T09:24:58.2281811Z e6225129924f: Waiting 2025-12-04T09:24:58.2281964Z dabb51b819a0: Waiting 2025-12-04T09:24:58.2282126Z 5cd6b9ec5c3c: Waiting 2025-12-04T09:24:58.2282282Z 8f36833a24d0: Waiting 2025-12-04T09:24:58.2282438Z ab53c5a853a5: Waiting 2025-12-04T09:24:58.2282596Z 613d2b1f3db8: Waiting 2025-12-04T09:24:58.2282754Z 821085416919: Waiting 2025-12-04T09:24:58.2282901Z 388169fffe8a: Waiting 2025-12-04T09:24:58.2283059Z ce3394c8f210: Waiting 2025-12-04T09:24:58.2283216Z 3519ac15be79: Waiting 2025-12-04T09:24:58.2283368Z 79fcfd297d9f: Waiting 2025-12-04T09:24:58.2283529Z b7d40b4fd1b9: Waiting 2025-12-04T09:24:58.2283691Z dd1cdd87320d: Waiting 2025-12-04T09:24:58.2283862Z c21a6e1cd03a: Waiting 2025-12-04T09:24:58.2284020Z 0aa3ea0b2754: Waiting 2025-12-04T09:24:58.2284175Z 6ca498e78b7b: Waiting 2025-12-04T09:24:58.2284324Z 74e0bdbb05d3: Waiting 2025-12-04T09:24:58.2284487Z 7bd89c134b49: Waiting 2025-12-04T09:24:58.2284646Z 952b753ec7be: Waiting 2025-12-04T09:24:58.2284796Z ba3aee7dcf03: Waiting 2025-12-04T09:24:58.2284956Z 5b6f6de4bdd1: Waiting 2025-12-04T09:24:58.2285114Z 8c8d59b8759d: Waiting 2025-12-04T09:24:58.2285265Z 3c960865867b: Waiting 2025-12-04T09:24:58.2285412Z fa23d9952f3c: Waiting 2025-12-04T09:24:58.2285568Z 074db79e3832: Waiting 2025-12-04T09:24:58.2285723Z c2aa9ea4d09a: Waiting 2025-12-04T09:24:58.2285870Z 226748f4ff23: Waiting 2025-12-04T09:24:58.2286028Z a9bbc9e426d3: Waiting 2025-12-04T09:24:58.2286183Z 1f6d5b941ea1: Waiting 2025-12-04T09:24:58.2286333Z 82f2371f6dc2: Waiting 2025-12-04T09:24:58.2286494Z bf45550ddbad: Waiting 2025-12-04T09:24:58.2286649Z c87301921afa: Waiting 2025-12-04T09:24:58.2286803Z 384d566a822f: Waiting 2025-12-04T09:24:58.2286960Z 4444f70d73ce: Waiting 2025-12-04T09:24:58.2287117Z 30fff4f4bad3: Waiting 2025-12-04T09:24:58.2287269Z cca4db1ba155: Waiting 2025-12-04T09:24:58.2287427Z 32ddcdd5dd69: Waiting 2025-12-04T09:24:58.2287580Z 648018658875: Waiting 2025-12-04T09:24:58.2287728Z a5c3888c3a0c: Waiting 2025-12-04T09:24:58.2287887Z 5e2f7eac20ad: Waiting 2025-12-04T09:24:58.2288049Z 0d1243af7593: Waiting 2025-12-04T09:24:58.2288192Z f6fbbbe6067e: Waiting 2025-12-04T09:24:58.2288340Z df645e678f95: Waiting 2025-12-04T09:24:58.2288488Z 52ad6c17d103: Waiting 2025-12-04T09:24:58.2288636Z a116a32ceaa1: Waiting 2025-12-04T09:24:58.2288784Z 27803b661d9e: Waiting 2025-12-04T09:24:58.2288927Z 34408374e32c: Waiting 2025-12-04T09:24:58.2289076Z 07dafc893dea: Waiting 2025-12-04T09:24:58.2289227Z f0ffa379f4eb: Waiting 2025-12-04T09:24:58.2289548Z b4e1efca22be: Waiting 2025-12-04T09:24:58.2289707Z ed902a3a4e3b: Waiting 2025-12-04T09:24:58.3191616Z 6dc15eca5138: Download complete 2025-12-04T09:24:58.4097529Z 821085416919: Verifying Checksum 2025-12-04T09:24:58.4097942Z 821085416919: Download complete 2025-12-04T09:24:58.4872372Z 3738646b3d92: Download complete 2025-12-04T09:24:58.5787776Z 63e5bc7682b8: Verifying Checksum 2025-12-04T09:24:58.5788109Z 63e5bc7682b8: Download complete 2025-12-04T09:24:58.5794327Z aa9a10a37b39: Download complete 2025-12-04T09:24:58.6641273Z 55bac08f3e18: Verifying Checksum 2025-12-04T09:24:58.6641860Z 55bac08f3e18: Download complete 2025-12-04T09:24:58.6725300Z acf6468f6aad: Verifying Checksum 2025-12-04T09:24:58.6725588Z acf6468f6aad: Download complete 2025-12-04T09:24:58.7572648Z 0f7d0dc70d2d: Verifying Checksum 2025-12-04T09:24:58.7572930Z 0f7d0dc70d2d: Download complete 2025-12-04T09:24:58.8446980Z 58c4d15d3bc1: Verifying Checksum 2025-12-04T09:24:58.8449445Z 58c4d15d3bc1: Download complete 2025-12-04T09:24:58.9262959Z 2f1cc47b61e1: Verifying Checksum 2025-12-04T09:24:58.9267656Z 2f1cc47b61e1: Download complete 2025-12-04T09:24:59.0018348Z d2472a74103d: Download complete 2025-12-04T09:24:59.6757919Z 63e5bc7682b8: Pull complete 2025-12-04T09:24:59.6908323Z 6dc15eca5138: Pull complete 2025-12-04T09:24:59.8161670Z c0c31a0e69b7: Verifying Checksum 2025-12-04T09:24:59.8164494Z c0c31a0e69b7: Download complete 2025-12-04T09:24:59.8261344Z 4f4fb700ef54: Download complete 2025-12-04T09:24:59.9134191Z 6a9b03ce41a7: Verifying Checksum 2025-12-04T09:24:59.9134520Z 6a9b03ce41a7: Download complete 2025-12-04T09:24:59.9891418Z 3519ac15be79: Verifying Checksum 2025-12-04T09:24:59.9891976Z 3519ac15be79: Download complete 2025-12-04T09:25:00.0905034Z dabb51b819a0: Verifying Checksum 2025-12-04T09:25:00.0905431Z dabb51b819a0: Download complete 2025-12-04T09:25:00.1969130Z 8f36833a24d0: Verifying Checksum 2025-12-04T09:25:00.1969684Z 8f36833a24d0: Download complete 2025-12-04T09:25:00.2620586Z ab53c5a853a5: Verifying Checksum 2025-12-04T09:25:00.2620910Z ab53c5a853a5: Download complete 2025-12-04T09:25:00.3512056Z 025a0e5e6ac1: Verifying Checksum 2025-12-04T09:25:00.3512351Z 025a0e5e6ac1: Download complete 2025-12-04T09:25:00.4160202Z ce3394c8f210: Verifying Checksum 2025-12-04T09:25:00.4165715Z ce3394c8f210: Download complete 2025-12-04T09:25:00.5021832Z a5c3888c3a0c: Download complete 2025-12-04T09:25:00.5919018Z ed902a3a4e3b: Verifying Checksum 2025-12-04T09:25:00.5919579Z ed902a3a4e3b: Download complete 2025-12-04T09:25:00.6715989Z b4e1efca22be: Download complete 2025-12-04T09:25:00.7702499Z 79fcfd297d9f: Verifying Checksum 2025-12-04T09:25:00.7703444Z 79fcfd297d9f: Download complete 2025-12-04T09:25:00.8462548Z 7bd89c134b49: Verifying Checksum 2025-12-04T09:25:00.8464329Z 7bd89c134b49: Download complete 2025-12-04T09:25:01.4277308Z 459f4df18f07: Verifying Checksum 2025-12-04T09:25:01.4277936Z 459f4df18f07: Download complete 2025-12-04T09:25:01.5258302Z dd1cdd87320d: Verifying Checksum 2025-12-04T09:25:01.5258879Z dd1cdd87320d: Download complete 2025-12-04T09:25:01.6070718Z c21a6e1cd03a: Verifying Checksum 2025-12-04T09:25:01.6071028Z c21a6e1cd03a: Download complete 2025-12-04T09:25:01.6909986Z 0aa3ea0b2754: Verifying Checksum 2025-12-04T09:25:01.6910334Z 0aa3ea0b2754: Download complete 2025-12-04T09:25:01.7798820Z 74e0bdbb05d3: Verifying Checksum 2025-12-04T09:25:01.7801199Z 74e0bdbb05d3: Download complete 2025-12-04T09:25:02.0202898Z ba3aee7dcf03: Verifying Checksum 2025-12-04T09:25:02.0203183Z ba3aee7dcf03: Download complete 2025-12-04T09:25:02.0861268Z 8c8d59b8759d: Verifying Checksum 2025-12-04T09:25:02.0861579Z 8c8d59b8759d: Download complete 2025-12-04T09:25:02.1632576Z 3c960865867b: Download complete 2025-12-04T09:25:02.2456037Z 074db79e3832: Verifying Checksum 2025-12-04T09:25:02.2456391Z 074db79e3832: Download complete 2025-12-04T09:25:02.3480720Z 226748f4ff23: Download complete 2025-12-04T09:25:02.4586735Z 75d817336f1d: Verifying Checksum 2025-12-04T09:25:02.4587017Z 75d817336f1d: Download complete 2025-12-04T09:25:05.4975905Z b7d40b4fd1b9: Verifying Checksum 2025-12-04T09:25:05.4976204Z b7d40b4fd1b9: Download complete 2025-12-04T09:25:05.5941416Z e6225129924f: Verifying Checksum 2025-12-04T09:25:05.5942302Z e6225129924f: Download complete 2025-12-04T09:25:05.6661334Z 5cd6b9ec5c3c: Verifying Checksum 2025-12-04T09:25:05.6661871Z 5cd6b9ec5c3c: Download complete 2025-12-04T09:25:05.7385399Z 613d2b1f3db8: Verifying Checksum 2025-12-04T09:25:05.7386750Z 613d2b1f3db8: Download complete 2025-12-04T09:25:08.9208390Z 388169fffe8a: Verifying Checksum 2025-12-04T09:25:08.9209028Z 388169fffe8a: Download complete 2025-12-04T09:25:12.0906799Z 459f4df18f07: Pull complete 2025-12-04T09:25:12.4289533Z 821085416919: Pull complete 2025-12-04T09:25:12.7886324Z 3738646b3d92: Pull complete 2025-12-04T09:25:13.0078193Z aa9a10a37b39: Pull complete 2025-12-04T09:25:13.2342718Z 55bac08f3e18: Pull complete 2025-12-04T09:25:13.4577750Z acf6468f6aad: Pull complete 2025-12-04T09:25:16.7347731Z c0c31a0e69b7: Pull complete 2025-12-04T09:25:17.0076179Z 0f7d0dc70d2d: Pull complete 2025-12-04T09:25:17.2511297Z 58c4d15d3bc1: Pull complete 2025-12-04T09:25:17.5323229Z 2f1cc47b61e1: Pull complete 2025-12-04T09:25:17.8028695Z d2472a74103d: Pull complete 2025-12-04T09:25:32.5802089Z d1fba7293688: Verifying Checksum 2025-12-04T09:25:32.5806394Z d1fba7293688: Download complete 2025-12-04T09:25:32.6750853Z 952b753ec7be: Verifying Checksum 2025-12-04T09:25:32.6753701Z 952b753ec7be: Download complete 2025-12-04T09:25:32.7727945Z 5b6f6de4bdd1: Verifying Checksum 2025-12-04T09:25:32.7732335Z 5b6f6de4bdd1: Download complete 2025-12-04T09:25:32.8600785Z fa23d9952f3c: Download complete 2025-12-04T09:25:32.9371139Z c2aa9ea4d09a: Verifying Checksum 2025-12-04T09:25:32.9371729Z c2aa9ea4d09a: Download complete 2025-12-04T09:25:33.0204020Z a9bbc9e426d3: Verifying Checksum 2025-12-04T09:25:33.0208174Z a9bbc9e426d3: Download complete 2025-12-04T09:25:33.1063201Z 82f2371f6dc2: Download complete 2025-12-04T09:25:33.1785444Z c87301921afa: Verifying Checksum 2025-12-04T09:25:33.1789014Z c87301921afa: Download complete 2025-12-04T09:25:33.2670282Z 384d566a822f: Verifying Checksum 2025-12-04T09:25:33.2670572Z 384d566a822f: Download complete 2025-12-04T09:25:33.3561268Z 32ddcdd5dd69: Download complete 2025-12-04T09:25:33.4519107Z a116a32ceaa1: Verifying Checksum 2025-12-04T09:25:33.4519651Z a116a32ceaa1: Download complete 2025-12-04T09:25:33.5899391Z f6fbbbe6067e: Verifying Checksum 2025-12-04T09:25:33.5899768Z f6fbbbe6067e: Download complete 2025-12-04T09:25:33.6776901Z 648018658875: Download complete 2025-12-04T09:25:33.7736408Z 5e2f7eac20ad: Verifying Checksum 2025-12-04T09:25:33.7736699Z 5e2f7eac20ad: Download complete 2025-12-04T09:25:33.8530885Z 0d1243af7593: Download complete 2025-12-04T09:25:33.9421245Z 1f6d5b941ea1: Download complete 2025-12-04T09:25:34.0235086Z 4444f70d73ce: Verifying Checksum 2025-12-04T09:25:34.0235397Z 4444f70d73ce: Download complete 2025-12-04T09:25:34.1065760Z bf45550ddbad: Verifying Checksum 2025-12-04T09:25:34.1066124Z bf45550ddbad: Download complete 2025-12-04T09:25:36.6270011Z 30fff4f4bad3: Verifying Checksum 2025-12-04T09:25:36.6270315Z 30fff4f4bad3: Download complete 2025-12-04T09:25:36.7102196Z cca4db1ba155: Download complete 2025-12-04T09:25:36.8041477Z f0ffa379f4eb: Download complete 2025-12-04T09:25:36.9112951Z df645e678f95: Verifying Checksum 2025-12-04T09:25:36.9113323Z df645e678f95: Download complete 2025-12-04T09:25:36.9990635Z 52ad6c17d103: Verifying Checksum 2025-12-04T09:25:36.9990942Z 52ad6c17d103: Download complete 2025-12-04T09:25:37.0712444Z 34408374e32c: Verifying Checksum 2025-12-04T09:25:37.0712748Z 34408374e32c: Download complete 2025-12-04T09:25:37.1849910Z 27803b661d9e: Verifying Checksum 2025-12-04T09:25:37.1850242Z 27803b661d9e: Download complete 2025-12-04T09:25:37.7805105Z 07dafc893dea: Verifying Checksum 2025-12-04T09:25:37.7805428Z 07dafc893dea: Download complete 2025-12-04T09:26:15.2560757Z 6ca498e78b7b: Verifying Checksum 2025-12-04T09:26:15.2561074Z 6ca498e78b7b: Download complete 2025-12-04T09:26:42.1373326Z d1fba7293688: Pull complete 2025-12-04T09:26:42.6683202Z 4f4fb700ef54: Pull complete 2025-12-04T09:26:43.1974657Z 6a9b03ce41a7: Pull complete 2025-12-04T09:26:43.7388292Z 3519ac15be79: Pull complete 2025-12-04T09:26:44.1154516Z dabb51b819a0: Pull complete 2025-12-04T09:26:44.5079707Z 8f36833a24d0: Pull complete 2025-12-04T09:26:44.7359782Z ab53c5a853a5: Pull complete 2025-12-04T09:26:44.9865347Z 025a0e5e6ac1: Pull complete 2025-12-04T09:26:45.1740107Z ce3394c8f210: Pull complete 2025-12-04T09:26:45.2284883Z a5c3888c3a0c: Pull complete 2025-12-04T09:26:45.3003518Z ed902a3a4e3b: Pull complete 2025-12-04T09:26:45.3434383Z b4e1efca22be: Pull complete 2025-12-04T09:26:45.4092335Z 79fcfd297d9f: Pull complete 2025-12-04T09:26:45.4747576Z 7bd89c134b49: Pull complete 2025-12-04T09:26:56.8296271Z b7d40b4fd1b9: Pull complete 2025-12-04T09:26:57.0136531Z dd1cdd87320d: Pull complete 2025-12-04T09:26:57.3229967Z c21a6e1cd03a: Pull complete 2025-12-04T09:26:58.2477911Z 0aa3ea0b2754: Pull complete 2025-12-04T09:26:58.4444171Z 74e0bdbb05d3: Pull complete 2025-12-04T09:26:59.0981044Z ba3aee7dcf03: Pull complete 2025-12-04T09:26:59.5343593Z 8c8d59b8759d: Pull complete 2025-12-04T09:26:59.9769886Z 3c960865867b: Pull complete 2025-12-04T09:27:00.8143616Z 074db79e3832: Pull complete 2025-12-04T09:27:01.3006229Z 226748f4ff23: Pull complete 2025-12-04T09:27:01.8017925Z 75d817336f1d: Pull complete 2025-12-04T09:28:11.4183086Z 850870a17b94: Verifying Checksum 2025-12-04T09:28:11.4183466Z 850870a17b94: Download complete 2025-12-04T09:32:08.0926490Z 850870a17b94: Pull complete 2025-12-04T09:32:08.1198104Z e6225129924f: Pull complete 2025-12-04T09:32:08.1459012Z 5cd6b9ec5c3c: Pull complete 2025-12-04T09:32:08.1990349Z 613d2b1f3db8: Pull complete 2025-12-04T09:32:10.5949907Z 388169fffe8a: Pull complete 2025-12-04T09:34:27.4680555Z 6ca498e78b7b: Pull complete 2025-12-04T09:34:27.8339234Z 952b753ec7be: Pull complete 2025-12-04T09:34:28.2765764Z 5b6f6de4bdd1: Pull complete 2025-12-04T09:34:29.1074225Z fa23d9952f3c: Pull complete 2025-12-04T09:34:29.9842765Z c2aa9ea4d09a: Pull complete 2025-12-04T09:34:30.2101047Z a9bbc9e426d3: Pull complete 2025-12-04T09:34:30.6530301Z 82f2371f6dc2: Pull complete 2025-12-04T09:34:31.3407588Z c87301921afa: Pull complete 2025-12-04T09:34:31.7123846Z 384d566a822f: Pull complete 2025-12-04T09:34:32.6101581Z 32ddcdd5dd69: Pull complete 2025-12-04T09:34:33.5316259Z a116a32ceaa1: Pull complete 2025-12-04T09:34:34.0691986Z f6fbbbe6067e: Pull complete 2025-12-04T09:34:34.6504496Z 648018658875: Pull complete 2025-12-04T09:34:35.1173382Z 5e2f7eac20ad: Pull complete 2025-12-04T09:34:35.9456580Z 0d1243af7593: Pull complete 2025-12-04T09:34:36.3213273Z 1f6d5b941ea1: Pull complete 2025-12-04T09:34:37.1565439Z 4444f70d73ce: Pull complete 2025-12-04T09:34:37.5933052Z bf45550ddbad: Pull complete 2025-12-04T09:34:46.3763744Z 30fff4f4bad3: Pull complete 2025-12-04T09:34:46.8389949Z cca4db1ba155: Pull complete 2025-12-04T09:34:47.2886745Z f0ffa379f4eb: Pull complete 2025-12-04T09:34:47.7799395Z df645e678f95: Pull complete 2025-12-04T09:34:48.3036940Z 52ad6c17d103: Pull complete 2025-12-04T09:34:48.7485579Z 34408374e32c: Pull complete 2025-12-04T09:34:49.5057716Z 27803b661d9e: Pull complete 2025-12-04T09:34:51.6912558Z 07dafc893dea: Pull complete 2025-12-04T09:34:51.7341225Z Digest: sha256:b178ee928adfcab963ebbc6ea05ea1a0f0c605bb095e9076f010bf92d150869d 2025-12-04T09:34:51.7400766Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:34:51.7428093Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:34:51.7478699Z ##[group]Run echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-12-04T09:34:51.7479301Z echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-12-04T09:34:51.7487753Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:34:51.7487992Z env: 2025-12-04T09:34:51.7488138Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:34:51.7488310Z ##[endgroup] 2025-12-04T09:34:51.7608017Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2025-12-04T09:34:51.7608272Z with: 2025-12-04T09:34:51.7608425Z driver-version: 580.82.07 2025-12-04T09:34:51.7608587Z env: 2025-12-04T09:34:51.7608733Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:34:51.7608899Z ##[endgroup] 2025-12-04T09:34:51.7619073Z ##[group]Run echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-12-04T09:34:51.7619574Z echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-12-04T09:34:51.7624029Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:34:51.7624256Z env: 2025-12-04T09:34:51.7624401Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:34:51.7624562Z ##[endgroup] 2025-12-04T09:34:51.7669280Z ##[group]Run set -euo pipefail 2025-12-04T09:34:51.7669502Z set -euo pipefail 2025-12-04T09:34:51.7669704Z  2025-12-04T09:34:51.7669842Z has_gpu=false 2025-12-04T09:34:51.7670004Z devices="" 2025-12-04T09:34:51.7670151Z  2025-12-04T09:34:51.7670324Z if command -v nvidia-smi >/dev/null 2>&1; then 2025-12-04T09:34:51.7670604Z  if nvidia-smi -L >/tmp/nvidia_devices 2>/dev/null; then 2025-12-04T09:34:51.7670837Z  has_gpu=true 2025-12-04T09:34:51.7671029Z  devices=$(cat /tmp/nvidia_devices) 2025-12-04T09:34:51.7671217Z  fi 2025-12-04T09:34:51.7671370Z fi 2025-12-04T09:34:51.7671510Z  2025-12-04T09:34:51.7671653Z if [ "$has_gpu" = false ]; then 2025-12-04T09:34:51.7671895Z  if ls /dev/nvidia* >/tmp/nvidia_devices 2>/dev/null; then 2025-12-04T09:34:51.7672122Z  has_gpu=true 2025-12-04T09:34:51.7672305Z  devices=$(cat /tmp/nvidia_devices) 2025-12-04T09:34:51.7672490Z  fi 2025-12-04T09:34:51.7672628Z fi 2025-12-04T09:34:51.7672765Z  2025-12-04T09:34:51.7672956Z if [ "$has_gpu" = false ] && command -v lspci >/dev/null 2>&1; then 2025-12-04T09:34:51.7673469Z  if lspci | grep -i 'nvidia' >/tmp/nvidia_devices 2>/dev/null; then 2025-12-04T09:34:51.7673720Z  has_gpu=true 2025-12-04T09:34:51.7673904Z  devices=$(cat /tmp/nvidia_devices) 2025-12-04T09:34:51.7674099Z  fi 2025-12-04T09:34:51.7674247Z fi 2025-12-04T09:34:51.7674376Z  2025-12-04T09:34:51.7674572Z printf 'HAS_NVIDIA=%s\n' "$has_gpu" >> "$GITHUB_OUTPUT" 2025-12-04T09:34:51.7674895Z printf 'DETECTED_DEVICES<> "$GITHUB_OUTPUT" 2025-12-04T09:34:51.7678760Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:34:51.7678979Z env: 2025-12-04T09:34:51.7679126Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:34:51.7679293Z ##[endgroup] 2025-12-04T09:34:51.8002981Z ##[group]Run if [ "${HAS_NVIDIA}" = "true" ]; then 2025-12-04T09:34:51.8003317Z if [ "${HAS_NVIDIA}" = "true" ]; then 2025-12-04T09:34:51.8003572Z  echo "HAS_NVIDIA_GPU=true" >> "${GITHUB_ENV}" 2025-12-04T09:34:51.8003900Z  echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" 2025-12-04T09:34:51.8004196Z else 2025-12-04T09:34:51.8004394Z  echo "HAS_NVIDIA_GPU=false" >> "${GITHUB_ENV}" 2025-12-04T09:34:51.8004622Z fi 2025-12-04T09:34:51.8009293Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:34:51.8009552Z env: 2025-12-04T09:34:51.8009702Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:34:51.8009868Z HAS_NVIDIA: false 2025-12-04T09:34:51.8010028Z ##[endgroup] 2025-12-04T09:34:51.8075887Z Prepare all required actions 2025-12-04T09:34:51.8094162Z ##[group]Run ./.github/actions/get-workflow-job-id 2025-12-04T09:34:51.8094381Z with: 2025-12-04T09:34:51.8094869Z github-token: *** 2025-12-04T09:34:51.8095025Z env: 2025-12-04T09:34:51.8095164Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:34:51.8095343Z HAS_NVIDIA_GPU: false 2025-12-04T09:34:51.8095509Z ##[endgroup] 2025-12-04T09:34:51.8105197Z ##[group]Run set -eux 2025-12-04T09:34:51.8105367Z set -eux 2025-12-04T09:34:51.8105632Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2025-12-04T09:34:51.8110324Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:34:51.8110579Z env: 2025-12-04T09:34:51.8110747Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:34:51.8110940Z HAS_NVIDIA_GPU: false 2025-12-04T09:34:51.8111390Z GITHUB_TOKEN: *** 2025-12-04T09:34:51.8111552Z ##[endgroup] 2025-12-04T09:34:51.8134749Z + python3 .github/scripts/get_workflow_job_id.py 19923066595 i-04ab61807b125ed1d 2025-12-04T09:34:53.7090656Z Setting output job-id=57118563323 2025-12-04T09:34:53.7092856Z Setting output job-name=periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_huggingface, 1, 1, linux.8xlarge.amx) 2025-12-04T09:34:53.7196853Z ##[group]Run python3 -m pip install psutil==5.9.8 dataclasses_json==0.6.7 nvidia-ml-py==11.525.84 2025-12-04T09:34:53.7197352Z python3 -m pip install psutil==5.9.8 dataclasses_json==0.6.7 nvidia-ml-py==11.525.84 2025-12-04T09:34:53.7197988Z python3 -m tools.stats.monitor --log-interval "$MONITOR_LOG_INTERVAL" --data-collect-interval "$MONITOR_DATA_COLLECT_INTERVAL" > usage_log.txt 2>&1 & 2025-12-04T09:34:53.7198550Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2025-12-04T09:34:53.7203765Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:34:53.7204008Z env: 2025-12-04T09:34:53.7204160Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:34:53.7204337Z HAS_NVIDIA_GPU: false 2025-12-04T09:34:53.7204494Z JOB_ID: 57118563323 2025-12-04T09:34:53.7204832Z JOB_NAME: periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_huggingface, 1, 1, linux.8xlarge.amx) 2025-12-04T09:34:53.7205240Z WORKFLOW_NAME: inductor-periodic 2025-12-04T09:34:53.7205426Z WORKFLOW_RUN_ID: 19923066595 2025-12-04T09:34:53.7205602Z MONITOR_LOG_INTERVAL: 5 2025-12-04T09:34:53.7206175Z MONITOR_DATA_COLLECT_INTERVAL: 1 2025-12-04T09:34:53.7206358Z ##[endgroup] 2025-12-04T09:34:53.9536977Z Defaulting to user installation because normal site-packages is not writeable 2025-12-04T09:34:54.2014645Z Collecting psutil==5.9.8 2025-12-04T09:34:54.2157565Z Downloading psutil-5.9.8-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (288 kB) 2025-12-04T09:34:54.2698786Z Collecting dataclasses_json==0.6.7 2025-12-04T09:34:54.2730987Z Downloading dataclasses_json-0.6.7-py3-none-any.whl (28 kB) 2025-12-04T09:34:54.2965421Z Collecting nvidia-ml-py==11.525.84 2025-12-04T09:34:54.2997862Z Downloading nvidia_ml_py-11.525.84-py3-none-any.whl (34 kB) 2025-12-04T09:34:54.3271399Z Collecting typing-inspect<1,>=0.4.0 2025-12-04T09:34:54.3300844Z Downloading typing_inspect-0.9.0-py3-none-any.whl (8.8 kB) 2025-12-04T09:34:54.4032041Z Collecting marshmallow<4.0.0,>=3.18.0 2025-12-04T09:34:54.4069130Z Downloading marshmallow-3.26.1-py3-none-any.whl (50 kB) 2025-12-04T09:34:54.4488676Z Collecting packaging>=17.0 2025-12-04T09:34:54.4519675Z Downloading packaging-25.0-py3-none-any.whl (66 kB) 2025-12-04T09:34:54.4735250Z Collecting mypy-extensions>=0.3.0 2025-12-04T09:34:54.4769516Z Downloading mypy_extensions-1.1.0-py3-none-any.whl (5.0 kB) 2025-12-04T09:34:54.5127294Z Collecting typing-extensions>=3.7.4 2025-12-04T09:34:54.5160492Z Downloading typing_extensions-4.15.0-py3-none-any.whl (44 kB) 2025-12-04T09:34:54.5904902Z Installing collected packages: typing-extensions, packaging, mypy-extensions, typing-inspect, marshmallow, psutil, nvidia-ml-py, dataclasses-json 2025-12-04T09:34:54.8257225Z Successfully installed dataclasses-json-0.6.7 marshmallow-3.26.1 mypy-extensions-1.1.0 nvidia-ml-py-11.525.84 packaging-25.0 psutil-5.9.8 typing-extensions-4.15.0 typing-inspect-0.9.0 2025-12-04T09:34:54.9833300Z Prepare all required actions 2025-12-04T09:34:54.9833755Z Getting action download info 2025-12-04T09:34:55.1469975Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2025-12-04T09:34:55.3954210Z Download action repository 'actions/download-artifact@v4' (SHA:d3f86a106a0bac45b974a628896c90dbdf5c8093) 2025-12-04T09:34:55.7404924Z ##[group]Run ./.github/actions/download-build-artifacts 2025-12-04T09:34:55.7405161Z with: 2025-12-04T09:34:55.7405336Z name: linux-jammy-py3.10-gcc11-build 2025-12-04T09:34:55.7405551Z s3-bucket: gha-artifacts 2025-12-04T09:34:55.7406240Z env: 2025-12-04T09:34:55.7406393Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:34:55.7406568Z HAS_NVIDIA_GPU: false 2025-12-04T09:34:55.7406727Z ##[endgroup] 2025-12-04T09:34:55.7446177Z ##[group]Run seemethere/download-artifact-s3@v4 2025-12-04T09:34:55.7446396Z with: 2025-12-04T09:34:55.7446566Z name: linux-jammy-py3.10-gcc11-build 2025-12-04T09:34:55.7446812Z s3-bucket: gha-artifacts 2025-12-04T09:34:55.7446997Z region: us-east-1 2025-12-04T09:34:55.7447137Z env: 2025-12-04T09:34:55.7447277Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:34:55.7447442Z HAS_NVIDIA_GPU: false 2025-12-04T09:34:55.7447594Z ##[endgroup] 2025-12-04T09:34:56.1462931Z (node:48487) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2025-12-04T09:34:56.1463632Z 2025-12-04T09:34:56.1463779Z Please migrate your code to use AWS SDK for JavaScript (v3). 2025-12-04T09:34:56.1464116Z For more information, check the migration guide at https://a.co/7PzMCcy 2025-12-04T09:34:56.1464501Z (Use `node --trace-warnings ...` to show where the warning was created) 2025-12-04T09:34:57.2686123Z Found 1 objects with prefix pytorch/pytorch/19923066595/linux-jammy-py3.10-gcc11-build/ 2025-12-04T09:34:57.2690648Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2025-12-04T09:35:04.8975154Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2025-12-04T09:35:04.8982420Z Artifact download has finished successfully 2025-12-04T09:35:04.9223317Z ##[group]Run unzip -o artifacts.zip 2025-12-04T09:35:04.9223555Z unzip -o artifacts.zip 2025-12-04T09:35:04.9228001Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:35:04.9228225Z env: 2025-12-04T09:35:04.9228372Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:35:04.9228545Z HAS_NVIDIA_GPU: false 2025-12-04T09:35:04.9228705Z ##[endgroup] 2025-12-04T09:35:04.9305501Z Archive: artifacts.zip 2025-12-04T09:35:04.9306045Z creating: dist/ 2025-12-04T09:35:05.9743155Z inflating: dist/torch-2.10.0a0+gitffd9b0f-cp310-cp310-linux_x86_64.whl 2025-12-04T09:35:05.9744771Z creating: dist/vision/ 2025-12-04T09:35:05.9816232Z inflating: dist/vision/torchvision-0.25.0a0+617079d-cp310-cp310-linux_x86_64.whl 2025-12-04T09:35:05.9820688Z creating: dist/audio/ 2025-12-04T09:35:05.9840523Z inflating: dist/audio/torchaudio-2.10.0a0+e90a398-cp310-cp310-linux_x86_64.whl 2025-12-04T09:35:05.9842259Z creating: dist/ao/ 2025-12-04T09:35:05.9875833Z inflating: dist/ao/torchao-0.7.0+git51c87b6e-py3-none-any.whl 2025-12-04T09:35:05.9980732Z inflating: dist/.ninja_log 2025-12-04T09:35:05.9981212Z creating: build/custom_test_artifacts/ 2025-12-04T09:35:05.9981604Z creating: build/custom_test_artifacts/custom-op-build/ 2025-12-04T09:35:05.9982649Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2025-12-04T09:35:05.9983254Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2025-12-04T09:35:05.9984173Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2025-12-04T09:35:05.9984610Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/ 2025-12-04T09:35:05.9984994Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CMakeSystem.cmake 2025-12-04T09:35:05.9985825Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CompilerIdC/ 2025-12-04T09:35:05.9986234Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CompilerIdC/tmp/ 2025-12-04T09:35:05.9986960Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CompilerIdC/CMakeCCompilerId.c 2025-12-04T09:35:05.9987903Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CompilerIdC/a.out 2025-12-04T09:35:05.9988370Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CMakeCCompiler.cmake 2025-12-04T09:35:05.9988782Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CompilerIdCXX/ 2025-12-04T09:35:05.9989452Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CompilerIdCXX/tmp/ 2025-12-04T09:35:05.9990047Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CompilerIdCXX/CMakeCXXCompilerId.cpp 2025-12-04T09:35:05.9995555Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CompilerIdCXX/a.out 2025-12-04T09:35:05.9999640Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CMakeCXXCompiler.cmake 2025-12-04T09:35:06.0001603Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CMakeDetermineCompilerABI_C.bin 2025-12-04T09:35:06.0002265Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CMakeDetermineCompilerABI_CXX.bin 2025-12-04T09:35:06.0007174Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2025-12-04T09:35:06.0008937Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2025-12-04T09:35:06.0009505Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2025-12-04T09:35:06.0014704Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2025-12-04T09:35:06.0019741Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2025-12-04T09:35:06.0021423Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2025-12-04T09:35:06.0021891Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2025-12-04T09:35:06.0022323Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2025-12-04T09:35:06.0022760Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2025-12-04T09:35:06.0023191Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2025-12-04T09:35:06.0023635Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2025-12-04T09:35:06.0024058Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2025-12-04T09:35:06.0024496Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2025-12-04T09:35:06.0183020Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2025-12-04T09:35:06.0184758Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2025-12-04T09:35:06.0185342Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2025-12-04T09:35:06.0190522Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2025-12-04T09:35:06.0194784Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2025-12-04T09:35:06.0196942Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2025-12-04T09:35:06.0197541Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2025-12-04T09:35:06.0202870Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2025-12-04T09:35:06.0207394Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2025-12-04T09:35:06.0209526Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2025-12-04T09:35:06.0210045Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2025-12-04T09:35:06.0210560Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2025-12-04T09:35:06.0271723Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2025-12-04T09:35:06.0272436Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2025-12-04T09:35:06.0272922Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2025-12-04T09:35:06.0273482Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2025-12-04T09:35:06.0273909Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2025-12-04T09:35:06.0274329Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2025-12-04T09:35:06.0274729Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2025-12-04T09:35:06.0275064Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2025-12-04T09:35:06.0275413Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2025-12-04T09:35:06.0421887Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2025-12-04T09:35:06.0470222Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2025-12-04T09:35:06.0471980Z creating: build/custom_test_artifacts/jit-hook-build/ 2025-12-04T09:35:06.0472441Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2025-12-04T09:35:06.0472872Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2025-12-04T09:35:06.0473849Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2025-12-04T09:35:06.0478284Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/ 2025-12-04T09:35:06.0480362Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CMakeSystem.cmake 2025-12-04T09:35:06.0480922Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CompilerIdC/ 2025-12-04T09:35:06.0484659Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CompilerIdC/tmp/ 2025-12-04T09:35:06.0486645Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CompilerIdC/CMakeCCompilerId.c 2025-12-04T09:35:06.0487244Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CompilerIdC/a.out 2025-12-04T09:35:06.0491447Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CMakeCCompiler.cmake 2025-12-04T09:35:06.0493445Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CompilerIdCXX/ 2025-12-04T09:35:06.0493996Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CompilerIdCXX/tmp/ 2025-12-04T09:35:06.0498443Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CompilerIdCXX/CMakeCXXCompilerId.cpp 2025-12-04T09:35:06.0502971Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CompilerIdCXX/a.out 2025-12-04T09:35:06.0503464Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CMakeCXXCompiler.cmake 2025-12-04T09:35:06.0503952Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CMakeDetermineCompilerABI_C.bin 2025-12-04T09:35:06.0504445Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CMakeDetermineCompilerABI_CXX.bin 2025-12-04T09:35:06.0505123Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2025-12-04T09:35:06.0505508Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2025-12-04T09:35:06.0506235Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2025-12-04T09:35:06.0506660Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2025-12-04T09:35:06.0507138Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2025-12-04T09:35:06.0507593Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2025-12-04T09:35:06.0508125Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2025-12-04T09:35:06.0508553Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2025-12-04T09:35:06.0508997Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2025-12-04T09:35:06.0509441Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2025-12-04T09:35:06.0509880Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2025-12-04T09:35:06.0510303Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2025-12-04T09:35:06.0510768Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2025-12-04T09:35:06.0556188Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2025-12-04T09:35:06.0561192Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2025-12-04T09:35:06.0565423Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2025-12-04T09:35:06.0569481Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2025-12-04T09:35:06.0571649Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2025-12-04T09:35:06.0572149Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2025-12-04T09:35:06.0577009Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2025-12-04T09:35:06.0578775Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2025-12-04T09:35:06.0579367Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2025-12-04T09:35:06.0591431Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2025-12-04T09:35:06.0591945Z creating: build/custom_test_artifacts/custom-backend-build/ 2025-12-04T09:35:06.0592402Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2025-12-04T09:35:06.0592794Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2025-12-04T09:35:06.0598340Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2025-12-04T09:35:06.0598847Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/ 2025-12-04T09:35:06.0599284Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CMakeSystem.cmake 2025-12-04T09:35:06.0599752Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CompilerIdC/ 2025-12-04T09:35:06.0600206Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CompilerIdC/tmp/ 2025-12-04T09:35:06.0600719Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CompilerIdC/CMakeCCompilerId.c 2025-12-04T09:35:06.0601253Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CompilerIdC/a.out 2025-12-04T09:35:06.0601739Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CMakeCCompiler.cmake 2025-12-04T09:35:06.0602471Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CompilerIdCXX/ 2025-12-04T09:35:06.0602936Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CompilerIdCXX/tmp/ 2025-12-04T09:35:06.0603580Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CompilerIdCXX/CMakeCXXCompilerId.cpp 2025-12-04T09:35:06.0607995Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CompilerIdCXX/a.out 2025-12-04T09:35:06.0609984Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CMakeCXXCompiler.cmake 2025-12-04T09:35:06.0610881Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CMakeDetermineCompilerABI_C.bin 2025-12-04T09:35:06.0614950Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CMakeDetermineCompilerABI_CXX.bin 2025-12-04T09:35:06.0616854Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2025-12-04T09:35:06.0617397Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2025-12-04T09:35:06.0622046Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2025-12-04T09:35:06.0625715Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2025-12-04T09:35:06.0628044Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2025-12-04T09:35:06.0628601Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2025-12-04T09:35:06.0629081Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2025-12-04T09:35:06.0629566Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2025-12-04T09:35:06.0630052Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2025-12-04T09:35:06.0630531Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2025-12-04T09:35:06.0631003Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2025-12-04T09:35:06.0631475Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2025-12-04T09:35:06.0631977Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2025-12-04T09:35:06.0714443Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2025-12-04T09:35:06.0719260Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2025-12-04T09:35:06.0723014Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2025-12-04T09:35:06.0724934Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2025-12-04T09:35:06.0725587Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2025-12-04T09:35:06.0729818Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2025-12-04T09:35:06.0734995Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2025-12-04T09:35:06.0737129Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2025-12-04T09:35:06.0741981Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2025-12-04T09:35:06.0742705Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2025-12-04T09:35:06.0743973Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2025-12-04T09:35:06.0744590Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2025-12-04T09:35:06.0781137Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2025-12-04T09:35:06.0781856Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2025-12-04T09:35:06.0782869Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2025-12-04T09:35:06.0783594Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2025-12-04T09:35:06.0784001Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2025-12-04T09:35:06.0784390Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2025-12-04T09:35:06.0784755Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2025-12-04T09:35:06.0785096Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2025-12-04T09:35:06.0785440Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2025-12-04T09:35:06.0872641Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2025-12-04T09:35:06.0905547Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2025-12-04T09:35:06.0907051Z creating: build/lib/ 2025-12-04T09:35:06.0976163Z inflating: build/lib/libprotobuf-lite.a 2025-12-04T09:35:06.1354740Z inflating: build/lib/libprotobuf.a 2025-12-04T09:35:06.1780529Z inflating: build/lib/libprotoc.a 2025-12-04T09:35:06.1788861Z inflating: build/lib/libpthreadpool.a 2025-12-04T09:35:06.1793103Z inflating: build/lib/libcpuinfo.a 2025-12-04T09:35:06.1803216Z inflating: build/lib/libcpuinfo_internals.a 2025-12-04T09:35:06.1804927Z inflating: build/lib/libclog.a 2025-12-04T09:35:06.1819529Z inflating: build/lib/libpytorch_qnnpack.a 2025-12-04T09:35:06.1819965Z inflating: build/lib/libnnpack_reference_layers.a 2025-12-04T09:35:06.1981308Z inflating: build/lib/libmicrokernels-prod.a 2025-12-04T09:35:06.1995151Z inflating: build/lib/libnnpack.a 2025-12-04T09:35:06.2751223Z inflating: build/lib/libmicrokernels-all.a 2025-12-04T09:35:06.2811732Z inflating: build/lib/libgtest.a 2025-12-04T09:35:06.2826860Z inflating: build/lib/libgmock.a 2025-12-04T09:35:06.2828890Z inflating: build/lib/libgtest_main.a 2025-12-04T09:35:06.2829174Z inflating: build/lib/libgmock_main.a 2025-12-04T09:35:06.2904299Z inflating: build/lib/libXNNPACK.a 2025-12-04T09:35:06.2969387Z inflating: build/lib/libbenchmark.a 2025-12-04T09:35:06.2974480Z inflating: build/lib/libbenchmark_main.a 2025-12-04T09:35:06.2976449Z inflating: build/lib/libjitprofiling.a 2025-12-04T09:35:06.2976804Z inflating: build/lib/libittnotify.a 2025-12-04T09:35:06.3031696Z inflating: build/lib/libasmjit.a 2025-12-04T09:35:06.4018635Z inflating: build/lib/libfbgemm.a 2025-12-04T09:35:06.4044278Z inflating: build/lib/libtensorpipe_uv.a 2025-12-04T09:35:06.4504485Z inflating: build/lib/libtensorpipe.a 2025-12-04T09:35:06.4607836Z inflating: build/lib/libgloo.a 2025-12-04T09:35:06.4647339Z inflating: build/lib/libonnx_proto.a 2025-12-04T09:35:06.5253290Z inflating: build/lib/libonnx.a 2025-12-04T09:35:07.3824671Z inflating: build/lib/libdnnl.a 2025-12-04T09:35:07.3841590Z inflating: build/lib/libfmt.a 2025-12-04T09:35:07.4081703Z inflating: build/lib/libkineto.a 2025-12-04T09:35:07.4183442Z inflating: build/lib/libc10.so 2025-12-04T09:35:07.4185538Z inflating: build/lib/libtorch_global_deps.so 2025-12-04T09:35:10.0602489Z inflating: build/lib/libtorch_cpu.so 2025-12-04T09:35:10.0607090Z inflating: build/lib/libtorch.so 2025-12-04T09:35:10.0664189Z inflating: build/lib/libtorchbind_test.so 2025-12-04T09:35:10.0682809Z inflating: build/lib/libjitbackend_test.so 2025-12-04T09:35:10.0702715Z inflating: build/lib/libbackend_with_compiler.so 2025-12-04T09:35:10.0725159Z inflating: build/lib/libaoti_custom_ops.so 2025-12-04T09:35:10.0727161Z inflating: build/lib/libshm.so 2025-12-04T09:35:10.2696705Z inflating: build/lib/libtorch_python.so 2025-12-04T09:35:10.2730212Z inflating: build/lib/libnnapi_backend.so 2025-12-04T09:35:10.2735591Z creating: build/bin/ 2025-12-04T09:35:10.2739980Z creating: build/bin/CMakeFiles/ 2025-12-04T09:35:10.2743855Z inflating: build/bin/cmake_install.cmake 2025-12-04T09:35:10.2747781Z inflating: build/bin/CTestTestfile.cmake 2025-12-04T09:35:10.3129224Z inflating: build/bin/protoc-3.13.0.0 2025-12-04T09:35:10.3525571Z inflating: build/bin/protoc 2025-12-04T09:35:10.3578049Z inflating: build/bin/c10_AllocatorConfig_test 2025-12-04T09:35:10.3626058Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2025-12-04T09:35:10.3677231Z inflating: build/bin/c10_DeviceGuard_test 2025-12-04T09:35:10.3727622Z inflating: build/bin/c10_Device_test 2025-12-04T09:35:10.3785916Z inflating: build/bin/c10_DispatchKeySet_test 2025-12-04T09:35:10.3837665Z inflating: build/bin/c10_Scalar_test 2025-12-04T09:35:10.3884034Z inflating: build/bin/c10_StreamGuard_test 2025-12-04T09:35:10.3936847Z inflating: build/bin/c10_SymInt_test 2025-12-04T09:35:10.3989792Z inflating: build/bin/c10_InlineDeviceGuard_test 2025-12-04T09:35:10.4043871Z inflating: build/bin/c10_InlineStreamGuard_test 2025-12-04T09:35:10.4096190Z inflating: build/bin/c10_SizesAndStrides_test 2025-12-04T09:35:10.4163809Z inflating: build/bin/c10_cow_test 2025-12-04T09:35:10.4214509Z inflating: build/bin/c10_Bitset_test 2025-12-04T09:35:10.4261964Z inflating: build/bin/c10_ArrayRef_test 2025-12-04T09:35:10.4307742Z inflating: build/bin/c10_ConstexprCrc_test 2025-12-04T09:35:10.4357793Z inflating: build/bin/c10_DeadlockDetection_test 2025-12-04T09:35:10.4411128Z inflating: build/bin/c10_Enumerate_test 2025-12-04T09:35:10.4464395Z inflating: build/bin/c10_LeftRight_test 2025-12-04T09:35:10.4515495Z inflating: build/bin/c10_NetworkFlow_test 2025-12-04T09:35:10.4565445Z inflating: build/bin/c10_Half_test 2025-12-04T09:35:10.4616994Z inflating: build/bin/c10_IntrusiveList_test 2025-12-04T09:35:10.4663970Z inflating: build/bin/c10_Synchronized_test 2025-12-04T09:35:10.4717701Z inflating: build/bin/c10_ThreadLocal_test 2025-12-04T09:35:10.4767077Z inflating: build/bin/c10_Semaphore_test 2025-12-04T09:35:10.4820450Z inflating: build/bin/c10_bfloat16_test 2025-12-04T09:35:10.4870339Z inflating: build/bin/c10_accumulate_test 2025-12-04T09:35:10.4918819Z inflating: build/bin/c10_TypeIndex_test 2025-12-04T09:35:10.4968967Z inflating: build/bin/c10_exception_test 2025-12-04T09:35:10.5018408Z inflating: build/bin/c10_bit_cast_test 2025-12-04T09:35:10.5071342Z inflating: build/bin/c10_complex_math_test 2025-12-04T09:35:10.5119476Z inflating: build/bin/c10_error_test 2025-12-04T09:35:10.5166472Z inflating: build/bin/c10_flags_test 2025-12-04T09:35:10.5223680Z inflating: build/bin/c10_complex_test 2025-12-04T09:35:10.5270866Z inflating: build/bin/c10_irange_test 2025-12-04T09:35:10.5319693Z inflating: build/bin/c10_generic_math_test 2025-12-04T09:35:10.5367631Z inflating: build/bin/c10_nofatal_test 2025-12-04T09:35:10.5419548Z inflating: build/bin/c10_lazy_test 2025-12-04T09:35:10.5473438Z inflating: build/bin/c10_logging_test 2025-12-04T09:35:10.5615378Z inflating: build/bin/c10_intrusive_ptr_test 2025-12-04T09:35:10.5685317Z inflating: build/bin/c10_optional_test 2025-12-04T09:35:10.5737033Z inflating: build/bin/c10_registry_test 2025-12-04T09:35:10.5796962Z inflating: build/bin/c10_ordered_preserving_dict_test 2025-12-04T09:35:10.5846483Z inflating: build/bin/c10_ssize_test 2025-12-04T09:35:10.5985198Z inflating: build/bin/c10_small_vector_test 2025-12-04T09:35:10.6039863Z inflating: build/bin/c10_string_util_test 2025-12-04T09:35:10.6087121Z inflating: build/bin/c10_string_view_test 2025-12-04T09:35:10.6135685Z inflating: build/bin/c10_tempfile_test 2025-12-04T09:35:10.6178385Z inflating: build/bin/c10_intrusive_ptr_benchmark 2025-12-04T09:35:10.6231705Z inflating: build/bin/c10_typeid_test 2025-12-04T09:35:10.6746315Z inflating: build/bin/vec_test_all_types_DEFAULT 2025-12-04T09:35:10.7280237Z inflating: build/bin/vec_test_all_types_AVX512 2025-12-04T09:35:10.7820894Z inflating: build/bin/vec_test_all_types_AVX2 2025-12-04T09:35:10.7910842Z inflating: build/bin/test_aoti_abi_check 2025-12-04T09:35:10.7960028Z inflating: build/bin/test_vec_half_DEFAULT 2025-12-04T09:35:10.8006506Z inflating: build/bin/test_vec_half_AVX512 2025-12-04T09:35:10.8055587Z inflating: build/bin/test_vec_half_AVX2 2025-12-04T09:35:10.8105823Z inflating: build/bin/BackoffTest 2025-12-04T09:35:10.8157467Z inflating: build/bin/FileStoreTest 2025-12-04T09:35:10.8208778Z inflating: build/bin/static_runtime_bench 2025-12-04T09:35:10.8436124Z inflating: build/bin/static_runtime_test 2025-12-04T09:35:10.8504409Z inflating: build/bin/Dict_test 2025-12-04T09:35:10.8556037Z inflating: build/bin/Dimname_test 2025-12-04T09:35:10.8617811Z inflating: build/bin/MaybeOwned_test 2025-12-04T09:35:10.8670777Z inflating: build/bin/NamedTensor_test 2025-12-04T09:35:10.8728081Z inflating: build/bin/apply_utils_test 2025-12-04T09:35:10.8783406Z inflating: build/bin/atest 2025-12-04T09:35:10.8844938Z inflating: build/bin/basic 2025-12-04T09:35:10.8896436Z inflating: build/bin/broadcast_test 2025-12-04T09:35:10.8943903Z inflating: build/bin/cpu_allocator_test 2025-12-04T09:35:10.9000034Z inflating: build/bin/cpu_generator_test 2025-12-04T09:35:10.9051029Z inflating: build/bin/cpu_profiling_allocator_test 2025-12-04T09:35:10.9135779Z inflating: build/bin/cpu_rng_test 2025-12-04T09:35:10.9184986Z inflating: build/bin/dlconvertor_test 2025-12-04T09:35:10.9240474Z inflating: build/bin/extension_backend_test 2025-12-04T09:35:10.9293805Z inflating: build/bin/half_test 2025-12-04T09:35:10.9383360Z inflating: build/bin/ivalue_test 2025-12-04T09:35:10.9431301Z inflating: build/bin/lazy_tensor_test 2025-12-04T09:35:10.9482294Z inflating: build/bin/math_kernel_test 2025-12-04T09:35:10.9533625Z inflating: build/bin/memory_format_test 2025-12-04T09:35:10.9585321Z inflating: build/bin/memory_overlapping_test 2025-12-04T09:35:10.9638481Z inflating: build/bin/mobile_memory_cleanup 2025-12-04T09:35:10.9690113Z inflating: build/bin/native_test 2025-12-04T09:35:10.9737456Z inflating: build/bin/operator_name_test 2025-12-04T09:35:10.9787257Z inflating: build/bin/operators_test 2025-12-04T09:35:10.9838451Z inflating: build/bin/packedtensoraccessor_test 2025-12-04T09:35:10.9902361Z inflating: build/bin/pow_test 2025-12-04T09:35:10.9954759Z inflating: build/bin/quantized_test 2025-12-04T09:35:11.0003168Z inflating: build/bin/reduce_ops_test 2025-12-04T09:35:11.0052243Z inflating: build/bin/reportMemoryUsage_test 2025-12-04T09:35:11.0106020Z inflating: build/bin/scalar_tensor_test 2025-12-04T09:35:11.0161228Z inflating: build/bin/scalar_test 2025-12-04T09:35:11.0209829Z inflating: build/bin/StorageUtils_test 2025-12-04T09:35:11.0259300Z inflating: build/bin/stride_properties_test 2025-12-04T09:35:11.0332326Z inflating: build/bin/tensor_iterator_test 2025-12-04T09:35:11.0385586Z inflating: build/bin/test_parallel 2025-12-04T09:35:11.0434315Z inflating: build/bin/thread_init_test 2025-12-04T09:35:11.0487807Z inflating: build/bin/type_ptr_test 2025-12-04T09:35:11.0543923Z inflating: build/bin/type_test 2025-12-04T09:35:11.0594280Z inflating: build/bin/undefined_tensor_test 2025-12-04T09:35:11.0641888Z inflating: build/bin/verify_api_visibility 2025-12-04T09:35:11.0709060Z inflating: build/bin/legacy_vmap_test 2025-12-04T09:35:11.0759435Z inflating: build/bin/weakref_test 2025-12-04T09:35:11.0808734Z inflating: build/bin/wrapdim_test 2025-12-04T09:35:11.0857703Z inflating: build/bin/xla_tensor_test 2025-12-04T09:35:11.0914736Z inflating: build/bin/IListRef_test 2025-12-04T09:35:11.1011097Z inflating: build/bin/List_test 2025-12-04T09:35:11.1074504Z inflating: build/bin/KernelFunction_test 2025-12-04T09:35:11.1186123Z inflating: build/bin/kernel_function_legacy_test 2025-12-04T09:35:11.1272665Z inflating: build/bin/kernel_function_test 2025-12-04T09:35:11.1389202Z inflating: build/bin/kernel_lambda_legacy_test 2025-12-04T09:35:11.1483092Z inflating: build/bin/kernel_lambda_test 2025-12-04T09:35:11.1539966Z inflating: build/bin/kernel_stackbased_test 2025-12-04T09:35:11.1626749Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2025-12-04T09:35:11.1675915Z inflating: build/bin/CppSignature_test 2025-12-04T09:35:11.1730169Z inflating: build/bin/backend_fallback_test 2025-12-04T09:35:11.1775752Z inflating: build/bin/op_allowlist_test 2025-12-04T09:35:11.2051072Z inflating: build/bin/op_registration_test 2025-12-04T09:35:11.2116251Z inflating: build/bin/inline_container_test 2025-12-04T09:35:11.2423755Z inflating: build/bin/test_lazy 2025-12-04T09:35:11.2477002Z inflating: build/bin/TCPStoreTest 2025-12-04T09:35:11.2529233Z inflating: build/bin/HashStoreTest 2025-12-04T09:35:11.3503114Z inflating: build/bin/test_jit 2025-12-04T09:35:11.3565918Z inflating: build/bin/ProcessGroupGlooTest 2025-12-04T09:35:11.3620483Z inflating: build/bin/test_aoti_inference 2025-12-04T09:35:11.3622472Z inflating: build/bin/example_allreduce 2025-12-04T09:35:11.3675159Z inflating: build/bin/test_dist_autograd 2025-12-04T09:35:11.3740555Z inflating: build/bin/test_cpp_rpc 2025-12-04T09:35:11.3741085Z inflating: build/bin/parallel_benchmark 2025-12-04T09:35:11.4764788Z inflating: build/bin/test_api 2025-12-04T09:35:11.4769704Z inflating: build/bin/torch_shm_manager 2025-12-04T09:35:11.4770055Z creating: .additional_ci_files/ 2025-12-04T09:35:11.4823349Z inflating: .additional_ci_files/test-times.json 2025-12-04T09:35:11.5025930Z inflating: .additional_ci_files/test-class-times.json 2025-12-04T09:35:11.5074363Z ##[group]Run rm artifacts.zip 2025-12-04T09:35:11.5074568Z rm artifacts.zip 2025-12-04T09:35:11.5079324Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:35:11.5079587Z env: 2025-12-04T09:35:11.5079728Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:35:11.5079899Z HAS_NVIDIA_GPU: false 2025-12-04T09:35:11.5080062Z ##[endgroup] 2025-12-04T09:35:11.6005413Z ##[group]Run df -H 2025-12-04T09:35:11.6005594Z df -H 2025-12-04T09:35:11.6010261Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:35:11.6010511Z env: 2025-12-04T09:35:11.6010651Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:35:11.6010824Z HAS_NVIDIA_GPU: false 2025-12-04T09:35:11.6010987Z ##[endgroup] 2025-12-04T09:35:11.6051729Z Filesystem Size Used Avail Use% Mounted on 2025-12-04T09:35:11.6052217Z devtmpfs 4.2M 0 4.2M 0% /dev 2025-12-04T09:35:11.6052560Z tmpfs 67G 0 67G 0% /dev/shm 2025-12-04T09:35:11.6052897Z tmpfs 27G 791k 27G 1% /run 2025-12-04T09:35:11.6053670Z /dev/nvme0n1p1 215G 72G 144G 34% / 2025-12-04T09:35:11.6053992Z tmpfs 67G 13k 67G 1% /tmp 2025-12-04T09:35:11.6054245Z /dev/nvme0n1p128 11M 1.4M 9.2M 13% /boot/efi 2025-12-04T09:35:11.6080085Z Prepare all required actions 2025-12-04T09:35:11.6081301Z Getting action download info 2025-12-04T09:35:11.7722949Z ##[group]Run ./.github/actions/download-td-artifacts 2025-12-04T09:35:11.7723186Z with: 2025-12-04T09:35:11.7723333Z env: 2025-12-04T09:35:11.7723473Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:35:11.7723647Z HAS_NVIDIA_GPU: false 2025-12-04T09:35:11.7723809Z ##[endgroup] 2025-12-04T09:35:11.7795141Z ##[group]Run seemethere/download-artifact-s3@v4 2025-12-04T09:35:11.7795352Z with: 2025-12-04T09:35:11.7795499Z name: td_results 2025-12-04T09:35:11.7795651Z s3-bucket: gha-artifacts 2025-12-04T09:35:11.7795825Z region: us-east-1 2025-12-04T09:35:11.7795975Z env: 2025-12-04T09:35:11.7796111Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:35:11.7796277Z HAS_NVIDIA_GPU: false 2025-12-04T09:35:11.7796436Z ##[endgroup] 2025-12-04T09:35:12.2769627Z (node:48509) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2025-12-04T09:35:12.2770705Z 2025-12-04T09:35:12.2771435Z Please migrate your code to use AWS SDK for JavaScript (v3). 2025-12-04T09:35:12.2771840Z For more information, check the migration guide at https://a.co/7PzMCcy 2025-12-04T09:35:12.2772208Z (Use `node --trace-warnings ...` to show where the warning was created) 2025-12-04T09:35:12.3606633Z Found 0 objects with prefix pytorch/pytorch/19923066595/td_results/ 2025-12-04T09:35:12.3611070Z Artifact download has finished successfully 2025-12-04T09:35:12.3808334Z ##[group]Run mkdir -p .additional_ci_files 2025-12-04T09:35:12.3808578Z mkdir -p .additional_ci_files 2025-12-04T09:35:12.3808839Z mv td_results.json .additional_ci_files/td_results.json || true 2025-12-04T09:35:12.3813426Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:35:12.3813648Z env: 2025-12-04T09:35:12.3813802Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:35:12.3813975Z HAS_NVIDIA_GPU: false 2025-12-04T09:35:12.3814153Z ##[endgroup] 2025-12-04T09:35:12.3868524Z mv: cannot stat 'td_results.json': No such file or directory 2025-12-04T09:35:12.3911667Z ##[group]Run .github/scripts/parse_ref.py 2025-12-04T09:35:12.3911928Z .github/scripts/parse_ref.py 2025-12-04T09:35:12.3916096Z shell: /usr/bin/bash -e {0} 2025-12-04T09:35:12.3916279Z env: 2025-12-04T09:35:12.3916432Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:35:12.3916609Z HAS_NVIDIA_GPU: false 2025-12-04T09:35:12.3916767Z ##[endgroup] 2025-12-04T09:35:12.4283112Z Setting output branch=main 2025-12-04T09:35:12.4369175Z Prepare all required actions 2025-12-04T09:35:12.4369455Z Getting action download info 2025-12-04T09:35:12.5675932Z ##[group]Run ./.github/actions/filter-test-configs 2025-12-04T09:35:12.5676144Z with: 2025-12-04T09:35:12.5676464Z github-token: *** 2025-12-04T09:35:12.5682021Z test-matrix: {"include": [{"config": "cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_avx2_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_timm", "shard": 1, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_timm", "shard": 2, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}]} 2025-12-04T09:35:12.5687854Z job-name: periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_huggingface, 1, 1, linux.8xlarge.amx) 2025-12-04T09:35:12.5688182Z env: 2025-12-04T09:35:12.5688326Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:35:12.5688595Z HAS_NVIDIA_GPU: false 2025-12-04T09:35:12.5688749Z ##[endgroup] 2025-12-04T09:35:12.5720454Z ##[group]Run nick-fields/retry@v3.0.0 2025-12-04T09:35:12.5720655Z with: 2025-12-04T09:35:12.5720804Z shell: bash 2025-12-04T09:35:12.5720958Z timeout_minutes: 10 2025-12-04T09:35:12.5721113Z max_attempts: 5 2025-12-04T09:35:12.5721278Z retry_wait_seconds: 30 2025-12-04T09:35:12.5721737Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.2 2025-12-04T09:35:12.5722206Z polling_interval_seconds: 1 2025-12-04T09:35:12.5722377Z warning_on_retry: true 2025-12-04T09:35:12.5722544Z continue_on_error: false 2025-12-04T09:35:12.5722709Z env: 2025-12-04T09:35:12.5722846Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:35:12.5723017Z HAS_NVIDIA_GPU: false 2025-12-04T09:35:12.5723311Z GITHUB_TOKEN: *** 2025-12-04T09:35:12.5723468Z ##[endgroup] 2025-12-04T09:35:12.6771528Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.2 2025-12-04T09:35:12.8448384Z Defaulting to user installation because normal site-packages is not writeable 2025-12-04T09:35:13.6003863Z Collecting requests==2.27.1 2025-12-04T09:35:13.6146999Z Downloading requests-2.27.1-py2.py3-none-any.whl (63 kB) 2025-12-04T09:35:13.8488239Z Collecting pyyaml==6.0.2 2025-12-04T09:35:13.8521911Z Downloading PyYAML-6.0.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (737 kB) 2025-12-04T09:35:13.9865589Z Collecting certifi>=2017.4.17 2025-12-04T09:35:13.9903221Z Downloading certifi-2025.11.12-py3-none-any.whl (159 kB) 2025-12-04T09:35:14.0374363Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (1.25.10) 2025-12-04T09:35:14.3212966Z Collecting charset-normalizer~=2.0.0 2025-12-04T09:35:14.3248107Z Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB) 2025-12-04T09:35:14.3706504Z Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (2.10) 2025-12-04T09:35:14.4270855Z Installing collected packages: charset-normalizer, certifi, requests, pyyaml 2025-12-04T09:35:14.8103261Z Successfully installed certifi-2025.11.12 charset-normalizer-2.0.12 pyyaml-6.0.2 requests-2.27.1 2025-12-04T09:35:15.6439994Z Command completed after 1 attempt(s). 2025-12-04T09:35:15.6512844Z ##[group]Run set -x 2025-12-04T09:35:15.6513115Z set -x 2025-12-04T09:35:15.6513412Z  2025-12-04T09:35:15.6513657Z # Use relative path here as this could be checked out anywhere, not necessarily 2025-12-04T09:35:15.6513943Z # in runner workspace 2025-12-04T09:35:15.6514189Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2025-12-04T09:35:15.6518765Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:35:15.6519005Z env: 2025-12-04T09:35:15.6519151Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:35:15.6519328Z HAS_NVIDIA_GPU: false 2025-12-04T09:35:15.6519505Z ##[endgroup] 2025-12-04T09:35:15.6542431Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2025-12-04T09:35:15.6679252Z Setting output branch=main 2025-12-04T09:35:15.6726530Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2025-12-04T09:35:15.6726812Z echo "Workflow: ${GITHUB_WORKFLOW}" 2025-12-04T09:35:15.6727013Z echo "Job name: ${JOB_NAME}" 2025-12-04T09:35:15.6727216Z  2025-12-04T09:35:15.6727454Z # Use relative path here as this could be checked out anywhere, not necessarily 2025-12-04T09:35:15.6727729Z # in runner workspace 2025-12-04T09:35:15.6727989Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2025-12-04T09:35:15.6728270Z  --workflow "${GITHUB_WORKFLOW}" \ 2025-12-04T09:35:15.6728477Z  --job-name "${JOB_NAME}" \ 2025-12-04T09:35:15.6734320Z  --test-matrix "{"include": [{"config": "cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_avx2_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_timm", "shard": 1, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_timm", "shard": 2, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}]}" \ 2025-12-04T09:35:15.6740262Z  --selected-test-configs "" \ 2025-12-04T09:35:15.6740489Z  --pr-number "${PR_NUMBER}" \ 2025-12-04T09:35:15.6740711Z  --tag "${TAG}" \ 2025-12-04T09:35:15.6740903Z  --event-name "${EVENT_NAME}" \ 2025-12-04T09:35:15.6741120Z  --schedule "${SCHEDULE}" \ 2025-12-04T09:35:15.6741328Z  --branch "${HEAD_BRANCH}" 2025-12-04T09:35:15.6745888Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:35:15.6746141Z env: 2025-12-04T09:35:15.6746301Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:35:15.6746527Z HAS_NVIDIA_GPU: false 2025-12-04T09:35:15.6746890Z GITHUB_TOKEN: *** 2025-12-04T09:35:15.6747249Z JOB_NAME: periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_huggingface, 1, 1, linux.8xlarge.amx) 2025-12-04T09:35:15.6747654Z PR_NUMBER: 2025-12-04T09:35:15.6747805Z TAG: 2025-12-04T09:35:15.6747971Z EVENT_NAME: schedule 2025-12-04T09:35:15.6748140Z SCHEDULE: 45 0,4,8,12,16,20 * * 1-5 2025-12-04T09:35:15.6748318Z HEAD_BRANCH: main 2025-12-04T09:35:15.6748473Z ##[endgroup] 2025-12-04T09:35:15.6770186Z Workflow: inductor-periodic 2025-12-04T09:35:15.6775487Z Job name: periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_huggingface, 1, 1, linux.8xlarge.amx) 2025-12-04T09:35:15.8879148Z Setting output keep-going=True 2025-12-04T09:35:15.8879494Z Setting output ci-verbose-test-logs=False 2025-12-04T09:35:15.8879759Z Setting output ci-test-showlocals=False 2025-12-04T09:35:15.8880030Z Setting output ci-no-test-timeout=False 2025-12-04T09:35:15.8880316Z Setting output ci-no-td=False 2025-12-04T09:35:15.8880553Z Setting output ci-td-distributed=False 2025-12-04T09:35:15.8880803Z Setting output is-unstable=False 2025-12-04T09:35:15.8881037Z Setting output reenabled-issues= 2025-12-04T09:35:15.8887221Z Setting output test-matrix={"include": [{"config": "cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_avx2_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_timm", "shard": 1, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_timm", "shard": 2, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}]} 2025-12-04T09:35:15.8893567Z Setting output is-test-matrix-empty=False 2025-12-04T09:35:15.9004115Z ##[group]Run echo "Filtered matrix:" 2025-12-04T09:35:15.9004364Z echo "Filtered matrix:" 2025-12-04T09:35:15.9010400Z echo "{"include": [{"config": "cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_avx2_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_timm", "shard": 1, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_timm", "shard": 2, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}]}" 2025-12-04T09:35:15.9016351Z  2025-12-04T09:35:15.9016506Z echo 2025-12-04T09:35:15.9016698Z echo "Is the current job unstable? False" 2025-12-04T09:35:15.9016908Z  2025-12-04T09:35:15.9017054Z echo 2025-12-04T09:35:15.9017233Z echo "Is keep-going label set? True" 2025-12-04T09:35:15.9017431Z  2025-12-04T09:35:15.9017576Z echo 2025-12-04T09:35:15.9017741Z echo "Reenabled issues? " 2025-12-04T09:35:15.9022480Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:35:15.9022729Z env: 2025-12-04T09:35:15.9022890Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:35:15.9023074Z HAS_NVIDIA_GPU: false 2025-12-04T09:35:15.9023244Z ##[endgroup] 2025-12-04T09:35:15.9046333Z Filtered matrix: 2025-12-04T09:35:15.9053763Z {include: [{config: cpu_inductor_huggingface, shard: 1, num_shards: 1, runner: linux.8xlarge.amx}, {config: cpu_inductor_timm, shard: 1, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_inductor_timm, shard: 2, num_shards: 2, runner: linux.8xlarge.amx}, {config: dynamic_cpu_inductor_huggingface, shard: 1, num_shards: 1, runner: linux.8xlarge.amx}, {config: dynamic_cpu_inductor_timm, shard: 1, num_shards: 2, runner: linux.8xlarge.amx}, {config: dynamic_cpu_inductor_timm, shard: 2, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_inductor_freezing_avx2_huggingface, shard: 1, num_shards: 1, runner: linux.10xlarge.avx2}, {config: cpu_inductor_freezing_avx2_torchbench, shard: 1, num_shards: 2, runner: linux.10xlarge.avx2}, {config: cpu_inductor_freezing_avx2_torchbench, shard: 2, num_shards: 2, runner: linux.10xlarge.avx2}, {config: cpu_inductor_freezing_avx2_timm, shard: 1, num_shards: 2, runner: linux.10xlarge.avx2}, {config: cpu_inductor_freezing_avx2_timm, shard: 2, num_shards: 2, runner: linux.10xlarge.avx2}, {config: cpu_inductor_freezing_huggingface, shard: 1, num_shards: 1, runner: linux.8xlarge.amx}, {config: cpu_inductor_freezing_timm, shard: 1, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_inductor_freezing_timm, shard: 2, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_inductor_freezing_torchbench, shard: 1, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_inductor_freezing_torchbench, shard: 2, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_inductor_amp_freezing_huggingface, shard: 1, num_shards: 1, runner: linux.8xlarge.amx}, {config: cpu_inductor_amp_freezing_timm, shard: 1, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_inductor_amp_freezing_timm, shard: 2, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_inductor_amp_freezing_torchbench, shard: 1, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_inductor_amp_freezing_torchbench, shard: 2, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_aot_inductor_freezing_huggingface, shard: 1, num_shards: 1, runner: linux.8xlarge.amx}, {config: cpu_aot_inductor_freezing_timm, shard: 1, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_aot_inductor_freezing_timm, shard: 2, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_aot_inductor_freezing_torchbench, shard: 1, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_aot_inductor_freezing_torchbench, shard: 2, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_aot_inductor_amp_freezing_torchbench, shard: 1, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_aot_inductor_amp_freezing_torchbench, shard: 2, num_shards: 2, runner: linux.8xlarge.amx}, {config: dynamic_cpu_aot_inductor_freezing_torchbench, shard: 1, num_shards: 2, runner: linux.8xlarge.amx}, {config: dynamic_cpu_aot_inductor_freezing_torchbench, shard: 2, num_shards: 2, runner: linux.8xlarge.amx}, {config: dynamic_cpu_aot_inductor_amp_freezing_torchbench, shard: 1, num_shards: 2, runner: linux.8xlarge.amx}, {config: dynamic_cpu_aot_inductor_amp_freezing_torchbench, shard: 2, num_shards: 2, runner: linux.8xlarge.amx}]} 2025-12-04T09:35:15.9059676Z 2025-12-04T09:35:15.9059765Z Is the current job unstable? False 2025-12-04T09:35:15.9059927Z 2025-12-04T09:35:15.9060026Z Is keep-going label set? True 2025-12-04T09:35:15.9060182Z 2025-12-04T09:35:15.9060261Z Reenabled issues? 2025-12-04T09:35:15.9121464Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2025-12-04T09:35:15.9121789Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2025-12-04T09:35:15.9125576Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:35:15.9125797Z env: 2025-12-04T09:35:15.9125951Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:35:15.9126124Z HAS_NVIDIA_GPU: false 2025-12-04T09:35:15.9126281Z JOB_TIMEOUT: 240 2025-12-04T09:35:15.9126436Z ##[endgroup] 2025-12-04T09:35:15.9204975Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2025-12-04T09:35:15.9205291Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2025-12-04T09:35:15.9205586Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2025-12-04T09:35:15.9209408Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:35:15.9209631Z env: 2025-12-04T09:35:15.9209794Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:35:15.9209968Z HAS_NVIDIA_GPU: false 2025-12-04T09:35:15.9210123Z ##[endgroup] 2025-12-04T09:35:15.9318153Z ##[group]Run set -x 2025-12-04T09:35:15.9318387Z set -x 2025-12-04T09:35:15.9318539Z  2025-12-04T09:35:15.9318713Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2025-12-04T09:35:15.9318955Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2025-12-04T09:35:15.9319202Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2025-12-04T09:35:15.9319429Z  TEST_COMMAND=.ci/onnx/test.sh 2025-12-04T09:35:15.9319611Z else 2025-12-04T09:35:15.9319782Z  TEST_COMMAND=.ci/pytorch/test.sh 2025-12-04T09:35:15.9319978Z fi 2025-12-04T09:35:15.9320109Z  2025-12-04T09:35:15.9320278Z # Leaving 1GB for the runner and other things 2025-12-04T09:35:15.9320620Z TOTAL_AVAILABLE_MEMORY_IN_GB=$(awk '/MemTotal/ { printf "%.3f \n", $2/1024/1024 - 1 }' /proc/meminfo) 2025-12-04T09:35:15.9321224Z # https://docs.docker.com/engine/containers/resource_constraints/#--memory-swap-details, the 3GB swap 2025-12-04T09:35:15.9321627Z # comes from https://github.com/pytorch/test-infra/pull/6058 2025-12-04T09:35:15.9321936Z TOTAL_MEMORY_WITH_SWAP=$(("${TOTAL_AVAILABLE_MEMORY_IN_GB%.*}" + 3)) 2025-12-04T09:35:15.9322289Z  2025-12-04T09:35:15.9322461Z if [[ ${BUILD_ENVIRONMENT} == *"s390x"* ]]; then 2025-12-04T09:35:15.9322678Z  SHM_OPTS= 2025-12-04T09:35:15.9322846Z  JENKINS_USER= 2025-12-04T09:35:15.9323065Z  # ensure that docker container cleanly exits in 12 hours 2025-12-04T09:35:15.9323391Z  # if for some reason cleanup action doesn't stop container 2025-12-04T09:35:15.9323637Z  # when job is cancelled 2025-12-04T09:35:15.9323838Z  DOCKER_SHELL_CMD="sleep 12h" 2025-12-04T09:35:15.9324043Z  USED_IMAGE="${DOCKER_IMAGE_S390X}" 2025-12-04T09:35:15.9324240Z else 2025-12-04T09:35:15.9324407Z  SHM_OPTS="--shm-size=${SHM_SIZE}" 2025-12-04T09:35:15.9324613Z  JENKINS_USER="--user jenkins" 2025-12-04T09:35:15.9324808Z  DOCKER_SHELL_CMD= 2025-12-04T09:35:15.9324991Z  USED_IMAGE="${DOCKER_IMAGE}" 2025-12-04T09:35:15.9325165Z fi 2025-12-04T09:35:15.9325302Z  2025-12-04T09:35:15.9325514Z # detached container should get cleaned up by teardown_ec2_linux 2025-12-04T09:35:15.9325828Z # TODO: Stop building test binaries as part of the build phase 2025-12-04T09:35:15.9326181Z # Used for GPU_FLAG, SHM_OPTS, JENKINS_USER and DOCKER_SHELL_CMD since that doesn't play nice 2025-12-04T09:35:15.9326489Z # shellcheck disable=SC2086,SC2090 2025-12-04T09:35:15.9326697Z container_name=$(docker run \ 2025-12-04T09:35:15.9326884Z  ${GPU_FLAG:-} \ 2025-12-04T09:35:15.9327077Z  ${SCCACHE_SERVER_PORT_DOCKER_FLAG:-} \ 2025-12-04T09:35:15.9327350Z  -e BUILD_ENVIRONMENT \ 2025-12-04T09:35:15.9327539Z  -e PR_NUMBER \ 2025-12-04T09:35:15.9327709Z  -e GITHUB_ACTIONS \ 2025-12-04T09:35:15.9327904Z  -e GITHUB_REPOSITORY \ 2025-12-04T09:35:15.9328088Z  -e GITHUB_WORKFLOW \ 2025-12-04T09:35:15.9328261Z  -e GITHUB_JOB \ 2025-12-04T09:35:15.9328431Z  -e GITHUB_RUN_ID \ 2025-12-04T09:35:15.9328608Z  -e GITHUB_RUN_NUMBER \ 2025-12-04T09:35:15.9328783Z  -e GITHUB_RUN_ATTEMPT \ 2025-12-04T09:35:15.9328999Z  -e JOB_ID \ 2025-12-04T09:35:15.9329162Z  -e JOB_NAME \ 2025-12-04T09:35:15.9329327Z  -e BASE_SHA \ 2025-12-04T09:35:15.9329478Z  -e BRANCH \ 2025-12-04T09:35:15.9329634Z  -e SHA1 \ 2025-12-04T09:35:15.9329791Z  -e AWS_DEFAULT_REGION \ 2025-12-04T09:35:15.9329966Z  -e IN_WHEEL_TEST \ 2025-12-04T09:35:15.9330133Z  -e SHARD_NUMBER \ 2025-12-04T09:35:15.9330304Z  -e TEST_CONFIG \ 2025-12-04T09:35:15.9330466Z  -e NUM_TEST_SHARDS \ 2025-12-04T09:35:15.9330726Z  -e REENABLED_ISSUES \ 2025-12-04T09:35:15.9330917Z  -e CONTINUE_THROUGH_ERROR \ 2025-12-04T09:35:15.9331100Z  -e VERBOSE_TEST_LOGS \ 2025-12-04T09:35:15.9331282Z  -e TEST_SHOWLOCALS \ 2025-12-04T09:35:15.9331460Z  -e NO_TEST_TIMEOUT \ 2025-12-04T09:35:15.9331630Z  -e NO_TD \ 2025-12-04T09:35:15.9331790Z  -e TD_DISTRIBUTED \ 2025-12-04T09:35:15.9331963Z  -e PR_LABELS \ 2025-12-04T09:35:15.9332152Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2025-12-04T09:35:15.9332349Z  -e SCCACHE_BUCKET \ 2025-12-04T09:35:15.9332521Z  -e SCCACHE_REGION \ 2025-12-04T09:35:15.9332689Z  -e XLA_CUDA \ 2025-12-04T09:35:15.9332862Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2025-12-04T09:35:15.9333079Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2025-12-04T09:35:15.9333345Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2025-12-04T09:35:15.9333566Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2025-12-04T09:35:15.9333769Z  -e HUGGING_FACE_HUB_TOKEN \ 2025-12-04T09:35:15.9333969Z  -e VLLM_TEST_HUGGING_FACE_TOKEN \ 2025-12-04T09:35:15.9334174Z  -e SCRIBE_GRAPHQL_ACCESS_TOKEN \ 2025-12-04T09:35:15.9334359Z  -e DASHBOARD_TAG \ 2025-12-04T09:35:15.9334540Z  -e ARTIFACTS_FILE_SUFFIX \ 2025-12-04T09:35:15.9334759Z  --memory="${TOTAL_AVAILABLE_MEMORY_IN_GB%.*}g" \ 2025-12-04T09:35:15.9334997Z  --memory-swap="${TOTAL_MEMORY_WITH_SWAP}g" \ 2025-12-04T09:35:15.9335241Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2025-12-04T09:35:15.9335477Z  --security-opt seccomp=unconfined \ 2025-12-04T09:35:15.9335678Z  --cap-add=SYS_PTRACE \ 2025-12-04T09:35:15.9335853Z  --ipc=host \ 2025-12-04T09:35:15.9336017Z  ${SHM_OPTS} \ 2025-12-04T09:35:15.9336182Z  --tty \ 2025-12-04T09:35:15.9336325Z  --detach \ 2025-12-04T09:35:15.9336497Z  --name="${container_name}" \ 2025-12-04T09:35:15.9336688Z  ${JENKINS_USER} \ 2025-12-04T09:35:15.9336892Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2025-12-04T09:35:15.9337127Z  -w /var/lib/jenkins/workspace \ 2025-12-04T09:35:15.9337318Z  "${USED_IMAGE}" \ 2025-12-04T09:35:15.9337487Z  ${DOCKER_SHELL_CMD} 2025-12-04T09:35:15.9337643Z ) 2025-12-04T09:35:15.9337850Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2025-12-04T09:35:15.9338087Z  2025-12-04T09:35:15.9338250Z if [[ ${BUILD_ENVIRONMENT} == *"s390x"* ]]; then 2025-12-04T09:35:15.9338588Z  docker exec -t "${container_name}" sh -c "python3 -m pip install -r .ci/docker/requirements-ci.txt" 2025-12-04T09:35:15.9338887Z fi 2025-12-04T09:35:15.9339026Z  2025-12-04T09:35:15.9339302Z docker exec -t "${container_name}" sh -c "python3 -m pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2025-12-04T09:35:15.9343357Z shell: /usr/bin/bash -e {0} 2025-12-04T09:35:15.9343523Z env: 2025-12-04T09:35:15.9343669Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:35:15.9343846Z HAS_NVIDIA_GPU: false 2025-12-04T09:35:15.9344041Z BUILD_ENVIRONMENT: linux-jammy-py3.10-gcc11-build 2025-12-04T09:35:15.9344261Z PR_NUMBER: 2025-12-04T09:35:15.9344418Z GITHUB_REPOSITORY: pytorch/pytorch 2025-12-04T09:35:15.9344610Z GITHUB_WORKFLOW: inductor-periodic 2025-12-04T09:35:15.9344795Z GITHUB_JOB: test 2025-12-04T09:35:15.9344950Z GITHUB_RUN_ID: 19923066595 2025-12-04T09:35:15.9345114Z GITHUB_RUN_NUMBER: 67027 2025-12-04T09:35:15.9345282Z GITHUB_RUN_ATTEMPT: 1 2025-12-04T09:35:15.9345439Z JOB_ID: 57118563323 2025-12-04T09:35:15.9345774Z JOB_NAME: periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_huggingface, 1, 1, linux.8xlarge.amx) 2025-12-04T09:35:15.9346120Z BRANCH: main 2025-12-04T09:35:15.9346293Z SHA1: ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:35:15.9346596Z BASE_SHA: ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:35:15.9346821Z TEST_CONFIG: cpu_inductor_freezing_huggingface 2025-12-04T09:35:15.9347022Z SHARD_NUMBER: 1 2025-12-04T09:35:15.9347174Z NUM_TEST_SHARDS: 1 2025-12-04T09:35:15.9347320Z EXTRA_FLAGS: 2025-12-04T09:35:15.9347472Z OP_BENCHMARK_TESTS: 2025-12-04T09:35:15.9347633Z REENABLED_ISSUES: 2025-12-04T09:35:15.9347786Z CONTINUE_THROUGH_ERROR: True 2025-12-04T09:35:15.9347964Z VERBOSE_TEST_LOGS: False 2025-12-04T09:35:15.9348129Z TEST_SHOWLOCALS: False 2025-12-04T09:35:15.9348291Z NO_TEST_TIMEOUT: False 2025-12-04T09:35:15.9348439Z NO_TD: False 2025-12-04T09:35:15.9348584Z TD_DISTRIBUTED: False 2025-12-04T09:35:15.9348779Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2025-12-04T09:35:15.9348990Z SCCACHE_REGION: us-east-1 2025-12-04T09:35:15.9349197Z SHM_SIZE: 1g 2025-12-04T09:35:15.9349677Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:35:15.9350487Z DOCKER_IMAGE_S390X: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:35:15.9350975Z XLA_CUDA: 2025-12-04T09:35:15.9351201Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2025-12-04T09:35:15.9351476Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2025-12-04T09:35:15.9351676Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2025-12-04T09:35:15.9351866Z DASHBOARD_TAG: 2025-12-04T09:35:15.9352197Z VLLM_TEST_HUGGING_FACE_TOKEN: *** 2025-12-04T09:35:15.9352457Z HUGGING_FACE_HUB_TOKEN: *** 2025-12-04T09:35:15.9352712Z SCRIBE_GRAPHQL_ACCESS_TOKEN: *** 2025-12-04T09:35:15.9353094Z ARTIFACTS_FILE_SUFFIX: test-cpu_inductor_freezing_huggingface-1-1-linux.8xlarge.amx_57118563323 2025-12-04T09:35:15.9353416Z ##[endgroup] 2025-12-04T09:35:15.9376959Z + [[ cpu_inductor_freezing_huggingface == \m\u\l\t\i\g\p\u ]] 2025-12-04T09:35:15.9378827Z + [[ linux-jammy-py3.10-gcc11-build == *onnx* ]] 2025-12-04T09:35:15.9379189Z + TEST_COMMAND=.ci/pytorch/test.sh 2025-12-04T09:35:15.9383853Z ++ awk '/MemTotal/ { printf "%.3f \n", $2/1024/1024 - 1 }' /proc/meminfo 2025-12-04T09:35:16.0351468Z + TOTAL_AVAILABLE_MEMORY_IN_GB='122.780 ' 2025-12-04T09:35:16.0351950Z + TOTAL_MEMORY_WITH_SWAP=125 2025-12-04T09:35:16.0352207Z + [[ linux-jammy-py3.10-gcc11-build == *\s\3\9\0\x* ]] 2025-12-04T09:35:16.0352443Z + SHM_OPTS=--shm-size=1g 2025-12-04T09:35:16.0352751Z + JENKINS_USER='--user jenkins' 2025-12-04T09:35:16.0352950Z + DOCKER_SHELL_CMD= 2025-12-04T09:35:16.0353581Z + USED_IMAGE=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:35:16.0363626Z +++ nproc --ignore=2 2025-12-04T09:35:16.0522114Z ++ docker run -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=30 -e SCCACHE_BUCKET -e SCCACHE_REGION -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e VLLM_TEST_HUGGING_FACE_TOKEN -e SCRIBE_GRAPHQL_ACCESS_TOKEN -e DASHBOARD_TAG -e ARTIFACTS_FILE_SUFFIX --memory=122g --memory-swap=125g --env-file=/tmp/github_env_19923066595 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=1g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:35:27.8819745Z + container_name=267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:35:27.8820330Z + echo DOCKER_CONTAINER_ID=267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:35:27.8824743Z + [[ linux-jammy-py3.10-gcc11-build == *\s\3\9\0\x* ]] 2025-12-04T09:35:27.8826940Z ++ echo dist/torch-2.10.0a0+gitffd9b0f-cp310-cp310-linux_x86_64.whl 2025-12-04T09:35:27.8828032Z + docker exec -t 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb sh -c 'python3 -m pip install dist/torch-2.10.0a0+gitffd9b0f-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2025-12-04T09:35:28.2875199Z Processing ./dist/torch-2.10.0a0+gitffd9b0f-cp310-cp310-linux_x86_64.whl (from torch==2.10.0a0+gitffd9b0f) 2025-12-04T09:35:28.4886751Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.10.0a0+gitffd9b0f->torch==2.10.0a0+gitffd9b0f) (3.18.0) 2025-12-04T09:35:28.4887575Z Requirement already satisfied: typing-extensions>=4.10.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.10.0a0+gitffd9b0f->torch==2.10.0a0+gitffd9b0f) (4.15.0) 2025-12-04T09:35:28.4890293Z Requirement already satisfied: sympy>=1.13.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.10.0a0+gitffd9b0f->torch==2.10.0a0+gitffd9b0f) (1.13.3) 2025-12-04T09:35:28.4897351Z Requirement already satisfied: networkx>=2.5.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.10.0a0+gitffd9b0f->torch==2.10.0a0+gitffd9b0f) (2.8.8) 2025-12-04T09:35:28.4898145Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.10.0a0+gitffd9b0f->torch==2.10.0a0+gitffd9b0f) (3.1.6) 2025-12-04T09:35:28.4898926Z Requirement already satisfied: fsspec>=0.8.5 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.10.0a0+gitffd9b0f->torch==2.10.0a0+gitffd9b0f) (2025.10.0) 2025-12-04T09:35:28.4909020Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.10.0a0+gitffd9b0f->torch==2.10.0a0+gitffd9b0f) (3.3.0) 2025-12-04T09:35:28.5173297Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.10.0a0+gitffd9b0f->torch==2.10.0a0+gitffd9b0f) (1.22.4) 2025-12-04T09:35:28.5188012Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy>=1.13.3->torch==2.10.0a0+gitffd9b0f->torch==2.10.0a0+gitffd9b0f) (1.3.0) 2025-12-04T09:35:28.5226391Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.10.0a0+gitffd9b0f->torch==2.10.0a0+gitffd9b0f) (3.0.3) 2025-12-04T09:35:29.2427342Z Installing collected packages: torch 2025-12-04T09:35:36.1266059Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2025-12-04T09:35:36.1267057Z dall-e 0.1 requires torchvision, which is not installed. 2025-12-04T09:35:36.1267339Z effdet 0.4.1 requires torchvision, which is not installed. 2025-12-04T09:35:36.1267640Z python-doctr 1.0.0 requires torchvision>=0.15.0, which is not installed. 2025-12-04T09:35:36.1268015Z pytorch-labs-segment-anything-fast 0.2 requires torchao, which is not installed. 2025-12-04T09:35:36.1268460Z pytorch-labs-segment-anything-fast 0.2 requires torchvision>=0.17.0.dev20231026, which is not installed. 2025-12-04T09:35:36.1268930Z timm 1.0.22 requires torchvision, which is not installed. 2025-12-04T09:35:36.1269249Z Successfully installed torch-2.10.0a0+gitffd9b0f 2025-12-04T09:35:36.2372140Z + export TERM=vt100 2025-12-04T09:35:36.2374571Z + TERM=vt100 2025-12-04T09:35:36.2378586Z ++ dirname .ci/pytorch/test.sh 2025-12-04T09:35:36.2378808Z + source .ci/pytorch/common.sh 2025-12-04T09:35:36.2387805Z +++ dirname .ci/pytorch/common.sh 2025-12-04T09:35:36.2393177Z ++ source .ci/pytorch/common_utils.sh 2025-12-04T09:35:36.2393474Z +++ declare -f -t trap_add 2025-12-04T09:35:36.2394011Z ++ set -ex -o pipefail 2025-12-04T09:35:36.2394238Z ++ [[ linux-jammy-py3.10-gcc11-build == *rocm* ]] 2025-12-04T09:35:36.2394489Z ++ BUILD_TEST_LIBTORCH=0 2025-12-04T09:35:36.2394693Z ++ dirname .ci/pytorch/test.sh 2025-12-04T09:35:36.2409888Z + source .ci/pytorch/common-build.sh 2025-12-04T09:35:36.2414582Z ++ [[ linux-jammy-py3.10-gcc11-build != *win-* ]] 2025-12-04T09:35:36.2418765Z ++++ dirname .ci/pytorch/common-build.sh 2025-12-04T09:35:36.2426251Z +++ cd .ci/pytorch 2025-12-04T09:35:36.2431121Z +++ pwd -P 2025-12-04T09:35:36.2435927Z ++ script_dir=/var/lib/jenkins/workspace/.ci/pytorch 2025-12-04T09:35:36.2440611Z ++ [[ linux-jammy-py3.10-gcc11-build == *-pch* ]] 2025-12-04T09:35:36.2440879Z ++ which sccache 2025-12-04T09:35:36.2447175Z ++ [[ -z ossci-compiler-cache-circleci-v2 ]] 2025-12-04T09:35:36.2447457Z ++ sccache --stop-server 2025-12-04T09:35:36.2470933Z ++ true 2025-12-04T09:35:36.2473645Z ++ rm -f /var/lib/jenkins/sccache_error.log 2025-12-04T09:35:36.2474014Z ++ trap_add sccache_epilogue EXIT 2025-12-04T09:35:36.2474375Z ++ trap_add_cmd=sccache_epilogue 2025-12-04T09:35:36.2474580Z ++ shift 2025-12-04T09:35:36.2474735Z ++ for trap_add_name in "$@" 2025-12-04T09:35:36.2489067Z ++++ trap -p EXIT 2025-12-04T09:35:36.2494458Z +++ eval 'extract_trap_cmd ' 2025-12-04T09:35:36.2494838Z ++++ extract_trap_cmd 2025-12-04T09:35:36.2495029Z ++++ printf '%s\n' '' 2025-12-04T09:35:36.2495200Z +++ printf '%s\n' sccache_epilogue 2025-12-04T09:35:36.2495521Z ++ trap -- ' 2025-12-04T09:35:36.2495696Z sccache_epilogue' EXIT 2025-12-04T09:35:36.2496198Z ++ [[ -n 1 ]] 2025-12-04T09:35:36.2500945Z ++ echo 'Skipping sccache server initialization, setting environment variables' 2025-12-04T09:35:36.2501527Z Skipping sccache server initialization, setting environment variables 2025-12-04T09:35:36.2501934Z ++ export SCCACHE_IDLE_TIMEOUT=0 2025-12-04T09:35:36.2502152Z ++ SCCACHE_IDLE_TIMEOUT=0 2025-12-04T09:35:36.2502454Z ++ export SCCACHE_ERROR_LOG=/var/lib/jenkins/sccache_error.log 2025-12-04T09:35:36.2503332Z ++ SCCACHE_ERROR_LOG=/var/lib/jenkins/sccache_error.log 2025-12-04T09:35:36.2512933Z ++ export RUST_LOG=sccache::server=error 2025-12-04T09:35:36.2513374Z ++ RUST_LOG=sccache::server=error 2025-12-04T09:35:36.2513589Z ++ sccache --zero-stats 2025-12-04T09:35:36.3990169Z Statistics zeroed. 2025-12-04T09:35:36.3994966Z ++ which ccache 2025-12-04T09:35:36.4028113Z + [[ linux-jammy-py3.10-gcc11-build != *rocm* ]] 2025-12-04T09:35:36.4028425Z + [[ linux-jammy-py3.10-gcc11-build != *s390x* ]] 2025-12-04T09:35:36.4028672Z + [[ -d /var/lib/jenkins/workspace ]] 2025-12-04T09:35:36.4028877Z ++ stat -c %u /var/lib/jenkins/workspace 2025-12-04T09:35:36.4043138Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2025-12-04T09:35:36.4043425Z + trap_add cleanup_workspace EXIT 2025-12-04T09:35:36.4043635Z + trap_add_cmd=cleanup_workspace 2025-12-04T09:35:36.4043818Z + shift 2025-12-04T09:35:36.4043971Z + for trap_add_name in "$@" 2025-12-04T09:35:36.4047752Z +++ trap -p EXIT 2025-12-04T09:35:36.4053557Z ++ eval 'extract_trap_cmd trap -- '\'' 2025-12-04T09:35:36.4058106Z sccache_epilogue'\'' EXIT' 2025-12-04T09:35:36.4063293Z +++ extract_trap_cmd trap -- ' 2025-12-04T09:35:36.4067019Z sccache_epilogue' EXIT 2025-12-04T09:35:36.4067244Z +++ printf '%s\n' ' 2025-12-04T09:35:36.4067399Z sccache_epilogue' 2025-12-04T09:35:36.4067573Z ++ printf '%s\n' cleanup_workspace 2025-12-04T09:35:36.4067766Z + trap -- ' 2025-12-04T09:35:36.4067902Z sccache_epilogue 2025-12-04T09:35:36.4068060Z cleanup_workspace' EXIT 2025-12-04T09:35:36.4068263Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2025-12-04T09:35:36.8454166Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2025-12-04T09:35:36.8469501Z + [[ linux-jammy-py3.10-gcc11-build == *cuda* ]] 2025-12-04T09:35:36.8470069Z + echo 'Environment variables:' 2025-12-04T09:35:36.8470274Z Environment variables: 2025-12-04T09:35:36.8470432Z + env 2025-12-04T09:35:36.8479562Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2025-12-04T09:35:36.8481513Z CONTINUE_THROUGH_ERROR=True 2025-12-04T09:35:36.8481905Z BUILD_ENVIRONMENT=linux-jammy-py3.10-gcc11-build 2025-12-04T09:35:36.8482509Z VLLM_TEST_HUGGING_FACE_TOKEN=*** 2025-12-04T09:35:36.8482818Z HOSTNAME=267402d4cbe5 2025-12-04T09:35:36.8483205Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_ca6c2ac3-ce1c-41a2-95f1-cf47e926186e 2025-12-04T09:35:36.8484174Z GITHUB_ACTION=__run_3 2025-12-04T09:35:36.8484444Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2025-12-04T09:35:36.8484650Z GITHUB_RUN_NUMBER=67027 2025-12-04T09:35:36.8484893Z TEST_CONFIG=cpu_inductor_freezing_huggingface 2025-12-04T09:35:36.8485497Z GITHUB_REPOSITORY_OWNER_ID=21003710 2025-12-04T09:35:36.8485803Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2025-12-04T09:35:36.8486105Z SCCACHE_IDLE_TIMEOUT=0 2025-12-04T09:35:36.8486522Z SCRIBE_GRAPHQL_ACCESS_TOKEN=*** 2025-12-04T09:35:36.8486808Z GITHUB_TRIGGERING_ACTOR=pytorchmergebot 2025-12-04T09:35:36.8487070Z GITHUB_REF_TYPE=branch 2025-12-04T09:35:36.8487348Z BASE_SHA=ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:35:36.8487565Z XLA_CUDA= 2025-12-04T09:35:36.8487713Z NCCL_LIB_DIR=/usr/local/cuda/lib64/ 2025-12-04T09:35:36.8487980Z HUGGING_FACE_HUB_TOKEN=*** 2025-12-04T09:35:36.8488419Z *** 2025-12-04T09:35:36.8488572Z GITHUB_REPOSITORY_ID=65600975 2025-12-04T09:35:36.8488745Z GITHUB_ACTIONS=true 2025-12-04T09:35:36.8488937Z SCCACHE_ERROR_LOG=/var/lib/jenkins/sccache_error.log 2025-12-04T09:35:36.8489182Z SHA1=ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:35:36.8489408Z GITHUB_SHA=ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:35:36.8489800Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/heads/main 2025-12-04T09:35:36.8490110Z UCC_HOME=/usr 2025-12-04T09:35:36.8490263Z VERBOSE_TEST_LOGS=False 2025-12-04T09:35:36.8490424Z GITHUB_REF=refs/heads/main 2025-12-04T09:35:36.8490591Z SHARD_NUMBER=1 2025-12-04T09:35:36.8490747Z GITHUB_REF_PROTECTED=true 2025-12-04T09:35:36.8490909Z HOME=/var/lib/jenkins 2025-12-04T09:35:36.8491099Z GITHUB_API_URL=https://api.github.com 2025-12-04T09:35:36.8491307Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2025-12-04T09:35:36.8491486Z UCX_COMMIT= 2025-12-04T09:35:36.8491620Z USE_SYSTEM_NCCL=1 2025-12-04T09:35:36.8491769Z NUM_TEST_SHARDS=1 2025-12-04T09:35:36.8491926Z UCX_HOME=/usr 2025-12-04T09:35:36.8492252Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_ca6c2ac3-ce1c-41a2-95f1-cf47e926186e 2025-12-04T09:35:36.8492777Z JOB_NAME=periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_huggingface, 1, 1, linux.8xlarge.amx) 2025-12-04T09:35:36.8493366Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_ca6c2ac3-ce1c-41a2-95f1-cf47e926186e 2025-12-04T09:35:36.8493988Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2025-12-04T09:35:36.8494269Z GITHUB_EVENT_NAME=schedule 2025-12-04T09:35:36.8494434Z DASHBOARD_TAG= 2025-12-04T09:35:36.8494615Z GITHUB_RUN_ID=19923066595 2025-12-04T09:35:36.8494812Z INSTALLED_OPENBLAS= 2025-12-04T09:35:36.8495266Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_ca6c2ac3-ce1c-41a2-95f1-cf47e926186e 2025-12-04T09:35:36.8495639Z GITHUB_ACTOR=pytorchmergebot 2025-12-04T09:35:36.8495799Z PR_NUMBER= 2025-12-04T09:35:36.8495934Z DESIRED_CUDA= 2025-12-04T09:35:36.8496076Z GITHUB_RUN_ATTEMPT=1 2025-12-04T09:35:36.8496237Z ANACONDA_PYTHON_VERSION=3.10 2025-12-04T09:35:36.8496512Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2025-12-04T09:35:36.8496844Z TERM=vt100 2025-12-04T09:35:36.8496977Z INSTALLED_VISION=yes 2025-12-04T09:35:36.8497132Z BRANCH=main 2025-12-04T09:35:36.8497304Z SCCACHE_REGION=us-east-1 2025-12-04T09:35:36.8497514Z OPENSSL_ROOT_DIR=/opt/openssl 2025-12-04T09:35:36.8497877Z BUILD_AOT_INDUCTOR_TEST=yes 2025-12-04T09:35:36.8498062Z CUDA_PATH=/usr/local/cuda 2025-12-04T09:35:36.8498505Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2025-12-04T09:35:36.8498998Z GITHUB_SERVER_URL=https://github.com 2025-12-04T09:35:36.8499261Z UCC_COMMIT= 2025-12-04T09:35:36.8499433Z REENABLED_ISSUES= 2025-12-04T09:35:36.8499626Z DOCS=yes 2025-12-04T09:35:36.8499796Z SHLVL=1 2025-12-04T09:35:36.8499928Z MAX_JOBS=30 2025-12-04T09:35:36.8500061Z GITHUB_ACTOR_ID=97764156 2025-12-04T09:35:36.8500273Z GITHUB_WORKFLOW_SHA=ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:35:36.8500503Z GITHUB_REF_NAME=main 2025-12-04T09:35:36.8500727Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2025-12-04T09:35:36.8500976Z GITHUB_JOB=test 2025-12-04T09:35:36.8501124Z NO_TEST_TIMEOUT=False 2025-12-04T09:35:36.8501346Z TD_DISTRIBUTED=False 2025-12-04T09:35:36.8501515Z GITHUB_REPOSITORY=pytorch/pytorch 2025-12-04T09:35:36.8501704Z GITHUB_RETENTION_DAYS=90 2025-12-04T09:35:36.8501874Z OPENSSL_DIR=/opt/openssl 2025-12-04T09:35:36.8502042Z GITHUB_ACTION_REPOSITORY= 2025-12-04T09:35:36.8502491Z PATH=/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2025-12-04T09:35:36.8502937Z GITHUB_BASE_REF= 2025-12-04T09:35:36.8503077Z INSTALLED_ACL= 2025-12-04T09:35:36.8503355Z ARTIFACTS_FILE_SUFFIX=test-cpu_inductor_freezing_huggingface-1-1-linux.8xlarge.amx_57118563323 2025-12-04T09:35:36.8503658Z CI=true 2025-12-04T09:35:36.8503794Z GITHUB_REPOSITORY_OWNER=pytorch 2025-12-04T09:35:36.8504028Z RUST_LOG=sccache::server=error 2025-12-04T09:35:36.8504197Z JOB_ID=57118563323 2025-12-04T09:35:36.8504334Z GITHUB_HEAD_REF= 2025-12-04T09:35:36.8504481Z GITHUB_ACTION_REF= 2025-12-04T09:35:36.8504666Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2025-12-04T09:35:36.8504876Z TEST_SHOWLOCALS=False 2025-12-04T09:35:36.8505050Z GITHUB_WORKFLOW=inductor-periodic 2025-12-04T09:35:36.8505244Z DEBIAN_FRONTEND=noninteractive 2025-12-04T09:35:36.8505618Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_ca6c2ac3-ce1c-41a2-95f1-cf47e926186e 2025-12-04T09:35:36.8506125Z NO_TD=False 2025-12-04T09:35:36.8506323Z SKIP_SCCACHE_INITIALIZATION=1 2025-12-04T09:35:36.8506613Z NCCL_INCLUDE_DIR=/usr/local/cuda/include/ 2025-12-04T09:35:36.8506867Z _=/usr/bin/env 2025-12-04T09:35:36.8507146Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2025-12-04T09:35:36.8734686Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2025-12-04T09:35:36.8736782Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2025-12-04T09:35:36.8737321Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2025-12-04T09:35:36.8737805Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2025-12-04T09:35:36.8738184Z + BUILD_DIR=build 2025-12-04T09:35:36.8738461Z + BUILD_RENAMED_DIR=build_renamed 2025-12-04T09:35:36.8738766Z + BUILD_BIN_DIR=build/bin 2025-12-04T09:35:36.8739554Z + SHARD_NUMBER=1 2025-12-04T09:35:36.8739769Z + NUM_TEST_SHARDS=1 2025-12-04T09:35:36.8739978Z + export TORCH_SERIALIZATION_DEBUG=1 2025-12-04T09:35:36.8740219Z + TORCH_SERIALIZATION_DEBUG=1 2025-12-04T09:35:36.8740401Z + export VALGRIND=ON 2025-12-04T09:35:36.8740552Z + VALGRIND=ON 2025-12-04T09:35:36.8740743Z + [[ linux-jammy-py3.10-gcc11-build == *clang9* ]] 2025-12-04T09:35:36.8740998Z + [[ linux-jammy-py3.10-gcc11-build == *xpu* ]] 2025-12-04T09:35:36.8741200Z + detect_cuda_arch 2025-12-04T09:35:36.8741379Z + [[ linux-jammy-py3.10-gcc11-build == *cuda* ]] 2025-12-04T09:35:36.8741613Z + [[ linux-jammy-py3.10-gcc11-build == *s390x* ]] 2025-12-04T09:35:36.8741810Z + [[ 0 == \1 ]] 2025-12-04T09:35:36.8741958Z + [[ True == \1 ]] 2025-12-04T09:35:36.8742152Z + [[ linux-jammy-py3.10-gcc11-build != *bazel* ]] 2025-12-04T09:35:36.8742591Z ++ realpath build/custom_test_artifacts 2025-12-04T09:35:36.8742961Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2025-12-04T09:35:36.8743259Z + [[ -n '' ]] 2025-12-04T09:35:36.8743424Z + echo 'Environment variables' 2025-12-04T09:35:36.8743606Z Environment variables 2025-12-04T09:35:36.8743765Z + env 2025-12-04T09:35:36.8770787Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2025-12-04T09:35:36.8771255Z CONTINUE_THROUGH_ERROR=True 2025-12-04T09:35:36.8771586Z BUILD_ENVIRONMENT=linux-jammy-py3.10-gcc11-build 2025-12-04T09:35:36.8772099Z VLLM_TEST_HUGGING_FACE_TOKEN=*** 2025-12-04T09:35:36.8772293Z HOSTNAME=267402d4cbe5 2025-12-04T09:35:36.8772653Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_ca6c2ac3-ce1c-41a2-95f1-cf47e926186e 2025-12-04T09:35:36.8773012Z GITHUB_ACTION=__run_3 2025-12-04T09:35:36.8773432Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2025-12-04T09:35:36.8773616Z GITHUB_RUN_NUMBER=67027 2025-12-04T09:35:36.8773817Z TEST_CONFIG=cpu_inductor_freezing_huggingface 2025-12-04T09:35:36.8774087Z GITHUB_REPOSITORY_OWNER_ID=21003710 2025-12-04T09:35:36.8774297Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2025-12-04T09:35:36.8774493Z SCCACHE_IDLE_TIMEOUT=0 2025-12-04T09:35:36.8774748Z SCRIBE_GRAPHQL_ACCESS_TOKEN=*** 2025-12-04T09:35:36.8774943Z GITHUB_TRIGGERING_ACTOR=pytorchmergebot 2025-12-04T09:35:36.8775138Z GITHUB_REF_TYPE=branch 2025-12-04T09:35:36.8775321Z BASE_SHA=ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:35:36.8775527Z XLA_CUDA= 2025-12-04T09:35:36.8775677Z NCCL_LIB_DIR=/usr/local/cuda/lib64/ 2025-12-04T09:35:36.8776069Z HUGGING_FACE_HUB_TOKEN=*** 2025-12-04T09:35:36.8776290Z *** 2025-12-04T09:35:36.8776464Z GITHUB_REPOSITORY_ID=65600975 2025-12-04T09:35:36.8776641Z GITHUB_ACTIONS=true 2025-12-04T09:35:36.8776824Z SCCACHE_ERROR_LOG=/var/lib/jenkins/sccache_error.log 2025-12-04T09:35:36.8777072Z SHA1=ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:35:36.8777302Z GITHUB_SHA=ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:35:36.8777632Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/heads/main 2025-12-04T09:35:36.8777934Z UCC_HOME=/usr 2025-12-04T09:35:36.8778086Z TORCH_SERIALIZATION_DEBUG=1 2025-12-04T09:35:36.8778256Z VERBOSE_TEST_LOGS=False 2025-12-04T09:35:36.8778413Z GITHUB_REF=refs/heads/main 2025-12-04T09:35:36.8778574Z SHARD_NUMBER=1 2025-12-04T09:35:36.8778726Z GITHUB_REF_PROTECTED=true 2025-12-04T09:35:36.8778883Z HOME=/var/lib/jenkins 2025-12-04T09:35:36.8779066Z GITHUB_API_URL=https://api.github.com 2025-12-04T09:35:36.8779273Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2025-12-04T09:35:36.8779440Z UCX_COMMIT= 2025-12-04T09:35:36.8779577Z USE_SYSTEM_NCCL=1 2025-12-04T09:35:36.8779725Z NUM_TEST_SHARDS=1 2025-12-04T09:35:36.8779858Z UCX_HOME=/usr 2025-12-04T09:35:36.8780187Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_ca6c2ac3-ce1c-41a2-95f1-cf47e926186e 2025-12-04T09:35:36.8780721Z JOB_NAME=periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_huggingface, 1, 1, linux.8xlarge.amx) 2025-12-04T09:35:36.8781235Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_ca6c2ac3-ce1c-41a2-95f1-cf47e926186e 2025-12-04T09:35:36.8781676Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2025-12-04T09:35:36.8781961Z GITHUB_EVENT_NAME=schedule 2025-12-04T09:35:36.8782123Z DASHBOARD_TAG= 2025-12-04T09:35:36.8782263Z GITHUB_RUN_ID=19923066595 2025-12-04T09:35:36.8782421Z INSTALLED_OPENBLAS= 2025-12-04T09:35:36.8782762Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_ca6c2ac3-ce1c-41a2-95f1-cf47e926186e 2025-12-04T09:35:36.8783132Z GITHUB_ACTOR=pytorchmergebot 2025-12-04T09:35:36.8783301Z PR_NUMBER= 2025-12-04T09:35:36.8783438Z DESIRED_CUDA= 2025-12-04T09:35:36.8783573Z GITHUB_RUN_ATTEMPT=1 2025-12-04T09:35:36.8783726Z VALGRIND=ON 2025-12-04T09:35:36.8783872Z ANACONDA_PYTHON_VERSION=3.10 2025-12-04T09:35:36.8784144Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2025-12-04T09:35:36.8784351Z TERM=vt100 2025-12-04T09:35:36.8784490Z INSTALLED_VISION=yes 2025-12-04T09:35:36.8784640Z BRANCH=main 2025-12-04T09:35:36.8784777Z SCCACHE_REGION=us-east-1 2025-12-04T09:35:36.8784954Z OPENSSL_ROOT_DIR=/opt/openssl 2025-12-04T09:35:36.8785135Z BUILD_AOT_INDUCTOR_TEST=yes 2025-12-04T09:35:36.8785303Z CUDA_PATH=/usr/local/cuda 2025-12-04T09:35:36.8785608Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2025-12-04T09:35:36.8785946Z GITHUB_SERVER_URL=https://github.com 2025-12-04T09:35:36.8786122Z UCC_COMMIT= 2025-12-04T09:35:36.8786260Z REENABLED_ISSUES= 2025-12-04T09:35:36.8786405Z DOCS=yes 2025-12-04T09:35:36.8786528Z SHLVL=1 2025-12-04T09:35:36.8786659Z MAX_JOBS=30 2025-12-04T09:35:36.8786798Z GITHUB_ACTOR_ID=97764156 2025-12-04T09:35:36.8787005Z GITHUB_WORKFLOW_SHA=ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:35:36.8787319Z GITHUB_REF_NAME=main 2025-12-04T09:35:36.8787552Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2025-12-04T09:35:36.8787800Z GITHUB_JOB=test 2025-12-04T09:35:36.8787941Z NO_TEST_TIMEOUT=False 2025-12-04T09:35:36.8788095Z TD_DISTRIBUTED=False 2025-12-04T09:35:36.8788264Z GITHUB_REPOSITORY=pytorch/pytorch 2025-12-04T09:35:36.8788441Z GITHUB_RETENTION_DAYS=90 2025-12-04T09:35:36.8788608Z OPENSSL_DIR=/opt/openssl 2025-12-04T09:35:36.8788778Z GITHUB_ACTION_REPOSITORY= 2025-12-04T09:35:36.8789212Z PATH=/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2025-12-04T09:35:36.8789645Z GITHUB_BASE_REF= 2025-12-04T09:35:36.8789793Z INSTALLED_ACL= 2025-12-04T09:35:36.8790056Z ARTIFACTS_FILE_SUFFIX=test-cpu_inductor_freezing_huggingface-1-1-linux.8xlarge.amx_57118563323 2025-12-04T09:35:36.8790353Z CI=true 2025-12-04T09:35:36.8790500Z GITHUB_REPOSITORY_OWNER=pytorch 2025-12-04T09:35:36.8790726Z RUST_LOG=sccache::server=error 2025-12-04T09:35:36.8790891Z JOB_ID=57118563323 2025-12-04T09:35:36.8791035Z GITHUB_HEAD_REF= 2025-12-04T09:35:36.8791179Z GITHUB_ACTION_REF= 2025-12-04T09:35:36.8791359Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2025-12-04T09:35:36.8791572Z TEST_SHOWLOCALS=False 2025-12-04T09:35:36.8791744Z GITHUB_WORKFLOW=inductor-periodic 2025-12-04T09:35:36.8791929Z DEBIAN_FRONTEND=noninteractive 2025-12-04T09:35:36.8792290Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_ca6c2ac3-ce1c-41a2-95f1-cf47e926186e 2025-12-04T09:35:36.8792651Z NO_TD=False 2025-12-04T09:35:36.8792800Z SKIP_SCCACHE_INITIALIZATION=1 2025-12-04T09:35:36.8793163Z NCCL_INCLUDE_DIR=/usr/local/cuda/include/ 2025-12-04T09:35:36.8793377Z _=/usr/bin/env 2025-12-04T09:35:36.8793527Z + echo 'Testing pytorch' 2025-12-04T09:35:36.8793700Z Testing pytorch 2025-12-04T09:35:36.8793860Z + export LANG=C.UTF-8 2025-12-04T09:35:36.8794023Z + LANG=C.UTF-8 2025-12-04T09:35:36.8794187Z + PR_NUMBER= 2025-12-04T09:35:36.8794389Z + [[ cpu_inductor_freezing_huggingface == \d\e\f\a\u\l\t ]] 2025-12-04T09:35:36.8794671Z + [[ cpu_inductor_freezing_huggingface == \d\i\s\t\r\i\b\u\t\e\d ]] 2025-12-04T09:35:36.8794929Z + [[ cpu_inductor_freezing_huggingface == \s\l\o\w ]] 2025-12-04T09:35:36.8795195Z + [[ linux-jammy-py3.10-gcc11-build == *slow-gradcheck* ]] 2025-12-04T09:35:36.8795459Z + [[ linux-jammy-py3.10-gcc11-build == *cuda* ]] 2025-12-04T09:35:36.8795683Z + [[ linux-jammy-py3.10-gcc11-build == *rocm* ]] 2025-12-04T09:35:36.8795909Z + [[ linux-jammy-py3.10-gcc11-build == *xpu* ]] 2025-12-04T09:35:36.8796149Z + [[ cpu_inductor_freezing_huggingface == *crossref* ]] 2025-12-04T09:35:36.8796385Z + [[ linux-jammy-py3.10-gcc11-build == *rocm* ]] 2025-12-04T09:35:36.8796608Z + [[ linux-jammy-py3.10-gcc11-build == *xpu* ]] 2025-12-04T09:35:36.8796839Z + [[ linux-jammy-py3.10-gcc11-build != *-bazel-* ]] 2025-12-04T09:35:36.8797060Z + pip_install ninja==1.10.2 2025-12-04T09:35:36.8797284Z + pip_install_pkg='python3 -m pip install --progress-bar off' 2025-12-04T09:35:36.8797610Z + python3 -m pip install --progress-bar off ninja==1.10.2 2025-12-04T09:35:37.2460857Z Collecting ninja==1.10.2 2025-12-04T09:35:37.2654648Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2025-12-04T09:35:37.2771044Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2025-12-04T09:35:37.9973240Z Installing collected packages: ninja 2025-12-04T09:35:37.9973807Z Attempting uninstall: ninja 2025-12-04T09:35:37.9980759Z Found existing installation: ninja 1.11.1.4 2025-12-04T09:35:37.9992866Z Uninstalling ninja-1.11.1.4: 2025-12-04T09:35:38.0037973Z Successfully uninstalled ninja-1.11.1.4 2025-12-04T09:35:38.0494813Z Successfully installed ninja-1.10.2 2025-12-04T09:35:38.1583373Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2025-12-04T09:35:38.1584797Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2025-12-04T09:35:38.1585349Z + [[ linux-jammy-py3.10-gcc11-build == *aarch64* ]] 2025-12-04T09:35:38.1585679Z + [[ linux-jammy-py3.10-gcc11-build == *asan* ]] 2025-12-04T09:35:38.1585926Z + [[ linux-jammy-py3.10-gcc11-build == *-debug* ]] 2025-12-04T09:35:38.1588032Z + [[ linux-jammy-py3.10-gcc11-build != *-bazel-* ]] 2025-12-04T09:35:38.1588440Z + echo 'We are not in debug mode: linux-jammy-py3.10-gcc11-build. Expect the assertion to pass' 2025-12-04T09:35:38.1588846Z We are not in debug mode: linux-jammy-py3.10-gcc11-build. Expect the assertion to pass 2025-12-04T09:35:38.1589118Z + cd test 2025-12-04T09:35:38.1589351Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2025-12-04T09:35:38.4401177Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:35:38.4402590Z import pynvml # type: ignore[import] 2025-12-04T09:35:39.2914823Z + [[ cpu_inductor_freezing_huggingface == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2025-12-04T09:35:39.2915570Z + [[ cpu_inductor_freezing_huggingface == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2025-12-04T09:35:39.2916143Z + [[ cpu_inductor_freezing_huggingface == \l\e\g\a\c\y\_\n\v\i\d\i\a\_\d\r\i\v\e\r ]] 2025-12-04T09:35:39.2916481Z + DYNAMO_BENCHMARK_FLAGS=() 2025-12-04T09:35:39.2916796Z + [[ cpu_inductor_freezing_huggingface == *pr_time_benchmarks* ]] 2025-12-04T09:35:39.2917152Z + [[ cpu_inductor_freezing_huggingface == *dynamo_eager* ]] 2025-12-04T09:35:39.2917454Z + [[ cpu_inductor_freezing_huggingface == *aot_eager* ]] 2025-12-04T09:35:39.2917794Z + [[ cpu_inductor_freezing_huggingface == *aot_inductor* ]] 2025-12-04T09:35:39.2918120Z + [[ cpu_inductor_freezing_huggingface == *max_autotune_inductor* ]] 2025-12-04T09:35:39.2918464Z + [[ cpu_inductor_freezing_huggingface == *inductor* ]] 2025-12-04T09:35:39.2918743Z + [[ cpu_inductor_freezing_huggingface != *perf* ]] 2025-12-04T09:35:39.2919033Z + DYNAMO_BENCHMARK_FLAGS+=(--inductor) 2025-12-04T09:35:39.2919300Z + [[ cpu_inductor_freezing_huggingface == *dynamic* ]] 2025-12-04T09:35:39.2919562Z + [[ cpu_inductor_freezing_huggingface == *cpu* ]] 2025-12-04T09:35:39.2919810Z + DYNAMO_BENCHMARK_FLAGS+=(--device cpu) 2025-12-04T09:35:39.3095443Z + [[ linux-jammy-py3.10-gcc11-build == *libtorch* ]] 2025-12-04T09:35:39.3095983Z + [[ linux-jammy-py3.10-gcc11-build == *-bazel-* ]] 2025-12-04T09:35:39.3096321Z + cd test 2025-12-04T09:35:39.3096614Z + python -c 'import torch; print(torch.__config__.show())' 2025-12-04T09:35:39.6069393Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:35:39.6070324Z import pynvml # type: ignore[import] 2025-12-04T09:35:40.2440536Z PyTorch built with: 2025-12-04T09:35:40.2442859Z - GCC 11.4 2025-12-04T09:35:40.2445390Z - C++ Version: 201703 2025-12-04T09:35:40.2445905Z - Intel(R) oneAPI Math Kernel Library Version 2024.2-Product Build 20240605 for Intel(R) 64 architecture applications 2025-12-04T09:35:40.2450080Z - Intel(R) MKL-DNN v3.7.1 (Git Hash 8d263e693366ef8db40acc569cc7d8edf644556d) 2025-12-04T09:35:40.2450394Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2025-12-04T09:35:40.2450624Z - LAPACK is enabled (usually provided by MKL) 2025-12-04T09:35:40.2450829Z - NNPACK is enabled 2025-12-04T09:35:40.2450998Z - CPU capability usage: AVX512 2025-12-04T09:35:40.2454101Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, COMMIT_SHA=ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -DC10_NODEPRECATED -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=range-loop-construct -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -faligned-new -Werror -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, TORCH_VERSION=2.10.0, USE_CUDA=OFF, USE_CUDNN=OFF, USE_CUSPARSELT=OFF, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, USE_XCCL=OFF, USE_XPU=OFF, 2025-12-04T09:35:40.2456745Z 2025-12-04T09:35:40.4650616Z + cd test 2025-12-04T09:35:40.4654699Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2025-12-04T09:35:40.7276357Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:35:40.7277261Z import pynvml # type: ignore[import] 2025-12-04T09:35:41.3385888Z ATen/Parallel: 2025-12-04T09:35:41.3387454Z at::get_num_threads() : 16 2025-12-04T09:35:41.3387741Z at::get_num_interop_threads() : 16 2025-12-04T09:35:41.3387955Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2025-12-04T09:35:41.3388140Z omp_get_max_threads() : 16 2025-12-04T09:35:41.3388487Z Intel(R) oneAPI Math Kernel Library Version 2024.2-Product Build 20240605 for Intel(R) 64 architecture applications 2025-12-04T09:35:41.3388875Z mkl_get_max_threads() : 16 2025-12-04T09:35:41.3389110Z Intel(R) MKL-DNN v3.7.1 (Git Hash 8d263e693366ef8db40acc569cc7d8edf644556d) 2025-12-04T09:35:41.3389378Z std::thread::hardware_concurrency() : 32 2025-12-04T09:35:41.3389576Z Environment variables: 2025-12-04T09:35:41.3389741Z OMP_NUM_THREADS : [not set] 2025-12-04T09:35:41.3389919Z MKL_NUM_THREADS : [not set] 2025-12-04T09:35:41.3390098Z ATen parallel backend: OpenMP 2025-12-04T09:35:41.3390212Z 2025-12-04T09:35:41.5491826Z + [[ cpu_inductor_freezing_huggingface == *numpy_2* ]] 2025-12-04T09:35:41.5494086Z + [[ linux-jammy-py3.10-gcc11-build == *aarch64* ]] 2025-12-04T09:35:41.5494497Z + [[ cpu_inductor_freezing_huggingface == *backward* ]] 2025-12-04T09:35:41.5494802Z + [[ cpu_inductor_freezing_huggingface == *libtorch_agnostic_targetting* ]] 2025-12-04T09:35:41.5495099Z + [[ cpu_inductor_freezing_huggingface == *xla* ]] 2025-12-04T09:35:41.5495401Z + [[ cpu_inductor_freezing_huggingface == *vllm* ]] 2025-12-04T09:35:41.5496018Z + [[ cpu_inductor_freezing_huggingface == *executorch* ]] 2025-12-04T09:35:41.5496298Z + [[ cpu_inductor_freezing_huggingface == \j\i\t\_\l\e\g\a\c\y ]] 2025-12-04T09:35:41.5496617Z + [[ cpu_inductor_freezing_huggingface == \q\u\a\n\t\i\z\a\t\i\o\n ]] 2025-12-04T09:35:41.5496969Z + [[ linux-jammy-py3.10-gcc11-build == *libtorch* ]] 2025-12-04T09:35:41.5497223Z + [[ cpu_inductor_freezing_huggingface == distributed ]] 2025-12-04T09:35:41.5497499Z + [[ cpu_inductor_freezing_huggingface == *operator_benchmark* ]] 2025-12-04T09:35:41.5497796Z + [[ cpu_inductor_freezing_huggingface == *operator_microbenchmark* ]] 2025-12-04T09:35:41.5498120Z + [[ cpu_inductor_freezing_huggingface == *attention_microbenchmark* ]] 2025-12-04T09:35:41.5498431Z + [[ cpu_inductor_freezing_huggingface == *inductor_distributed* ]] 2025-12-04T09:35:41.5498716Z + [[ cpu_inductor_freezing_huggingface == *inductor-halide* ]] 2025-12-04T09:35:41.5499108Z + [[ cpu_inductor_freezing_huggingface == *inductor-pallas* ]] 2025-12-04T09:35:41.5499412Z + [[ cpu_inductor_freezing_huggingface == *inductor-triton-cpu* ]] 2025-12-04T09:35:41.5499722Z + [[ cpu_inductor_freezing_huggingface == *inductor-micro-benchmark* ]] 2025-12-04T09:35:41.5500065Z + [[ cpu_inductor_freezing_huggingface == *aoti_cross_compile_for_windows* ]] 2025-12-04T09:35:41.5500373Z + [[ cpu_inductor_freezing_huggingface == *huggingface* ]] 2025-12-04T09:35:41.5500609Z + install_torchvision 2025-12-04T09:35:41.5500775Z + local orig_preload 2025-12-04T09:35:41.5500939Z + local commit 2025-12-04T09:35:41.5501102Z ++ get_pinned_commit vision 2025-12-04T09:35:41.5501288Z ++ cat .github/ci_commit_pins/vision.txt 2025-12-04T09:35:41.5501533Z + commit=617079d944b0e72632311c30ae2bbdf1168b901e 2025-12-04T09:35:41.5501748Z + orig_preload= 2025-12-04T09:35:41.5501898Z + '[' -n '' ']' 2025-12-04T09:35:41.5502145Z + [[ linux-jammy-py3.10-gcc11-build == *cuda* ]] 2025-12-04T09:35:41.5502659Z + pip_build_and_install git+https://github.com/pytorch/vision.git@617079d944b0e72632311c30ae2bbdf1168b901e dist/vision 2025-12-04T09:35:41.5503247Z + local build_target=git+https://github.com/pytorch/vision.git@617079d944b0e72632311c30ae2bbdf1168b901e 2025-12-04T09:35:41.5503846Z + local wheel_dir=dist/vision 2025-12-04T09:35:41.5504102Z + local found_whl=0 2025-12-04T09:35:41.5504373Z + for file in "${wheel_dir}"/*.whl 2025-12-04T09:35:41.5504719Z + [[ -f dist/vision/torchvision-0.25.0a0+617079d-cp310-cp310-linux_x86_64.whl ]] 2025-12-04T09:35:41.5505071Z + found_whl=1 2025-12-04T09:35:41.5505300Z + break 2025-12-04T09:35:41.5505484Z + '[' 1 == 0 ']' 2025-12-04T09:35:41.5505920Z + for file in "${wheel_dir}"/*.whl 2025-12-04T09:35:41.5506413Z + pip_install_whl dist/vision/torchvision-0.25.0a0+617079d-cp310-cp310-linux_x86_64.whl 2025-12-04T09:35:41.5506917Z + args=('dist/vision/torchvision-0.25.0a0+617079d-cp310-cp310-linux_x86_64.whl') 2025-12-04T09:35:41.5507279Z + local args 2025-12-04T09:35:41.5507627Z + [[ dist/vision/torchvision-0.25.0a0+617079d-cp310-cp310-linux_x86_64.whl == *\ * ]] 2025-12-04T09:35:41.5508000Z + for path in "${args[@]}" 2025-12-04T09:35:41.5508338Z + echo 'Installing dist/vision/torchvision-0.25.0a0+617079d-cp310-cp310-linux_x86_64.whl' 2025-12-04T09:35:41.5508834Z Installing dist/vision/torchvision-0.25.0a0+617079d-cp310-cp310-linux_x86_64.whl 2025-12-04T09:35:41.5509404Z + python3 -mpip install --no-index --no-deps dist/vision/torchvision-0.25.0a0+617079d-cp310-cp310-linux_x86_64.whl 2025-12-04T09:35:41.8173095Z Processing ./dist/vision/torchvision-0.25.0a0+617079d-cp310-cp310-linux_x86_64.whl 2025-12-04T09:35:41.8242456Z Installing collected packages: torchvision 2025-12-04T09:35:42.3877866Z Successfully installed torchvision-0.25.0a0+617079d 2025-12-04T09:35:42.4275514Z + '[' -n '' ']' 2025-12-04T09:35:42.4278368Z + id=0 2025-12-04T09:35:42.4278611Z + test_dynamo_benchmark huggingface 0 2025-12-04T09:35:42.4278819Z ++ pwd 2025-12-04T09:35:42.4279027Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2025-12-04T09:35:42.4279314Z + local suite=huggingface 2025-12-04T09:35:42.4279477Z + shift 2025-12-04T09:35:42.4279928Z + local shard_id=0 2025-12-04T09:35:42.4280087Z + shift 2025-12-04T09:35:42.4280218Z + extra_args=() 2025-12-04T09:35:42.4280373Z + local extra_args 2025-12-04T09:35:42.4280562Z + [[ linux-jammy-py3.10-gcc11-build == *cuda13* ]] 2025-12-04T09:35:42.4280833Z + [[ cpu_inductor_freezing_huggingface == *perf_compare* ]] 2025-12-04T09:35:42.4281084Z + [[ cpu_inductor_freezing_huggingface == *perf* ]] 2025-12-04T09:35:42.4281314Z + [[ cpu_inductor_freezing_huggingface == *cpu* ]] 2025-12-04T09:35:42.4281514Z + local dt=float32 2025-12-04T09:35:42.4281677Z + [[ cpu_inductor_freezing_huggingface == *amp* ]] 2025-12-04T09:35:42.4281912Z + [[ cpu_inductor_freezing_huggingface == *freezing* ]] 2025-12-04T09:35:42.4282221Z + test_single_dynamo_benchmark inference huggingface 0 --inference --float32 --freezing 2025-12-04T09:35:42.4288702Z ++ pwd 2025-12-04T09:35:42.4289110Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2025-12-04T09:35:42.4294382Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports 2025-12-04T09:35:42.4306929Z + local name=inference 2025-12-04T09:35:42.4307303Z + shift 2025-12-04T09:35:42.4307499Z + local suite=huggingface 2025-12-04T09:35:42.4307790Z + shift 2025-12-04T09:35:42.4307968Z + local shard_id=0 2025-12-04T09:35:42.4308114Z + shift 2025-12-04T09:35:42.4308261Z + partition_flags=() 2025-12-04T09:35:42.4308433Z + local partition_flags 2025-12-04T09:35:42.4308594Z + [[ -n 1 ]] 2025-12-04T09:35:42.4308738Z + [[ -n 0 ]] 2025-12-04T09:35:42.4309000Z + partition_flags=(--total-partitions "$NUM_TEST_SHARDS" --partition-id "$shard_id") 2025-12-04T09:35:42.4309349Z + [[ cpu_inductor_freezing_huggingface == *perf_compare* ]] 2025-12-04T09:35:42.4309601Z + [[ cpu_inductor_freezing_huggingface == *perf* ]] 2025-12-04T09:35:42.4309837Z + [[ cpu_inductor_freezing_huggingface == *_avx2* ]] 2025-12-04T09:35:42.4310076Z + [[ cpu_inductor_freezing_huggingface == *_avx512* ]] 2025-12-04T09:35:42.4310861Z + python benchmarks/dynamo/huggingface.py --ci --accuracy --timing --explain --print-compilation-time --inductor --device cpu --inference --float32 --freezing --total-partitions 1 --partition-id 0 --output /var/lib/jenkins/workspace/test/test-reports/inference_huggingface.csv 2025-12-04T09:35:43.2009060Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:35:43.2011960Z import pynvml # type: ignore[import] 2025-12-04T09:35:46.2963423Z 2025-12-04T09:35:46.2968208Z config.json: 0% 0.00/694 [00:00bcxy", (query, key)) # multiply 2025-12-04T09:37:13.9773350Z 2025-12-04T09:37:13.9773465Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9773985Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9774488Z layer_outputs = layer_module( 2025-12-04T09:37:13.9774864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9775229Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9775651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9776126Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9776547Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9776948Z self_outputs = self.self( 2025-12-04T09:37:13.9777339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:13.9777774Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:13.9778261Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:13.9778793Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:37:13.9779281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:37:13.9779698Z hidden_states = hidden_states.view( 2025-12-04T09:37:13.9779829Z 2025-12-04T09:37:13.9779945Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9780448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9780941Z layer_outputs = layer_module( 2025-12-04T09:37:13.9781299Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9781659Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9782060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9782478Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9782901Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9783305Z self_outputs = self.self( 2025-12-04T09:37:13.9783700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:13.9784141Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:13.9784642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:13.9785221Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:13.9785474Z 2025-12-04T09:37:13.9785576Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9786082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9786577Z layer_outputs = layer_module( 2025-12-04T09:37:13.9786955Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9787341Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9787788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9788272Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9788701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9789129Z self_outputs = self.self( 2025-12-04T09:37:13.9789534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:13.9789995Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:13.9790494Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:13.9791154Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:13.9791405Z 2025-12-04T09:37:13.9791516Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9792032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9792511Z layer_outputs = layer_module( 2025-12-04T09:37:13.9792867Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9793337Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9793753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9794194Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9794610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9795031Z self_outputs = self.self( 2025-12-04T09:37:13.9795421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:13.9795855Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:13.9796345Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:13.9796921Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:13.9797156Z 2025-12-04T09:37:13.9797241Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9797456Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9797669Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9797867Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9798101Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9798607Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9799087Z layer_outputs = layer_module( 2025-12-04T09:37:13.9799424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9799784Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9800198Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9800600Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9801020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9801496Z self_outputs = self.self( 2025-12-04T09:37:13.9801889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 535, in forward 2025-12-04T09:37:13.9802317Z diagonal_mask = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:13.9802812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 833, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:13.9803347Z self._mask_invalid_locations(diagonal_attention_scores, window_overlap) 2025-12-04T09:37:13.9803859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 763, in _mask_invalid_locations 2025-12-04T09:37:13.9804327Z ).where(beginning_mask.bool(), beginning_input) 2025-12-04T09:37:13.9804481Z 2025-12-04T09:37:13.9804594Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9804836Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9805356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9806044Z layer_outputs = layer_module( 2025-12-04T09:37:13.9806403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9806774Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9807188Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9807610Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9808032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9808463Z self_outputs = self.self( 2025-12-04T09:37:13.9808887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:37:13.9809295Z attn_scores += diagonal_mask 2025-12-04T09:37:13.9809425Z 2025-12-04T09:37:13.9809528Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9810058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9810540Z layer_outputs = layer_module( 2025-12-04T09:37:13.9810873Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9811230Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9811637Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9812041Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9812450Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9812853Z self_outputs = self.self( 2025-12-04T09:37:13.9813240Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:37:13.9813642Z attn_probs = nn.functional.softmax( 2025-12-04T09:37:13.9813777Z 2025-12-04T09:37:13.9813875Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9814382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9814857Z layer_outputs = layer_module( 2025-12-04T09:37:13.9815191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9815554Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9816039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9816458Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9816864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9817272Z self_outputs = self.self( 2025-12-04T09:37:13.9817664Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:13.9818108Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:13.9818640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:13.9819285Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:37:13.9819703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:13.9820042Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:13.9820199Z 2025-12-04T09:37:13.9820300Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9820803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9821292Z layer_outputs = layer_module( 2025-12-04T09:37:13.9821619Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9821965Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9822365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9822766Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9823156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9823547Z self_outputs = self.self( 2025-12-04T09:37:13.9823925Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:13.9824357Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:13.9824859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:13.9825382Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:37:13.9825868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:37:13.9826318Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:37:13.9826645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:13.9826980Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:13.9827128Z 2025-12-04T09:37:13.9827232Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9827727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9828191Z layer_outputs = layer_module( 2025-12-04T09:37:13.9828521Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9828870Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9829307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9829707Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9830104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9830486Z self_outputs = self.self( 2025-12-04T09:37:13.9830864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:13.9831298Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:13.9831796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:13.9832321Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:13.9832556Z 2025-12-04T09:37:13.9832656Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9833261Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9833781Z layer_outputs = layer_module( 2025-12-04T09:37:13.9834143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9834511Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9834934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9835332Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9835724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9836124Z self_outputs = self.self( 2025-12-04T09:37:13.9836509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:13.9836929Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:13.9837428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:13.9837963Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:13.9838156Z 2025-12-04T09:37:13.9838262Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9838740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9839200Z layer_outputs = layer_module( 2025-12-04T09:37:13.9839529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9839878Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9840273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9840672Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9841066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9841464Z self_outputs = self.self( 2025-12-04T09:37:13.9841839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:37:13.9842346Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:37:13.9842574Z 2025-12-04T09:37:13.9842658Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9842857Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9843114Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9843609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9844078Z layer_outputs = layer_module( 2025-12-04T09:37:13.9844403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9844752Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9845157Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:37:13.9845567Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:13.9845948Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:13.9846379Z return forward_fn(*input_tensors) 2025-12-04T09:37:13.9846780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:37:13.9847207Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:37:13.9847633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:37:13.9848075Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:13.9848445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:13.9848770Z return self.act(input) 2025-12-04T09:37:13.9848884Z 2025-12-04T09:37:13.9848959Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9849159Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9849375Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9849876Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9850342Z layer_outputs = layer_module( 2025-12-04T09:37:13.9850675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9851018Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9851420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9851824Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9852230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9852617Z self_outputs = self.self( 2025-12-04T09:37:13.9852995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:13.9853422Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:13.9853891Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:13.9854451Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:13.9854695Z 2025-12-04T09:37:13.9854768Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9854995Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9855474Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9855936Z layer_outputs = layer_module( 2025-12-04T09:37:13.9856264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9856640Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9857033Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9857431Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9857824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9858221Z self_outputs = self.self( 2025-12-04T09:37:13.9858590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:13.9859020Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:13.9859494Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:13.9860044Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:37:13.9860516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:37:13.9860915Z hidden_states = hidden_states.view( 2025-12-04T09:37:13.9861041Z 2025-12-04T09:37:13.9861147Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9861635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9862105Z layer_outputs = layer_module( 2025-12-04T09:37:13.9862442Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9862792Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9863186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9863598Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9863997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9864395Z self_outputs = self.self( 2025-12-04T09:37:13.9864771Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:13.9865204Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:13.9865682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:13.9866238Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:13.9866481Z 2025-12-04T09:37:13.9866580Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9867072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9867540Z layer_outputs = layer_module( 2025-12-04T09:37:13.9867863Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9868210Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9868609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9869009Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9869399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9869793Z self_outputs = self.self( 2025-12-04T09:37:13.9870209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:13.9870642Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:13.9871108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:13.9871662Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:13.9871888Z 2025-12-04T09:37:13.9871994Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9872500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9872967Z layer_outputs = layer_module( 2025-12-04T09:37:13.9873405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9873879Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9874312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9874749Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9875161Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9875560Z self_outputs = self.self( 2025-12-04T09:37:13.9875941Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:13.9876378Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:13.9876874Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:13.9877448Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:13.9877684Z 2025-12-04T09:37:13.9877763Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9877967Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9878194Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9878687Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9879162Z layer_outputs = layer_module( 2025-12-04T09:37:13.9879501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9879858Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9880261Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9880677Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9881084Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9881486Z self_outputs = self.self( 2025-12-04T09:37:13.9881868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:37:13.9882276Z attn_scores += diagonal_mask 2025-12-04T09:37:13.9882393Z 2025-12-04T09:37:13.9882499Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9882989Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9883463Z layer_outputs = layer_module( 2025-12-04T09:37:13.9883805Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9884166Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9884619Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9885033Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9885436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9885836Z self_outputs = self.self( 2025-12-04T09:37:13.9886216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:37:13.9886633Z attn_probs = nn.functional.softmax( 2025-12-04T09:37:13.9886761Z 2025-12-04T09:37:13.9886847Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9887062Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9887603Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9888085Z layer_outputs = layer_module( 2025-12-04T09:37:13.9888434Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9888789Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9889209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9889612Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9890015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9890405Z self_outputs = self.self( 2025-12-04T09:37:13.9890792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:13.9891241Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:13.9891747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:13.9892306Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:37:13.9892721Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:13.9893065Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:13.9893211Z 2025-12-04T09:37:13.9893310Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9893808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9894280Z layer_outputs = layer_module( 2025-12-04T09:37:13.9894619Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9894963Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9895367Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9895773Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9896171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9896562Z self_outputs = self.self( 2025-12-04T09:37:13.9896947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:13.9897388Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:13.9897938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:13.9898454Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:37:13.9898937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:37:13.9899389Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:37:13.9899708Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:13.9900049Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:13.9900204Z 2025-12-04T09:37:13.9900300Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9900787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9901275Z layer_outputs = layer_module( 2025-12-04T09:37:13.9901609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9901955Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9902350Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9902739Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9903136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9903532Z self_outputs = self.self( 2025-12-04T09:37:13.9903898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:13.9904330Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:13.9904834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:13.9905367Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:13.9905567Z 2025-12-04T09:37:13.9905667Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9906346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9906829Z layer_outputs = layer_module( 2025-12-04T09:37:13.9907171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9907526Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9907930Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9908345Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9908754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9909154Z self_outputs = self.self( 2025-12-04T09:37:13.9909543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:13.9909989Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:13.9910500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:13.9911054Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:13.9911260Z 2025-12-04T09:37:13.9911360Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9911937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9912414Z layer_outputs = layer_module( 2025-12-04T09:37:13.9912744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9913190Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9913639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9914067Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9914480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9914900Z self_outputs = self.self( 2025-12-04T09:37:13.9915300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:37:13.9915861Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:37:13.9916104Z 2025-12-04T09:37:13.9916180Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9916384Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9916601Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9917104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9917579Z layer_outputs = layer_module( 2025-12-04T09:37:13.9917918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9918268Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9918678Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:37:13.9919102Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:13.9919500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:13.9919883Z return forward_fn(*input_tensors) 2025-12-04T09:37:13.9920286Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:37:13.9920733Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:37:13.9921179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:37:13.9921602Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:13.9921968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:13.9922304Z return self.act(input) 2025-12-04T09:37:13.9922409Z 2025-12-04T09:37:13.9922486Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9922684Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9922903Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9923383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9923837Z layer_outputs = layer_module( 2025-12-04T09:37:13.9924170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9924514Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9924905Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9925307Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9925732Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9926127Z self_outputs = self.self( 2025-12-04T09:37:13.9926502Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:13.9926929Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:13.9927405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:13.9927970Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:13.9928203Z 2025-12-04T09:37:13.9928277Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9928500Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9929022Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9929482Z layer_outputs = layer_module( 2025-12-04T09:37:13.9929808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9930152Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9930546Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9930936Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9931331Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9931723Z self_outputs = self.self( 2025-12-04T09:37:13.9932099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:13.9932521Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:13.9932993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:13.9933526Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:37:13.9933995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:37:13.9934380Z hidden_states = hidden_states.view( 2025-12-04T09:37:13.9934511Z 2025-12-04T09:37:13.9934609Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9935106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9935566Z layer_outputs = layer_module( 2025-12-04T09:37:13.9935892Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9936237Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9936634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9937023Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9937417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9937810Z self_outputs = self.self( 2025-12-04T09:37:13.9938186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:13.9938605Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:13.9939118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:13.9939673Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:13.9939905Z 2025-12-04T09:37:13.9940008Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9940489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9940973Z layer_outputs = layer_module( 2025-12-04T09:37:13.9941309Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9941654Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9942045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9942472Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9942864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9943246Z self_outputs = self.self( 2025-12-04T09:37:13.9943623Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:13.9944042Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:13.9944515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:13.9945056Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:13.9945289Z 2025-12-04T09:37:13.9945388Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9945886Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9946347Z layer_outputs = layer_module( 2025-12-04T09:37:13.9946674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9947019Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9947416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9947811Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9948199Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9948592Z self_outputs = self.self( 2025-12-04T09:37:13.9948968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:13.9949382Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:13.9949854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:13.9950405Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:13.9950633Z 2025-12-04T09:37:13.9950717Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9950911Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9951134Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9951637Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9952108Z layer_outputs = layer_module( 2025-12-04T09:37:13.9952445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9952854Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9953357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9953762Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9954161Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9954557Z self_outputs = self.self( 2025-12-04T09:37:13.9954939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:37:13.9955328Z attn_scores += diagonal_mask 2025-12-04T09:37:13.9955453Z 2025-12-04T09:37:13.9955549Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9956093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9956569Z layer_outputs = layer_module( 2025-12-04T09:37:13.9956901Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9957259Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9957671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9958082Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9958476Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9958883Z self_outputs = self.self( 2025-12-04T09:37:13.9959271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:37:13.9959684Z attn_probs = nn.functional.softmax( 2025-12-04T09:37:13.9959828Z 2025-12-04T09:37:13.9959905Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9960132Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9960635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9961107Z layer_outputs = layer_module( 2025-12-04T09:37:13.9961449Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9961808Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9962210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9962625Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9963037Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9963450Z self_outputs = self.self( 2025-12-04T09:37:13.9963830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:13.9964281Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:13.9964794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:13.9965355Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:37:13.9965760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:13.9966100Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:13.9966244Z 2025-12-04T09:37:13.9966350Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9966883Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9967339Z layer_outputs = layer_module( 2025-12-04T09:37:13.9967668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9968020Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9968413Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9968812Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9969208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9969623Z self_outputs = self.self( 2025-12-04T09:37:13.9969999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:13.9970432Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:13.9970928Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:13.9971443Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:37:13.9971916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:37:13.9972356Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:37:13.9972679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:13.9973017Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:13.9973159Z 2025-12-04T09:37:13.9973261Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9973750Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9974216Z layer_outputs = layer_module( 2025-12-04T09:37:13.9974539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9974888Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9975288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9975687Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9976070Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9976462Z self_outputs = self.self( 2025-12-04T09:37:13.9976845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:13.9977278Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:13.9977765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:13.9978302Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:13.9978505Z 2025-12-04T09:37:13.9978601Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9979082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9979538Z layer_outputs = layer_module( 2025-12-04T09:37:13.9979899Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9980250Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9980638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9981035Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9981427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9981819Z self_outputs = self.self( 2025-12-04T09:37:13.9982191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:13.9982621Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:13.9983122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:13.9983704Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:13.9983897Z 2025-12-04T09:37:13.9983993Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9984480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9984936Z layer_outputs = layer_module( 2025-12-04T09:37:13.9985265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9985606Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9986007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9986408Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9986801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9987183Z self_outputs = self.self( 2025-12-04T09:37:13.9987561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:37:13.9988058Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:37:13.9988283Z 2025-12-04T09:37:13.9988356Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9988560Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9988778Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9989266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9989721Z layer_outputs = layer_module( 2025-12-04T09:37:13.9990054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9990400Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9990797Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:37:13.9991196Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:13.9991582Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:13.9991960Z return forward_fn(*input_tensors) 2025-12-04T09:37:13.9992348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:37:13.9992780Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:37:13.9993335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:37:13.9993811Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:13.9994191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:13.9994544Z return self.act(input) 2025-12-04T09:37:13.9994649Z 2025-12-04T09:37:13.9994731Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9994928Z cudagraph partition due to non gpu ops 2025-12-04T09:37:13.9995156Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:13.9995646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:13.9996119Z layer_outputs = layer_module( 2025-12-04T09:37:13.9996450Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:13.9996841Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:13.9997237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:13.9997637Z self_attn_outputs = self.attention( 2025-12-04T09:37:13.9998023Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:13.9998418Z self_outputs = self.self( 2025-12-04T09:37:13.9998794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:13.9999209Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:13.9999684Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0000245Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0000475Z 2025-12-04T09:37:14.0000558Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0000775Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0001267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0001727Z layer_outputs = layer_module( 2025-12-04T09:37:14.0002056Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0002403Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0002801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0003199Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0003596Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0003981Z self_outputs = self.self( 2025-12-04T09:37:14.0004361Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0005066Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0005533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0006201Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:37:14.0006677Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:37:14.0007142Z hidden_states = hidden_states.view( 2025-12-04T09:37:14.0007271Z 2025-12-04T09:37:14.0007439Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0007929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0008399Z layer_outputs = layer_module( 2025-12-04T09:37:14.0008742Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0009093Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0009503Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0009916Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0010327Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0010774Z self_outputs = self.self( 2025-12-04T09:37:14.0011167Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0011598Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0012079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0012653Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0012899Z 2025-12-04T09:37:14.0013001Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0013511Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0013987Z layer_outputs = layer_module( 2025-12-04T09:37:14.0014322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0014685Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0015099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0015498Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0015910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0016311Z self_outputs = self.self( 2025-12-04T09:37:14.0016696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0017124Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0017607Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0018176Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0018409Z 2025-12-04T09:37:14.0018517Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0019013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0019489Z layer_outputs = layer_module( 2025-12-04T09:37:14.0019828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0020182Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0020583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0020992Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0021462Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0021852Z self_outputs = self.self( 2025-12-04T09:37:14.0022233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0022653Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0023127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0023672Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0023905Z 2025-12-04T09:37:14.0023980Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0024181Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0024430Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0024915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0025378Z layer_outputs = layer_module( 2025-12-04T09:37:14.0025709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0026052Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0026448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0026847Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0027239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0027626Z self_outputs = self.self( 2025-12-04T09:37:14.0028010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:37:14.0028406Z attn_scores += diagonal_mask 2025-12-04T09:37:14.0028520Z 2025-12-04T09:37:14.0028623Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0029101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0029562Z layer_outputs = layer_module( 2025-12-04T09:37:14.0029894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0030236Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0030631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0031030Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0031430Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0031818Z self_outputs = self.self( 2025-12-04T09:37:14.0032195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:37:14.0032593Z attn_probs = nn.functional.softmax( 2025-12-04T09:37:14.0032717Z 2025-12-04T09:37:14.0032799Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0033016Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0033615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0034114Z layer_outputs = layer_module( 2025-12-04T09:37:14.0034461Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0034838Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0035290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0035692Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0036088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0036484Z self_outputs = self.self( 2025-12-04T09:37:14.0036861Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0037297Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0037794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0038378Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:37:14.0038788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:14.0039123Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:14.0039266Z 2025-12-04T09:37:14.0039362Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0039853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0040315Z layer_outputs = layer_module( 2025-12-04T09:37:14.0040636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0040983Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0041380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0041781Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0042168Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0042559Z self_outputs = self.self( 2025-12-04T09:37:14.0042938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0043372Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0043865Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0044386Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:37:14.0044863Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:37:14.0045315Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:37:14.0045634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:14.0045970Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:14.0046112Z 2025-12-04T09:37:14.0046218Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0046707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0047165Z layer_outputs = layer_module( 2025-12-04T09:37:14.0047496Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0047845Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0048234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0048667Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0049063Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0049456Z self_outputs = self.self( 2025-12-04T09:37:14.0049831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0050268Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0050773Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0051314Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:14.0051543Z 2025-12-04T09:37:14.0051641Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0052123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0052577Z layer_outputs = layer_module( 2025-12-04T09:37:14.0052893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0053233Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0053622Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0054012Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0054391Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0054776Z self_outputs = self.self( 2025-12-04T09:37:14.0055152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0055576Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0056053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0056575Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:14.0056760Z 2025-12-04T09:37:14.0056869Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0057360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0057807Z layer_outputs = layer_module( 2025-12-04T09:37:14.0058136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0058484Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0058872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0059268Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0059662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0060050Z self_outputs = self.self( 2025-12-04T09:37:14.0060416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:37:14.0060906Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:37:14.0061134Z 2025-12-04T09:37:14.0061210Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0061412Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0061622Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0062135Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0062596Z layer_outputs = layer_module( 2025-12-04T09:37:14.0062920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0063266Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0063659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:37:14.0064063Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:14.0064441Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:14.0064857Z return forward_fn(*input_tensors) 2025-12-04T09:37:14.0065256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:37:14.0065691Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:37:14.0066110Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:37:14.0066541Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:14.0066905Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:14.0067232Z return self.act(input) 2025-12-04T09:37:14.0067343Z 2025-12-04T09:37:14.0067416Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0067614Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0067835Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0068322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0068788Z layer_outputs = layer_module( 2025-12-04T09:37:14.0069121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0069467Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0069856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0070252Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0070645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0071028Z self_outputs = self.self( 2025-12-04T09:37:14.0071410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0071838Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0072317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0072873Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0073174Z 2025-12-04T09:37:14.0073257Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0073492Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0073996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0074464Z layer_outputs = layer_module( 2025-12-04T09:37:14.0074816Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0075171Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0075602Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0076004Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0076403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0076792Z self_outputs = self.self( 2025-12-04T09:37:14.0077160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0077579Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0078056Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0078616Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:37:14.0079081Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:37:14.0079484Z hidden_states = hidden_states.view( 2025-12-04T09:37:14.0079607Z 2025-12-04T09:37:14.0079712Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0080219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0080673Z layer_outputs = layer_module( 2025-12-04T09:37:14.0081005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0081355Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0081744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0082146Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0082537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0082930Z self_outputs = self.self( 2025-12-04T09:37:14.0083301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0083722Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0084195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0084743Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0084974Z 2025-12-04T09:37:14.0085075Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0085573Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0086035Z layer_outputs = layer_module( 2025-12-04T09:37:14.0086370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0086710Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0087111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0087511Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0087899Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0088292Z self_outputs = self.self( 2025-12-04T09:37:14.0088704Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0089126Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0089602Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0090159Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0090394Z 2025-12-04T09:37:14.0090492Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0090980Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0091438Z layer_outputs = layer_module( 2025-12-04T09:37:14.0091769Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0092175Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0092574Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0092960Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0093355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0093747Z self_outputs = self.self( 2025-12-04T09:37:14.0094119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0094540Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0095012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0095566Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0095797Z 2025-12-04T09:37:14.0095874Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0096077Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0096300Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0096792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0097245Z layer_outputs = layer_module( 2025-12-04T09:37:14.0097575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0097920Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0098319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0098712Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0099102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0099493Z self_outputs = self.self( 2025-12-04T09:37:14.0099864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:37:14.0100254Z attn_scores += diagonal_mask 2025-12-04T09:37:14.0100375Z 2025-12-04T09:37:14.0100470Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0100963Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0101415Z layer_outputs = layer_module( 2025-12-04T09:37:14.0101744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0102091Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0102529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0102916Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0103306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0103701Z self_outputs = self.self( 2025-12-04T09:37:14.0104069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:37:14.0104470Z attn_probs = nn.functional.softmax( 2025-12-04T09:37:14.0104598Z 2025-12-04T09:37:14.0104674Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0104900Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0105385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0106052Z layer_outputs = layer_module( 2025-12-04T09:37:14.0106390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0106748Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0107160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0107563Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0107962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0108355Z self_outputs = self.self( 2025-12-04T09:37:14.0108744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0109192Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0109694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0110249Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:37:14.0110656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:14.0110989Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:14.0111130Z 2025-12-04T09:37:14.0111235Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0111718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0112192Z layer_outputs = layer_module( 2025-12-04T09:37:14.0112529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0112882Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0113326Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0113745Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0114147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0114555Z self_outputs = self.self( 2025-12-04T09:37:14.0114931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0115371Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0115950Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0116475Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:37:14.0116973Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:37:14.0117434Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:37:14.0117768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:14.0118103Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:14.0118263Z 2025-12-04T09:37:14.0118366Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0118872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0119392Z layer_outputs = layer_module( 2025-12-04T09:37:14.0119726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0120080Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0120489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0120890Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0121297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0121701Z self_outputs = self.self( 2025-12-04T09:37:14.0122091Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0122530Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0123053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0123608Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:14.0123810Z 2025-12-04T09:37:14.0123918Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0124418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0124890Z layer_outputs = layer_module( 2025-12-04T09:37:14.0125235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0125594Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0125992Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0126405Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0126809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0127201Z self_outputs = self.self( 2025-12-04T09:37:14.0127594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0128029Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0128526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0129051Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:14.0129256Z 2025-12-04T09:37:14.0129352Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0129869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0130341Z layer_outputs = layer_module( 2025-12-04T09:37:14.0130664Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0131010Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0131405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0131805Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0132196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0132590Z self_outputs = self.self( 2025-12-04T09:37:14.0132968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:37:14.0133496Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:37:14.0133731Z 2025-12-04T09:37:14.0133806Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0134006Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0134225Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0134709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0135170Z layer_outputs = layer_module( 2025-12-04T09:37:14.0135497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0135847Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0136231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:37:14.0136643Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:14.0137031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:14.0137402Z return forward_fn(*input_tensors) 2025-12-04T09:37:14.0137795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:37:14.0138226Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:37:14.0138651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:37:14.0139080Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:14.0139444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:14.0139775Z return self.act(input) 2025-12-04T09:37:14.0139880Z 2025-12-04T09:37:14.0139966Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0140161Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0140385Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0140874Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0141329Z layer_outputs = layer_module( 2025-12-04T09:37:14.0141662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0142011Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0142407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0142797Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0143226Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0143620Z self_outputs = self.self( 2025-12-04T09:37:14.0143989Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0144411Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0144883Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0145435Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0145663Z 2025-12-04T09:37:14.0145744Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0145959Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0146480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0146944Z layer_outputs = layer_module( 2025-12-04T09:37:14.0147265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0147613Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0148010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0148408Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0148797Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0149188Z self_outputs = self.self( 2025-12-04T09:37:14.0149567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0149987Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0150459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0150983Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:37:14.0151456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:37:14.0151846Z hidden_states = hidden_states.view( 2025-12-04T09:37:14.0151978Z 2025-12-04T09:37:14.0152077Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0152571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0153057Z layer_outputs = layer_module( 2025-12-04T09:37:14.0153453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0153806Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0154211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0154619Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0155013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0155412Z self_outputs = self.self( 2025-12-04T09:37:14.0155798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0156213Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0156737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0157316Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0157561Z 2025-12-04T09:37:14.0157672Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0158176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0158662Z layer_outputs = layer_module( 2025-12-04T09:37:14.0159007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0159373Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0159782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0160230Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0160642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0161052Z self_outputs = self.self( 2025-12-04T09:37:14.0161432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0161864Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0162351Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0162911Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0163143Z 2025-12-04T09:37:14.0163243Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0163748Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0164223Z layer_outputs = layer_module( 2025-12-04T09:37:14.0164552Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0164910Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0165315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0165718Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0166113Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0166519Z self_outputs = self.self( 2025-12-04T09:37:14.0166905Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0167353Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0167820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0168375Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0168602Z 2025-12-04T09:37:14.0168684Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0168882Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0169094Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0169438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0169505Z layer_outputs = layer_module( 2025-12-04T09:37:14.0169715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0169838Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0170109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0170186Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0170452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0170518Z self_outputs = self.self( 2025-12-04T09:37:14.0170796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:37:14.0170863Z attn_scores += diagonal_mask 2025-12-04T09:37:14.0170867Z 2025-12-04T09:37:14.0170962Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0171303Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0171399Z layer_outputs = layer_module( 2025-12-04T09:37:14.0171613Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0171686Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0171946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0172022Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0172280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0172349Z self_outputs = self.self( 2025-12-04T09:37:14.0172608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:37:14.0172683Z attn_probs = nn.functional.softmax( 2025-12-04T09:37:14.0172689Z 2025-12-04T09:37:14.0172767Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0172862Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0173193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0173259Z layer_outputs = layer_module( 2025-12-04T09:37:14.0173470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0173548Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0173817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0173883Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0174156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0174219Z self_outputs = self.self( 2025-12-04T09:37:14.0174482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0174591Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0174918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0175087Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:37:14.0175271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:14.0175369Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:14.0175374Z 2025-12-04T09:37:14.0175471Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0175828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0175905Z layer_outputs = layer_module( 2025-12-04T09:37:14.0176118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0176197Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0176462Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0176533Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0176805Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0176898Z self_outputs = self.self( 2025-12-04T09:37:14.0177162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0177278Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0177612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0177747Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:37:14.0178052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:37:14.0178136Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:37:14.0178327Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:14.0178420Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:14.0178426Z 2025-12-04T09:37:14.0178533Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0178873Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0178942Z layer_outputs = layer_module( 2025-12-04T09:37:14.0179161Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0179236Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0179509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0179582Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0179848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0179925Z self_outputs = self.self( 2025-12-04T09:37:14.0180197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0180306Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0180650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0180793Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:14.0180796Z 2025-12-04T09:37:14.0180899Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0181248Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0181315Z layer_outputs = layer_module( 2025-12-04T09:37:14.0181569Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0181645Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0181917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0181990Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0182251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0182325Z self_outputs = self.self( 2025-12-04T09:37:14.0182590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0182705Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0183033Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0183213Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:14.0183216Z 2025-12-04T09:37:14.0183317Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0183650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0183724Z layer_outputs = layer_module( 2025-12-04T09:37:14.0183935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0184007Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0184280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0184348Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0184615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0184688Z self_outputs = self.self( 2025-12-04T09:37:14.0184955Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:37:14.0185140Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:37:14.0185143Z 2025-12-04T09:37:14.0185218Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0185291Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0185395Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0185728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0185806Z layer_outputs = layer_module( 2025-12-04T09:37:14.0186019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0186091Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0186365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:37:14.0186444Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:14.0186699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:14.0186781Z return forward_fn(*input_tensors) 2025-12-04T09:37:14.0187050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:37:14.0187158Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:37:14.0187421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:37:14.0187561Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:14.0187774Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:14.0187840Z return self.act(input) 2025-12-04T09:37:14.0187843Z 2025-12-04T09:37:14.0187922Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0187993Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0188088Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0188431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0188498Z layer_outputs = layer_module( 2025-12-04T09:37:14.0188707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0188816Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0189091Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0189168Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0189432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0189497Z self_outputs = self.self( 2025-12-04T09:37:14.0189766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0189860Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0190191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0190365Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0190370Z 2025-12-04T09:37:14.0190441Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0190542Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0190875Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0190947Z layer_outputs = layer_module( 2025-12-04T09:37:14.0191154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0191227Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0191497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0191567Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0191835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0191905Z self_outputs = self.self( 2025-12-04T09:37:14.0192169Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0192270Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0192590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0192735Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:37:14.0193008Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:37:14.0193080Z hidden_states = hidden_states.view( 2025-12-04T09:37:14.0193085Z 2025-12-04T09:37:14.0193265Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0193652Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0193721Z layer_outputs = layer_module( 2025-12-04T09:37:14.0193947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0194021Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0194296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0194368Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0194674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0194780Z self_outputs = self.self( 2025-12-04T09:37:14.0195042Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0195137Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0195473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0195647Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0195651Z 2025-12-04T09:37:14.0195757Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0196093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0196160Z layer_outputs = layer_module( 2025-12-04T09:37:14.0196384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0196461Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0196736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0196806Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0197072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0197144Z self_outputs = self.self( 2025-12-04T09:37:14.0197406Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0197505Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0197828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0198005Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0198008Z 2025-12-04T09:37:14.0198113Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0198447Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0198520Z layer_outputs = layer_module( 2025-12-04T09:37:14.0198730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0198804Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0199077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0199150Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0199503Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0199576Z self_outputs = self.self( 2025-12-04T09:37:14.0199842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0199941Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0200262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0200433Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0200437Z 2025-12-04T09:37:14.0200521Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0200594Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0200699Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0201077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0201145Z layer_outputs = layer_module( 2025-12-04T09:37:14.0201368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0201441Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0201724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0201794Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0202065Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0202139Z self_outputs = self.self( 2025-12-04T09:37:14.0202410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:37:14.0202484Z attn_scores += diagonal_mask 2025-12-04T09:37:14.0202488Z 2025-12-04T09:37:14.0202591Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0202938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0203013Z layer_outputs = layer_module( 2025-12-04T09:37:14.0203227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0203301Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0203583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0203653Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0203949Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0204016Z self_outputs = self.self( 2025-12-04T09:37:14.0204291Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:37:14.0204371Z attn_probs = nn.functional.softmax( 2025-12-04T09:37:14.0204374Z 2025-12-04T09:37:14.0204446Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0204542Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0204891Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0204956Z layer_outputs = layer_module( 2025-12-04T09:37:14.0205175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0205250Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0206585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0206699Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0206973Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0207047Z self_outputs = self.self( 2025-12-04T09:37:14.0207313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0207426Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0207775Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0208007Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:37:14.0208205Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:14.0208298Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:14.0208302Z 2025-12-04T09:37:14.0208409Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0208753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0208818Z layer_outputs = layer_module( 2025-12-04T09:37:14.0209029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0209108Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0209374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0209454Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0209721Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0209785Z self_outputs = self.self( 2025-12-04T09:37:14.0210056Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0210163Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0210506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0210634Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:37:14.0210942Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:37:14.0211041Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:37:14.0211228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:14.0211328Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:14.0211331Z 2025-12-04T09:37:14.0211426Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0211762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0211838Z layer_outputs = layer_module( 2025-12-04T09:37:14.0212048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0212120Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0212401Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0212506Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0212781Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0212845Z self_outputs = self.self( 2025-12-04T09:37:14.0213106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0213223Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0213555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0213707Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:14.0213710Z 2025-12-04T09:37:14.0213834Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0214169Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0214243Z layer_outputs = layer_module( 2025-12-04T09:37:14.0214452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0214532Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0214798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0214870Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0215140Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0215203Z self_outputs = self.self( 2025-12-04T09:37:14.0215472Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0215585Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0215916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0216061Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:14.0216065Z 2025-12-04T09:37:14.0216159Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0216491Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0216564Z layer_outputs = layer_module( 2025-12-04T09:37:14.0216775Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0216859Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0217122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0217192Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0217461Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0217525Z self_outputs = self.self( 2025-12-04T09:37:14.0217793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:37:14.0217967Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:37:14.0217970Z 2025-12-04T09:37:14.0218045Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0218127Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0218226Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0218595Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0218662Z layer_outputs = layer_module( 2025-12-04T09:37:14.0218872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0218951Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0219217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:37:14.0219295Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:14.0219554Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:14.0219654Z return forward_fn(*input_tensors) 2025-12-04T09:37:14.0219933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:37:14.0220035Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:37:14.0220299Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:37:14.0220413Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:14.0220618Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:14.0220690Z return self.act(input) 2025-12-04T09:37:14.0220693Z 2025-12-04T09:37:14.0220765Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0220836Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0220936Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0221283Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0221349Z layer_outputs = layer_module( 2025-12-04T09:37:14.0221563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0221635Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0221906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0221973Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0222236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0222306Z self_outputs = self.self( 2025-12-04T09:37:14.0222566Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0222670Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0222992Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0223162Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0223165Z 2025-12-04T09:37:14.0223246Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0223340Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0223672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0223744Z layer_outputs = layer_module( 2025-12-04T09:37:14.0223952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0224035Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0224330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0224401Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0224669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0224732Z self_outputs = self.self( 2025-12-04T09:37:14.0225010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0225103Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0225420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0225606Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:37:14.0225874Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:37:14.0225951Z hidden_states = hidden_states.view( 2025-12-04T09:37:14.0225954Z 2025-12-04T09:37:14.0226050Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0226385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0226458Z layer_outputs = layer_module( 2025-12-04T09:37:14.0226668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0226740Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0227016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0227091Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0227363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0227426Z self_outputs = self.self( 2025-12-04T09:37:14.0227688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0227790Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0228115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0228293Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0228296Z 2025-12-04T09:37:14.0228393Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0228734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0228808Z layer_outputs = layer_module( 2025-12-04T09:37:14.0229020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0229100Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0229369Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0229437Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0229710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0229774Z self_outputs = self.self( 2025-12-04T09:37:14.0230040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0230211Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0230530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0230711Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0230714Z 2025-12-04T09:37:14.0230813Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0231156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0231231Z layer_outputs = layer_module( 2025-12-04T09:37:14.0231445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0231553Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0231829Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0231900Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0232179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0232244Z self_outputs = self.self( 2025-12-04T09:37:14.0232518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0232612Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0232943Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0233192Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0233201Z 2025-12-04T09:37:14.0233283Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0233370Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0233472Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0233839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0233920Z layer_outputs = layer_module( 2025-12-04T09:37:14.0234195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0234268Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0234553Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0234628Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0234912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0234978Z self_outputs = self.self( 2025-12-04T09:37:14.0235250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:37:14.0235329Z attn_scores += diagonal_mask 2025-12-04T09:37:14.0235333Z 2025-12-04T09:37:14.0235431Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0235781Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0235849Z layer_outputs = layer_module( 2025-12-04T09:37:14.0236071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0236157Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0236461Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0236532Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0236807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0236871Z self_outputs = self.self( 2025-12-04T09:37:14.0237139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:37:14.0237213Z attn_probs = nn.functional.softmax( 2025-12-04T09:37:14.0237216Z 2025-12-04T09:37:14.0237289Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0237394Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0237739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0237840Z layer_outputs = layer_module( 2025-12-04T09:37:14.0238049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0238121Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0238393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0238462Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0238727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0238799Z self_outputs = self.self( 2025-12-04T09:37:14.0239063Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0239183Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0239520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0239682Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:37:14.0239875Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:14.0239968Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:14.0239972Z 2025-12-04T09:37:14.0240075Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0240409Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0240478Z layer_outputs = layer_module( 2025-12-04T09:37:14.0240699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0240773Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0241046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0241119Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0241385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0241458Z self_outputs = self.self( 2025-12-04T09:37:14.0241720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0241836Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0242171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0242334Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:37:14.0242649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:37:14.0242735Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:37:14.0242923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:14.0243024Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:14.0243027Z 2025-12-04T09:37:14.0243125Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0243471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0243585Z layer_outputs = layer_module( 2025-12-04T09:37:14.0243801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0243885Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0244158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0244236Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0244506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0244572Z self_outputs = self.self( 2025-12-04T09:37:14.0244850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0244958Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0245311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0245457Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:14.0245460Z 2025-12-04T09:37:14.0245557Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0245906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0245974Z layer_outputs = layer_module( 2025-12-04T09:37:14.0246196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0246278Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0246541Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0246620Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0246887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0246951Z self_outputs = self.self( 2025-12-04T09:37:14.0247219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0247332Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0247680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0247817Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:14.0247820Z 2025-12-04T09:37:14.0247914Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0248285Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0248351Z layer_outputs = layer_module( 2025-12-04T09:37:14.0248567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0248639Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0248903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0248980Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0249244Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0249314Z self_outputs = self.self( 2025-12-04T09:37:14.0249577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:37:14.0249786Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:37:14.0249789Z 2025-12-04T09:37:14.0249871Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0249941Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0250034Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0250376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0250441Z layer_outputs = layer_module( 2025-12-04T09:37:14.0250658Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0250730Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0250997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:37:14.0251091Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:14.0251346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:14.0251424Z return forward_fn(*input_tensors) 2025-12-04T09:37:14.0251699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:37:14.0251802Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:37:14.0252083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:37:14.0252200Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:14.0252404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:14.0252479Z return self.act(input) 2025-12-04T09:37:14.0252483Z 2025-12-04T09:37:14.0252557Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0252636Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0252731Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0253076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0253151Z layer_outputs = layer_module( 2025-12-04T09:37:14.0253375Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0253452Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0253716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0253786Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0254096Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0254164Z self_outputs = self.self( 2025-12-04T09:37:14.0254424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0254523Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0254840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0255015Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0255019Z 2025-12-04T09:37:14.0255090Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0255184Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0255524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0255618Z layer_outputs = layer_module( 2025-12-04T09:37:14.0255827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0255899Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0256156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0256232Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0256492Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0256561Z self_outputs = self.self( 2025-12-04T09:37:14.0256817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0256916Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0257237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0257379Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:37:14.0257636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:37:14.0257713Z hidden_states = hidden_states.view( 2025-12-04T09:37:14.0257716Z 2025-12-04T09:37:14.0257812Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0258144Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0258213Z layer_outputs = layer_module( 2025-12-04T09:37:14.0258420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0258500Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0258758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0258834Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0259093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0259156Z self_outputs = self.self( 2025-12-04T09:37:14.0259422Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0259514Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0259860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0260033Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0260036Z 2025-12-04T09:37:14.0260132Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0260468Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0260533Z layer_outputs = layer_module( 2025-12-04T09:37:14.0260746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0260818Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0261079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0261193Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0261454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0261518Z self_outputs = self.self( 2025-12-04T09:37:14.0261785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0261877Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0262199Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0262364Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0262367Z 2025-12-04T09:37:14.0262462Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0262802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0262869Z layer_outputs = layer_module( 2025-12-04T09:37:14.0263080Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0263152Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0263407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0263481Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0263737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0263808Z self_outputs = self.self( 2025-12-04T09:37:14.0264067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0264160Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0264482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0264648Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0264651Z 2025-12-04T09:37:14.0264731Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0264803Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0264898Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0265229Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0265296Z layer_outputs = layer_module( 2025-12-04T09:37:14.0265501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0265614Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0265886Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0265963Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0266228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0266292Z self_outputs = self.self( 2025-12-04T09:37:14.0266564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:37:14.0266632Z attn_scores += diagonal_mask 2025-12-04T09:37:14.0266635Z 2025-12-04T09:37:14.0266731Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0267072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0267166Z layer_outputs = layer_module( 2025-12-04T09:37:14.0267382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0267456Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0267718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0267794Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0268060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0268131Z self_outputs = self.self( 2025-12-04T09:37:14.0268394Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:37:14.0268469Z attn_probs = nn.functional.softmax( 2025-12-04T09:37:14.0268473Z 2025-12-04T09:37:14.0268556Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0268650Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0268990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0269057Z layer_outputs = layer_module( 2025-12-04T09:37:14.0269265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0269346Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0269610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0269677Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0269954Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0270017Z self_outputs = self.self( 2025-12-04T09:37:14.0270282Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0270390Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0270721Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0270890Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:37:14.0271074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:14.0271170Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:14.0271175Z 2025-12-04T09:37:14.0271271Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0271636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0271709Z layer_outputs = layer_module( 2025-12-04T09:37:14.0271916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0271995Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0272257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0272326Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0272593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0272656Z self_outputs = self.self( 2025-12-04T09:37:14.0272959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0273073Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0273481Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0273619Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:37:14.0273922Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:37:14.0274008Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:37:14.0274204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:14.0274297Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:14.0274305Z 2025-12-04T09:37:14.0274410Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0274758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0274826Z layer_outputs = layer_module( 2025-12-04T09:37:14.0275054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0275128Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0275413Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0275485Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0275762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0275839Z self_outputs = self.self( 2025-12-04T09:37:14.0276143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0276253Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0276619Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0276765Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:14.0276769Z 2025-12-04T09:37:14.0276874Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0277215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0277283Z layer_outputs = layer_module( 2025-12-04T09:37:14.0277512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0277622Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0277907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0277979Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0278252Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0278326Z self_outputs = self.self( 2025-12-04T09:37:14.0278598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0278716Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0279057Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0279235Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:14.0279238Z 2025-12-04T09:37:14.0279344Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0279688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0279763Z layer_outputs = layer_module( 2025-12-04T09:37:14.0279977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0280052Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0280332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0280402Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0280678Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0280753Z self_outputs = self.self( 2025-12-04T09:37:14.0281025Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:37:14.0281211Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:37:14.0281214Z 2025-12-04T09:37:14.0281289Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0281363Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0281469Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0281811Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0281886Z layer_outputs = layer_module( 2025-12-04T09:37:14.0282105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0282179Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0282462Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:37:14.0282543Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:14.0282798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:14.0282878Z return forward_fn(*input_tensors) 2025-12-04T09:37:14.0283156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:37:14.0283267Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:37:14.0283540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:37:14.0283685Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:14.0283907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:14.0283974Z return self.act(input) 2025-12-04T09:37:14.0283978Z 2025-12-04T09:37:14.0284059Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0284132Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0284230Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0284589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0284658Z layer_outputs = layer_module( 2025-12-04T09:37:14.0284873Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0284985Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0285262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0285338Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0285614Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0285680Z self_outputs = self.self( 2025-12-04T09:37:14.0285959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0286052Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0286390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0286567Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0286572Z 2025-12-04T09:37:14.0286645Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0286747Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0287095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0287167Z layer_outputs = layer_module( 2025-12-04T09:37:14.0287382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0287457Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0287735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0287817Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0288085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0288156Z self_outputs = self.self( 2025-12-04T09:37:14.0288418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0288516Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0288840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0288983Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:37:14.0289264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:37:14.0289331Z hidden_states = hidden_states.view( 2025-12-04T09:37:14.0289337Z 2025-12-04T09:37:14.0289438Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0289804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0289872Z layer_outputs = layer_module( 2025-12-04T09:37:14.0290083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0290156Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0290423Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0290493Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0290749Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0290848Z self_outputs = self.self( 2025-12-04T09:37:14.0291117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0291212Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0291550Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0291719Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0291722Z 2025-12-04T09:37:14.0291827Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0292164Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0292229Z layer_outputs = layer_module( 2025-12-04T09:37:14.0292447Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0292524Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0292802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0292873Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0293145Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0293217Z self_outputs = self.self( 2025-12-04T09:37:14.0293484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0293584Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0293907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0294081Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0294085Z 2025-12-04T09:37:14.0294188Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0294524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0294597Z layer_outputs = layer_module( 2025-12-04T09:37:14.0294809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0294881Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0295156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0295225Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0295537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0295611Z self_outputs = self.self( 2025-12-04T09:37:14.0295878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0295980Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0296305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0296479Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0296482Z 2025-12-04T09:37:14.0296566Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0296639Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0296745Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0297116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0297183Z layer_outputs = layer_module( 2025-12-04T09:37:14.0297397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0297469Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0297741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0297811Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0298077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0298146Z self_outputs = self.self( 2025-12-04T09:37:14.0298410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:37:14.0298482Z attn_scores += diagonal_mask 2025-12-04T09:37:14.0298485Z 2025-12-04T09:37:14.0298588Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0298918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0298991Z layer_outputs = layer_module( 2025-12-04T09:37:14.0299198Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0299269Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0299541Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0299609Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0299887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0299951Z self_outputs = self.self( 2025-12-04T09:37:14.0300218Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:37:14.0300299Z attn_probs = nn.functional.softmax( 2025-12-04T09:37:14.0300303Z 2025-12-04T09:37:14.0300373Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0300467Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0300804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0300869Z layer_outputs = layer_module( 2025-12-04T09:37:14.0301082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0301156Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0301453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0301531Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0301798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0301870Z self_outputs = self.self( 2025-12-04T09:37:14.0302134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0302243Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0302591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0302786Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:37:14.0302978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:14.0303069Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:14.0303073Z 2025-12-04T09:37:14.0303166Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0303510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0303575Z layer_outputs = layer_module( 2025-12-04T09:37:14.0303782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0303860Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0304123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0304200Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0304467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0304530Z self_outputs = self.self( 2025-12-04T09:37:14.0304802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0304910Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0305253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0305378Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:37:14.0305823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:37:14.0305930Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:37:14.0306112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:14.0306209Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:14.0306213Z 2025-12-04T09:37:14.0306308Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0306640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0306716Z layer_outputs = layer_module( 2025-12-04T09:37:14.0306926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0306998Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0307284Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0307412Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0307693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0307770Z self_outputs = self.self( 2025-12-04T09:37:14.0308035Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0308151Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0308487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0308638Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:14.0308641Z 2025-12-04T09:37:14.0308735Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0309123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0309201Z layer_outputs = layer_module( 2025-12-04T09:37:14.0309411Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0309493Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0309765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0309837Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0310119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0310184Z self_outputs = self.self( 2025-12-04T09:37:14.0310473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0310588Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0310929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0311079Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:14.0311082Z 2025-12-04T09:37:14.0311177Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0311517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0311602Z layer_outputs = layer_module( 2025-12-04T09:37:14.0311815Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0311898Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0312231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0312302Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0312583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0312647Z self_outputs = self.self( 2025-12-04T09:37:14.0312924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:37:14.0313099Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:37:14.0313151Z 2025-12-04T09:37:14.0313234Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0313316Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0313419Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0313800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0313870Z layer_outputs = layer_module( 2025-12-04T09:37:14.0314081Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0314162Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0314429Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:37:14.0314510Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:14.0314768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:14.0314842Z return forward_fn(*input_tensors) 2025-12-04T09:37:14.0315162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:37:14.0315266Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:37:14.0315539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:37:14.0315657Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:14.0315867Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:14.0316240Z return self.act(input) 2025-12-04T09:37:14.0316351Z 2025-12-04T09:37:14.0316434Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0316636Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0316861Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0317364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0317841Z layer_outputs = layer_module( 2025-12-04T09:37:14.0318198Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0318557Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0318962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0319358Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0319816Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0320228Z self_outputs = self.self( 2025-12-04T09:37:14.0320628Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0321060Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0321555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0322134Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0322371Z 2025-12-04T09:37:14.0322509Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0322764Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0323271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0323736Z layer_outputs = layer_module( 2025-12-04T09:37:14.0324078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0324482Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0324934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0325345Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0325754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0326162Z self_outputs = self.self( 2025-12-04T09:37:14.0326544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0326973Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0327445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0327987Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:37:14.0328453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:37:14.0328852Z hidden_states = hidden_states.view( 2025-12-04T09:37:14.0328985Z 2025-12-04T09:37:14.0329084Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0329574Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0330030Z layer_outputs = layer_module( 2025-12-04T09:37:14.0330366Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0330711Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0331111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0331501Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0331900Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0332292Z self_outputs = self.self( 2025-12-04T09:37:14.0332665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0333085Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0333558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0334111Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0334342Z 2025-12-04T09:37:14.0334438Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0334932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0335389Z layer_outputs = layer_module( 2025-12-04T09:37:14.0335720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0336057Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0336450Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0336846Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0337239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0337620Z self_outputs = self.self( 2025-12-04T09:37:14.0337995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0338469Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0338942Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0339496Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0339731Z 2025-12-04T09:37:14.0339827Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0340317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0340774Z layer_outputs = layer_module( 2025-12-04T09:37:14.0341110Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0341497Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0341898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0342286Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0342680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0343072Z self_outputs = self.self( 2025-12-04T09:37:14.0343454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0343867Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0344339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0344894Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0345120Z 2025-12-04T09:37:14.0345207Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0345400Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0345622Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0346107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0346562Z layer_outputs = layer_module( 2025-12-04T09:37:14.0346900Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0347248Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0347639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0348031Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0348429Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0348818Z self_outputs = self.self( 2025-12-04T09:37:14.0349187Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:37:14.0349581Z attn_scores += diagonal_mask 2025-12-04T09:37:14.0349700Z 2025-12-04T09:37:14.0349798Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0350280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0350732Z layer_outputs = layer_module( 2025-12-04T09:37:14.0351065Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0351416Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0351848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0352250Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0352652Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0353048Z self_outputs = self.self( 2025-12-04T09:37:14.0353518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:37:14.0353934Z attn_probs = nn.functional.softmax( 2025-12-04T09:37:14.0354075Z 2025-12-04T09:37:14.0354153Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0354389Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0354905Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0355430Z layer_outputs = layer_module( 2025-12-04T09:37:14.0355770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0356127Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0356533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0356939Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0357340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0357737Z self_outputs = self.self( 2025-12-04T09:37:14.0358118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0358568Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0359087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0359654Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:37:14.0360076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:14.0360421Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:14.0360568Z 2025-12-04T09:37:14.0360677Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0361179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0361662Z layer_outputs = layer_module( 2025-12-04T09:37:14.0362008Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0362370Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0362773Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0363188Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0363597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0364002Z self_outputs = self.self( 2025-12-04T09:37:14.0364388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0364833Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0365343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0365905Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:37:14.0366425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:37:14.0366936Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:37:14.0367264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:14.0367596Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:14.0367746Z 2025-12-04T09:37:14.0367842Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0368334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0368828Z layer_outputs = layer_module( 2025-12-04T09:37:14.0369158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0369508Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0369910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0370310Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0370699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0371089Z self_outputs = self.self( 2025-12-04T09:37:14.0371466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0371892Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0372392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0372930Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:14.0373123Z 2025-12-04T09:37:14.0373227Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0373706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0374165Z layer_outputs = layer_module( 2025-12-04T09:37:14.0374496Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0374843Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0375227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0375623Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0376021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0376409Z self_outputs = self.self( 2025-12-04T09:37:14.0376779Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0377211Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0377707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0378234Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:14.0378434Z 2025-12-04T09:37:14.0378549Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0379076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0379546Z layer_outputs = layer_module( 2025-12-04T09:37:14.0379872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0380219Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0380609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0381005Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0381396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0381780Z self_outputs = self.self( 2025-12-04T09:37:14.0382160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:37:14.0382692Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:37:14.0382916Z 2025-12-04T09:37:14.0382998Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0383193Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0383415Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0383906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0384368Z layer_outputs = layer_module( 2025-12-04T09:37:14.0384688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0385032Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0385428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:37:14.0385831Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:14.0386218Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:14.0386598Z return forward_fn(*input_tensors) 2025-12-04T09:37:14.0386993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:37:14.0387417Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:37:14.0387842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:37:14.0388272Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:14.0388636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:14.0388958Z return self.act(input) 2025-12-04T09:37:14.0389071Z 2025-12-04T09:37:14.0389148Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0389347Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0389565Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0390056Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0390519Z layer_outputs = layer_module( 2025-12-04T09:37:14.0390853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0391195Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0391590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0391987Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0392412Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0392804Z self_outputs = self.self( 2025-12-04T09:37:14.0393351Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0393862Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0394412Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0395010Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0395283Z 2025-12-04T09:37:14.0395368Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0395625Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0396193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0396900Z layer_outputs = layer_module( 2025-12-04T09:37:14.0397622Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0398110Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0398645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0399245Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0418181Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0418631Z self_outputs = self.self( 2025-12-04T09:37:14.0419036Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0419485Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0419961Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0420492Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:37:14.0420960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:37:14.0421360Z hidden_states = hidden_states.view( 2025-12-04T09:37:14.0421491Z 2025-12-04T09:37:14.0421594Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0422087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0422562Z layer_outputs = layer_module( 2025-12-04T09:37:14.0422909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0423252Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0423653Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0424056Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0424447Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0424841Z self_outputs = self.self( 2025-12-04T09:37:14.0425221Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0425644Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0426106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0426851Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0427097Z 2025-12-04T09:37:14.0427201Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0427700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0428156Z layer_outputs = layer_module( 2025-12-04T09:37:14.0428492Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0428843Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0429240Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0429711Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0430118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0430521Z self_outputs = self.self( 2025-12-04T09:37:14.0430897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0431319Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0431798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0432356Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0432588Z 2025-12-04T09:37:14.0432691Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0433287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0433790Z layer_outputs = layer_module( 2025-12-04T09:37:14.0434152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0434511Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0434926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0435344Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0435750Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0436149Z self_outputs = self.self( 2025-12-04T09:37:14.0436547Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:37:14.0436988Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:37:14.0437485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:37:14.0438056Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:37:14.0438303Z 2025-12-04T09:37:14.0438383Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0438591Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0438814Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0439323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0439803Z layer_outputs = layer_module( 2025-12-04T09:37:14.0440205Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0440616Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0441033Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0441449Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0441871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0442274Z self_outputs = self.self( 2025-12-04T09:37:14.0442670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:37:14.0443084Z attn_scores += diagonal_mask 2025-12-04T09:37:14.0443202Z 2025-12-04T09:37:14.0443299Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0443810Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0444325Z layer_outputs = layer_module( 2025-12-04T09:37:14.0444661Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0445009Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0445418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0445828Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0446232Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0446632Z self_outputs = self.self( 2025-12-04T09:37:14.0447018Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:37:14.0447434Z attn_probs = nn.functional.softmax( 2025-12-04T09:37:14.0447561Z 2025-12-04T09:37:14.0447646Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0447872Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0448375Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0448858Z layer_outputs = layer_module( 2025-12-04T09:37:14.0449179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0449525Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0449920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0450313Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0450701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0451100Z self_outputs = self.self( 2025-12-04T09:37:14.0451477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0451921Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0452418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0452978Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:37:14.0453393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:14.0453727Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:14.0453879Z 2025-12-04T09:37:14.0453981Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0454503Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0454974Z layer_outputs = layer_module( 2025-12-04T09:37:14.0455299Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0455647Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0456043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0456437Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0456821Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0457214Z self_outputs = self.self( 2025-12-04T09:37:14.0457646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0458075Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0458565Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0459078Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:37:14.0459557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:37:14.0459994Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:37:14.0460318Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:37:14.0460647Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:37:14.0460791Z 2025-12-04T09:37:14.0460898Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0461376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0461830Z layer_outputs = layer_module( 2025-12-04T09:37:14.0462156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0462496Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0462885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0463283Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0463675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0464064Z self_outputs = self.self( 2025-12-04T09:37:14.0464435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0464871Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0465359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0465881Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:14.0466078Z 2025-12-04T09:37:14.0466173Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0466655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0467108Z layer_outputs = layer_module( 2025-12-04T09:37:14.0467428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0467801Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0468196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0468588Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0468970Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0469357Z self_outputs = self.self( 2025-12-04T09:37:14.0469729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:37:14.0470160Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:37:14.0470646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:37:14.0471210Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:37:14.0471407Z 2025-12-04T09:37:14.0471514Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0472003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0472456Z layer_outputs = layer_module( 2025-12-04T09:37:14.0472787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0473217Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0473647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:37:14.0474075Z self_attn_outputs = self.attention( 2025-12-04T09:37:14.0474500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:37:14.0474898Z self_outputs = self.self( 2025-12-04T09:37:14.0475283Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:37:14.0475807Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:37:14.0476041Z 2025-12-04T09:37:14.0476130Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0476339Z cudagraph partition due to non gpu ops 2025-12-04T09:37:14.0476561Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:14.0477077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:37:14.0477553Z layer_outputs = layer_module( 2025-12-04T09:37:14.0477894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:14.0478251Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:14.0478659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:37:14.0479079Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:14.0479472Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:14.0479863Z return forward_fn(*input_tensors) 2025-12-04T09:37:14.0480274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:37:14.0480725Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:37:14.0481158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:37:14.0481647Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:14.0482022Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:14.0482353Z return self.act(input) 2025-12-04T09:37:14.0482470Z 2025-12-04T09:37:14.0482546Z cudagraph partition due to non gpu ops 2025-12-04T09:37:52.4291503Z cudagraph partition due to non gpu ops 2025-12-04T09:37:52.4295698Z cudagraph partition due to non gpu ops 2025-12-04T09:37:52.4299730Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:52.4303902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1722, in torch_dynamo_resume_in_forward_at_1702 2025-12-04T09:37:52.4305367Z masked_lm_loss = loss_fct(prediction_scores.view(-1, self.config.vocab_size), labels.view(-1)) 2025-12-04T09:37:52.4306206Z 2025-12-04T09:37:53.8635456Z Compilation time (from dynamo_timed): 70.730265267 2025-12-04T09:37:53.8838360Z pass 2025-12-04T09:37:53.8842852Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:37:53.8847426Z TIMING: gc:0.0048 entire_frame_compile:70.73027 _recursive_pre_grad_passes:0.10929 _recursive_joint_graph_passes:0.91486 _recursive_post_grad_passes:0.817 async_compile.wait:2.80124 code_gen:48.06756 inductor_compile:53.724 backend_compile:65.38803 total_wall_time:70.73027 2025-12-04T09:37:53.8848741Z STATS: call_* op count: 1787 | FakeTensorMode.__torch_dispatch__:56316 | FakeTensor.__torch_dispatch__:8931 | ProxyTorchDispatchMode.__torch_dispatch__:10841 2025-12-04T09:37:53.8849227Z Dynamo produced 4 graphs covering 1787 ops with 4 graph breaks (1 unique) 2025-12-04T09:37:56.8893830Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:37:56.8894836Z import pynvml # type: ignore[import] 2025-12-04T09:38:00.0167983Z 2025-12-04T09:38:02.1794648Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:38:02.1796443Z loading model: 0it [00:02, ?it/s] 2025-12-04T09:38:02.1810453Z cpu eval BartForCausalLM 2025-12-04T09:38:03.7636653Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:38:04.2931712Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:38:04.8195850Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:38:13.0361757Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0365974Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0369470Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0371495Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0371875Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0374306Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0374667Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0379775Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0381465Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0381840Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0382089Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0382365Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0382671Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0382956Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0383364Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0386213Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0388005Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0388672Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0388978Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0389515Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0389898Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0390248Z res = mod(**inputs) 2025-12-04T09:38:13.0390639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0391036Z outputs = self.model.decoder( 2025-12-04T09:38:13.0391410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0391788Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0392138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0392496Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0392980Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0393514Z return func(*args, **kwargs) 2025-12-04T09:38:13.0393896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:38:13.0394309Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:38:13.0394724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0395107Z return func(*args, **kwargs) 2025-12-04T09:38:13.0395461Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:38:13.0395870Z attn_output, attn_weights = attention_interface( 2025-12-04T09:38:13.0396324Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:38:13.0396827Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:13.0397023Z 2025-12-04T09:38:13.0397106Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0397317Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0397563Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0397929Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0398257Z res = mod(**inputs) 2025-12-04T09:38:13.0398627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0399027Z outputs = self.model.decoder( 2025-12-04T09:38:13.0399405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0399794Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0400195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0400570Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0400946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0401326Z return func(*args, **kwargs) 2025-12-04T09:38:13.0401695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:38:13.0402141Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:38:13.0402533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:13.0402885Z return self.act(input) 2025-12-04T09:38:13.0402996Z 2025-12-04T09:38:13.0403082Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0403283Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0403491Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0403695Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0403956Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0404162Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0404361Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0404561Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0404786Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0405141Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0405461Z res = mod(**inputs) 2025-12-04T09:38:13.0406127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0406545Z outputs = self.model.decoder( 2025-12-04T09:38:13.0406911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0407341Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0407673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0408024Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0408389Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0408738Z return func(*args, **kwargs) 2025-12-04T09:38:13.0409087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:38:13.0409474Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:38:13.0409852Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0410201Z return func(*args, **kwargs) 2025-12-04T09:38:13.0410553Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:38:13.0410947Z attn_output, attn_weights = attention_interface( 2025-12-04T09:38:13.0411367Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:38:13.0411827Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:13.0412008Z 2025-12-04T09:38:13.0412083Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0412282Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0412495Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0412840Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0413149Z res = mod(**inputs) 2025-12-04T09:38:13.0413485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0413859Z outputs = self.model.decoder( 2025-12-04T09:38:13.0414241Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0414607Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0414933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0415282Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0415647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0416000Z return func(*args, **kwargs) 2025-12-04T09:38:13.0416345Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:38:13.0416749Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:38:13.0417125Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:13.0417498Z return self.act(input) 2025-12-04T09:38:13.0417612Z 2025-12-04T09:38:13.0417687Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0417895Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0418096Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0418292Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0418490Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0418689Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0418880Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0419078Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0419307Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0419645Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0420035Z res = mod(**inputs) 2025-12-04T09:38:13.0420384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0420817Z outputs = self.model.decoder( 2025-12-04T09:38:13.0421168Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0421543Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0421875Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0422213Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0422573Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0422933Z return func(*args, **kwargs) 2025-12-04T09:38:13.0423292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:38:13.0423671Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:38:13.0424058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0424415Z return func(*args, **kwargs) 2025-12-04T09:38:13.0424766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:38:13.0425147Z attn_output, attn_weights = attention_interface( 2025-12-04T09:38:13.0425572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:38:13.0426029Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:13.0426200Z 2025-12-04T09:38:13.0426274Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0426474Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0426696Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0427042Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0427345Z res = mod(**inputs) 2025-12-04T09:38:13.0427686Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0428053Z outputs = self.model.decoder( 2025-12-04T09:38:13.0428406Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0428772Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0429105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0429450Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0429802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0430157Z return func(*args, **kwargs) 2025-12-04T09:38:13.0430541Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:38:13.0430942Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:38:13.0431319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:13.0431648Z return self.act(input) 2025-12-04T09:38:13.0431752Z 2025-12-04T09:38:13.0431832Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0432022Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0432219Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0432413Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0432601Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0432794Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0432993Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0433285Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0433550Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0433912Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0434240Z res = mod(**inputs) 2025-12-04T09:38:13.0434611Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0434987Z outputs = self.model.decoder( 2025-12-04T09:38:13.0435350Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0435777Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0436119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0436483Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0436860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0437232Z return func(*args, **kwargs) 2025-12-04T09:38:13.0437603Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:38:13.0438013Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:38:13.0438419Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0438784Z return func(*args, **kwargs) 2025-12-04T09:38:13.0439149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:38:13.0439558Z attn_output, attn_weights = attention_interface( 2025-12-04T09:38:13.0440001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:38:13.0440482Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:13.0440675Z 2025-12-04T09:38:13.0440757Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0440970Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0441195Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0441558Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0441884Z res = mod(**inputs) 2025-12-04T09:38:13.0442317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0442720Z outputs = self.model.decoder( 2025-12-04T09:38:13.0443097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0443532Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0443867Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0444227Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0444634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0444994Z return func(*args, **kwargs) 2025-12-04T09:38:13.0445339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:38:13.0445747Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:38:13.0446120Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:13.0446440Z return self.act(input) 2025-12-04T09:38:13.0446553Z 2025-12-04T09:38:13.0446627Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0446823Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0447018Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0447232Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0447425Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0447620Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0447806Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0447998Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0448218Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0448550Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0448858Z res = mod(**inputs) 2025-12-04T09:38:13.0449202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0449572Z outputs = self.model.decoder( 2025-12-04T09:38:13.0449924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0450289Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0450630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0450972Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0451334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0451692Z return func(*args, **kwargs) 2025-12-04T09:38:13.0452041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:38:13.0452420Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:38:13.0452802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0453155Z return func(*args, **kwargs) 2025-12-04T09:38:13.0453499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:38:13.0453888Z attn_output, attn_weights = attention_interface( 2025-12-04T09:38:13.0454314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:38:13.0454770Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:13.0454943Z 2025-12-04T09:38:13.0455017Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0455216Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0455436Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0455777Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0456079Z res = mod(**inputs) 2025-12-04T09:38:13.0456424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0456791Z outputs = self.model.decoder( 2025-12-04T09:38:13.0457204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0457576Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0457912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0458259Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0458614Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0458972Z return func(*args, **kwargs) 2025-12-04T09:38:13.0459323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:38:13.0459722Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:38:13.0460097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:13.0460459Z return self.act(input) 2025-12-04T09:38:13.0460565Z 2025-12-04T09:38:13.0460649Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0460838Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0461035Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0461227Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0461410Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0461605Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0461796Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0461980Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0462200Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0462546Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0462856Z res = mod(**inputs) 2025-12-04T09:38:13.0463193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0463567Z outputs = self.model.decoder( 2025-12-04T09:38:13.0463928Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0464293Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0464618Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0464964Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0465331Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0465682Z return func(*args, **kwargs) 2025-12-04T09:38:13.0466032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:38:13.0466421Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:38:13.0466802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0467154Z return func(*args, **kwargs) 2025-12-04T09:38:13.0467500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:38:13.0467884Z attn_output, attn_weights = attention_interface( 2025-12-04T09:38:13.0468303Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:38:13.0468762Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:13.0468936Z 2025-12-04T09:38:13.0469009Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0469207Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0469423Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0469760Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0470069Z res = mod(**inputs) 2025-12-04T09:38:13.0470446Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0470857Z outputs = self.model.decoder( 2025-12-04T09:38:13.0471219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0471588Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0471918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0472265Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0472631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0473018Z return func(*args, **kwargs) 2025-12-04T09:38:13.0473437Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:38:13.0473903Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:38:13.0474288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:13.0474676Z return self.act(input) 2025-12-04T09:38:13.0474792Z 2025-12-04T09:38:13.0474867Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0475069Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0475275Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0475465Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0475665Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0475868Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0476059Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0476256Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0476480Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0476830Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0477147Z res = mod(**inputs) 2025-12-04T09:38:13.0477500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0477889Z outputs = self.model.decoder( 2025-12-04T09:38:13.0478255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0478629Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0478970Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0479318Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0479692Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0480062Z return func(*args, **kwargs) 2025-12-04T09:38:13.0480423Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:38:13.0480813Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:38:13.0481208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0481572Z return func(*args, **kwargs) 2025-12-04T09:38:13.0481923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:38:13.0482319Z attn_output, attn_weights = attention_interface( 2025-12-04T09:38:13.0482752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:38:13.0483221Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:13.0483399Z 2025-12-04T09:38:13.0483472Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0483702Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0483929Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0484280Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0484592Z res = mod(**inputs) 2025-12-04T09:38:13.0484945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0485329Z outputs = self.model.decoder( 2025-12-04T09:38:13.0485690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0486066Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0486409Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0486772Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0487169Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0487533Z return func(*args, **kwargs) 2025-12-04T09:38:13.0487892Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:38:13.0488309Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:38:13.0488678Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:13.0489005Z return self.act(input) 2025-12-04T09:38:13.0489109Z 2025-12-04T09:38:13.0489190Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0489381Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0489576Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0489770Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0489955Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0490148Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0490345Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0490531Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0490748Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0491093Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0491403Z res = mod(**inputs) 2025-12-04T09:38:13.0491741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0492114Z outputs = self.model.decoder( 2025-12-04T09:38:13.0492478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0492835Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0493164Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0493521Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0493883Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0494236Z return func(*args, **kwargs) 2025-12-04T09:38:13.0494591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:38:13.0494978Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:38:13.0495361Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0495709Z return func(*args, **kwargs) 2025-12-04T09:38:13.0496057Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:38:13.0496442Z attn_output, attn_weights = attention_interface( 2025-12-04T09:38:13.0497530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:38:13.0498003Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:13.0498180Z 2025-12-04T09:38:13.0498254Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0498451Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0498665Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0499013Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0499324Z res = mod(**inputs) 2025-12-04T09:38:13.0499658Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0500030Z outputs = self.model.decoder( 2025-12-04T09:38:13.0500390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0500784Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0501110Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0501455Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0501814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0502170Z return func(*args, **kwargs) 2025-12-04T09:38:13.0502512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:38:13.0502920Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:38:13.0503290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:13.0503611Z return self.act(input) 2025-12-04T09:38:13.0503726Z 2025-12-04T09:38:13.0503801Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0504005Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0504202Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0504386Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0504577Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0504769Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0504953Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0505142Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0505361Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0505841Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0506164Z res = mod(**inputs) 2025-12-04T09:38:13.0506515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0506887Z outputs = self.model.decoder( 2025-12-04T09:38:13.0507248Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0507620Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0507956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0508301Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0508667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0509028Z return func(*args, **kwargs) 2025-12-04T09:38:13.0509382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:38:13.0509766Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:38:13.0510149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0510510Z return func(*args, **kwargs) 2025-12-04T09:38:13.0510921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:38:13.0511311Z attn_output, attn_weights = attention_interface( 2025-12-04T09:38:13.0511734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:38:13.0512189Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:13.0512362Z 2025-12-04T09:38:13.0512435Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0512635Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0512863Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0513260Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0513576Z res = mod(**inputs) 2025-12-04T09:38:13.0513989Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0514382Z outputs = self.model.decoder( 2025-12-04T09:38:13.0514738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0515170Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0515519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0515878Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0516243Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0516614Z return func(*args, **kwargs) 2025-12-04T09:38:13.0516980Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:38:13.0517396Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:38:13.0517783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:13.0518123Z return self.act(input) 2025-12-04T09:38:13.0518229Z 2025-12-04T09:38:13.0518311Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0518505Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0518702Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0518899Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0519086Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0519284Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0519485Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0519673Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0519899Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0520249Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0520572Z res = mod(**inputs) 2025-12-04T09:38:13.0520920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0521308Z outputs = self.model.decoder( 2025-12-04T09:38:13.0521681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0522052Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0522396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0522760Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0523133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0523498Z return func(*args, **kwargs) 2025-12-04T09:38:13.0523861Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:38:13.0524307Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:38:13.0524709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0525074Z return func(*args, **kwargs) 2025-12-04T09:38:13.0525438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:38:13.0525845Z attn_output, attn_weights = attention_interface( 2025-12-04T09:38:13.0526282Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:38:13.0526764Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:13.0526952Z 2025-12-04T09:38:13.0527031Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0527265Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0527496Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0527835Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0528143Z res = mod(**inputs) 2025-12-04T09:38:13.0528479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0528850Z outputs = self.model.decoder( 2025-12-04T09:38:13.0529208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0529573Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0529902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0530246Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0530609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0530970Z return func(*args, **kwargs) 2025-12-04T09:38:13.0531316Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:38:13.0531724Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:38:13.0532090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:13.0532410Z return self.act(input) 2025-12-04T09:38:13.0532521Z 2025-12-04T09:38:13.0532597Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0532794Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0532989Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0533174Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0533368Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0533561Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0533750Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0533944Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0534163Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0534497Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0534807Z res = mod(**inputs) 2025-12-04T09:38:13.0535152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0535523Z outputs = self.model.decoder( 2025-12-04T09:38:13.0535881Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0536256Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0536589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0536934Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0537327Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0537682Z return func(*args, **kwargs) 2025-12-04T09:38:13.0538022Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:38:13.0538393Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:38:13.0538768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0539114Z return func(*args, **kwargs) 2025-12-04T09:38:13.0539452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:38:13.0539830Z attn_output, attn_weights = attention_interface( 2025-12-04T09:38:13.0540245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:38:13.0540733Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:13.0540906Z 2025-12-04T09:38:13.0540979Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0541177Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0541401Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0541739Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0542040Z res = mod(**inputs) 2025-12-04T09:38:13.0542383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0542751Z outputs = self.model.decoder( 2025-12-04T09:38:13.0543104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0543475Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0543809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0544153Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0544508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0544864Z return func(*args, **kwargs) 2025-12-04T09:38:13.0545209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:38:13.0545604Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:38:13.0545974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:13.0546300Z return self.act(input) 2025-12-04T09:38:13.0546403Z 2025-12-04T09:38:13.0546484Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0546676Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0546873Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0547070Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0547257Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0547452Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0547646Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0547832Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0548051Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0548392Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0548702Z res = mod(**inputs) 2025-12-04T09:38:13.0549036Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0549406Z outputs = self.model.decoder( 2025-12-04T09:38:13.0549765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0550159Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0550490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0550833Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0551190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0551529Z return func(*args, **kwargs) 2025-12-04T09:38:13.0551874Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:38:13.0552258Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:38:13.0552639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0552979Z return func(*args, **kwargs) 2025-12-04T09:38:13.0553438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:38:13.0553830Z attn_output, attn_weights = attention_interface( 2025-12-04T09:38:13.0554251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:38:13.0554712Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:13.0554897Z 2025-12-04T09:38:13.0554971Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0555174Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0555390Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0555738Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0556054Z res = mod(**inputs) 2025-12-04T09:38:13.0556390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:38:13.0556769Z outputs = self.model.decoder( 2025-12-04T09:38:13.0557132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:38:13.0557499Z layer_outputs = decoder_layer( 2025-12-04T09:38:13.0557825Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:13.0558169Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:13.0558530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:13.0558880Z return func(*args, **kwargs) 2025-12-04T09:38:13.0559230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:38:13.0559637Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:38:13.0560011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:13.0560333Z return self.act(input) 2025-12-04T09:38:13.0560445Z 2025-12-04T09:38:13.0560517Z cudagraph partition due to non gpu ops 2025-12-04T09:38:13.0560738Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0561081Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0561384Z res = mod(**inputs) 2025-12-04T09:38:13.0561727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1918, in forward 2025-12-04T09:38:13.0562103Z logits = self.lm_head(outputs[0]) 2025-12-04T09:38:13.0562224Z 2025-12-04T09:38:13.0562322Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:13.0562662Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:13.0562971Z res = mod(**inputs) 2025-12-04T09:38:13.0563341Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1924, in forward 2025-12-04T09:38:13.0563778Z loss = loss_fct(logits.view(-1, self.config.vocab_size), labels.view(-1)) 2025-12-04T09:38:13.0563975Z 2025-12-04T09:38:23.0479737Z Compilation time (from dynamo_timed): 17.142849242 2025-12-04T09:38:23.0794106Z pass 2025-12-04T09:38:23.0796878Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:38:23.0797666Z TIMING: _recursive_pre_grad_passes:0.02844 _recursive_joint_graph_passes:0.61352 _recursive_post_grad_passes:0.05289 async_compile.wait:0.77886 code_gen:9.57703 inductor_compile:11.19766 backend_compile:14.8982 gc:0.00039 entire_frame_compile:17.14285 total_wall_time:17.14285 2025-12-04T09:38:23.0798674Z STATS: call_* op count: 336 | FakeTensorMode.__torch_dispatch__:18857 | FakeTensor.__torch_dispatch__:3477 | ProxyTorchDispatchMode.__torch_dispatch__:2854 2025-12-04T09:38:23.0799543Z Dynamo produced 1 graphs covering 336 ops with 0 graph breaks (0 unique) 2025-12-04T09:38:25.5872668Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:38:25.5873632Z import pynvml # type: ignore[import] 2025-12-04T09:38:28.7773057Z 2025-12-04T09:38:29.7403742Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:38:29.7406548Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:38:29.7419518Z cpu eval BertForMaskedLM 2025-12-04T09:38:30.1734377Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:38:30.3873475Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:38:30.5931136Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:38:39.1511212Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1511564Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1511793Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1512002Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1512195Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1512396Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1512593Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1512790Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1512982Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1513350Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1513557Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1513754Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1513959Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1514204Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1514413Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1514628Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1514827Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1515016Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1515249Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1515483Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1515855Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1516174Z res = mod(**inputs) 2025-12-04T09:38:39.1516557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1516940Z outputs = self.bert( 2025-12-04T09:38:39.1517289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1517669Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1518413Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1518814Z layer_outputs = layer_module( 2025-12-04T09:38:39.1519170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1519535Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1519920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1520290Z return func(*args, **kwargs) 2025-12-04T09:38:39.1520658Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:38:39.1521044Z self_attention_outputs = self.attention( 2025-12-04T09:38:39.1521421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1522543Z return func(*args, **kwargs) 2025-12-04T09:38:39.1522913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:38:39.1523295Z self_outputs = self.self( 2025-12-04T09:38:39.1523657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1524017Z return func(*args, **kwargs) 2025-12-04T09:38:39.1524377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:38:39.1524814Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:39.1524998Z 2025-12-04T09:38:39.1525077Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1525279Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1525510Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1525868Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1526179Z res = mod(**inputs) 2025-12-04T09:38:39.1526534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1526908Z outputs = self.bert( 2025-12-04T09:38:39.1527258Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1527637Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1528007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1528381Z layer_outputs = layer_module( 2025-12-04T09:38:39.1528716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1529079Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1529456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1529853Z return func(*args, **kwargs) 2025-12-04T09:38:39.1530216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:38:39.1530597Z layer_output = apply_chunking_to_forward( 2025-12-04T09:38:39.1530998Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:38:39.1531386Z return forward_fn(*input_tensors) 2025-12-04T09:38:39.1531867Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:38:39.1532313Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:38:39.1532791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:38:39.1533214Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:38:39.1533602Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:39.1533942Z return self.act(input) 2025-12-04T09:38:39.1534059Z 2025-12-04T09:38:39.1534138Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1534343Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1534538Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1534739Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1534939Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1535136Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1535340Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1535551Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1535799Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1536145Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1536450Z res = mod(**inputs) 2025-12-04T09:38:39.1536796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1537168Z outputs = self.bert( 2025-12-04T09:38:39.1537514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1537881Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1538241Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1538597Z layer_outputs = layer_module( 2025-12-04T09:38:39.1538934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1539300Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1539677Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1540039Z return func(*args, **kwargs) 2025-12-04T09:38:39.1540405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:38:39.1540785Z self_attention_outputs = self.attention( 2025-12-04T09:38:39.1541153Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1541515Z return func(*args, **kwargs) 2025-12-04T09:38:39.1541872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:38:39.1542239Z self_outputs = self.self( 2025-12-04T09:38:39.1542593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1542968Z return func(*args, **kwargs) 2025-12-04T09:38:39.1543328Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:38:39.1543749Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:39.1543934Z 2025-12-04T09:38:39.1544010Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1544213Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1544434Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1544782Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1545092Z res = mod(**inputs) 2025-12-04T09:38:39.1545438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1545802Z outputs = self.bert( 2025-12-04T09:38:39.1546192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1546568Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1546932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1547304Z layer_outputs = layer_module( 2025-12-04T09:38:39.1547647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1548003Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1548368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1548733Z return func(*args, **kwargs) 2025-12-04T09:38:39.1549090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:38:39.1549520Z layer_output = apply_chunking_to_forward( 2025-12-04T09:38:39.1549911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:38:39.1550300Z return forward_fn(*input_tensors) 2025-12-04T09:38:39.1550694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:38:39.1551128Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:38:39.1551540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:38:39.1551945Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:38:39.1552316Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:39.1552646Z return self.act(input) 2025-12-04T09:38:39.1552763Z 2025-12-04T09:38:39.1552837Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1553042Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1553363Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1553562Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1553756Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1553953Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1554137Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1554330Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1554554Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1554895Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1555208Z res = mod(**inputs) 2025-12-04T09:38:39.1555556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1555928Z outputs = self.bert( 2025-12-04T09:38:39.1556275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1556644Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1557005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1557358Z layer_outputs = layer_module( 2025-12-04T09:38:39.1557693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1558042Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1558405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1558756Z return func(*args, **kwargs) 2025-12-04T09:38:39.1559113Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:38:39.1559496Z self_attention_outputs = self.attention( 2025-12-04T09:38:39.1559921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1560291Z return func(*args, **kwargs) 2025-12-04T09:38:39.1560652Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:38:39.1561022Z self_outputs = self.self( 2025-12-04T09:38:39.1561390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1561741Z return func(*args, **kwargs) 2025-12-04T09:38:39.1562093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:38:39.1562503Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:39.1562687Z 2025-12-04T09:38:39.1562809Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1563009Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1563234Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1563571Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1563882Z res = mod(**inputs) 2025-12-04T09:38:39.1564225Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1564585Z outputs = self.bert( 2025-12-04T09:38:39.1564930Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1565297Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1565697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1566051Z layer_outputs = layer_module( 2025-12-04T09:38:39.1566388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1566733Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1567097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1567444Z return func(*args, **kwargs) 2025-12-04T09:38:39.1567799Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:38:39.1568174Z layer_output = apply_chunking_to_forward( 2025-12-04T09:38:39.1568551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:38:39.1568931Z return forward_fn(*input_tensors) 2025-12-04T09:38:39.1569317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:38:39.1569756Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:38:39.1570153Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:38:39.1570550Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:38:39.1570913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:39.1571238Z return self.act(input) 2025-12-04T09:38:39.1571343Z 2025-12-04T09:38:39.1571417Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1571616Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1571807Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1571992Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1572183Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1572376Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1572564Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1572753Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1573009Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1573355Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1573677Z res = mod(**inputs) 2025-12-04T09:38:39.1574030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1574412Z outputs = self.bert( 2025-12-04T09:38:39.1574746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1575116Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1575484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1575895Z layer_outputs = layer_module( 2025-12-04T09:38:39.1576269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1576627Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1576985Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1577327Z return func(*args, **kwargs) 2025-12-04T09:38:39.1577686Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:38:39.1578065Z self_attention_outputs = self.attention( 2025-12-04T09:38:39.1578437Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1578784Z return func(*args, **kwargs) 2025-12-04T09:38:39.1579138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:38:39.1579506Z self_outputs = self.self( 2025-12-04T09:38:39.1579849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1580205Z return func(*args, **kwargs) 2025-12-04T09:38:39.1580562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:38:39.1580986Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:39.1581160Z 2025-12-04T09:38:39.1581235Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1581439Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1581665Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1582006Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1582318Z res = mod(**inputs) 2025-12-04T09:38:39.1582665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1583038Z outputs = self.bert( 2025-12-04T09:38:39.1583377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1583750Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1584116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1584486Z layer_outputs = layer_module( 2025-12-04T09:38:39.1584815Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1585165Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1585533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1585886Z return func(*args, **kwargs) 2025-12-04T09:38:39.1586271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:38:39.1586654Z layer_output = apply_chunking_to_forward( 2025-12-04T09:38:39.1587038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:38:39.1587415Z return forward_fn(*input_tensors) 2025-12-04T09:38:39.1587804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:38:39.1588244Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:38:39.1588647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:38:39.1589057Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:38:39.1589421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:39.1589796Z return self.act(input) 2025-12-04T09:38:39.1589905Z 2025-12-04T09:38:39.1589979Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1590183Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1590380Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1590571Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1590765Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1590962Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1591168Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1591360Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1591582Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1591940Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1592248Z res = mod(**inputs) 2025-12-04T09:38:39.1592594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1593007Z outputs = self.bert( 2025-12-04T09:38:39.1593505Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1593931Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1594346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1594804Z layer_outputs = layer_module( 2025-12-04T09:38:39.1595141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1595503Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1595879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1596252Z return func(*args, **kwargs) 2025-12-04T09:38:39.1596614Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:38:39.1596998Z self_attention_outputs = self.attention( 2025-12-04T09:38:39.1597374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1597729Z return func(*args, **kwargs) 2025-12-04T09:38:39.1598091Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:38:39.1598469Z self_outputs = self.self( 2025-12-04T09:38:39.1598819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1599175Z return func(*args, **kwargs) 2025-12-04T09:38:39.1599529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:38:39.1599959Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:39.1600177Z 2025-12-04T09:38:39.1600262Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1600455Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1600682Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1601030Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1601336Z res = mod(**inputs) 2025-12-04T09:38:39.1601686Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1602062Z outputs = self.bert( 2025-12-04T09:38:39.1602394Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1602757Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1603115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1603509Z layer_outputs = layer_module( 2025-12-04T09:38:39.1603822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1604159Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1604514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1604857Z return func(*args, **kwargs) 2025-12-04T09:38:39.1605191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:38:39.1605556Z layer_output = apply_chunking_to_forward( 2025-12-04T09:38:39.1606118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:38:39.1606497Z return forward_fn(*input_tensors) 2025-12-04T09:38:39.1606894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:38:39.1607333Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:38:39.1607738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:38:39.1608132Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:38:39.1608504Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:39.1608837Z return self.act(input) 2025-12-04T09:38:39.1608944Z 2025-12-04T09:38:39.1609029Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1609224Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1609424Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1609619Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1609807Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1610001Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1610193Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1610377Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1610596Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1610936Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1611237Z res = mod(**inputs) 2025-12-04T09:38:39.1611579Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1611940Z outputs = self.bert( 2025-12-04T09:38:39.1612284Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1612641Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1613000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1613477Z layer_outputs = layer_module( 2025-12-04T09:38:39.1613819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1614164Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1614528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1614888Z return func(*args, **kwargs) 2025-12-04T09:38:39.1615232Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:38:39.1615610Z self_attention_outputs = self.attention( 2025-12-04T09:38:39.1615976Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1616336Z return func(*args, **kwargs) 2025-12-04T09:38:39.1616734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:38:39.1617093Z self_outputs = self.self( 2025-12-04T09:38:39.1617442Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1617790Z return func(*args, **kwargs) 2025-12-04T09:38:39.1618138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:38:39.1618558Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:39.1618731Z 2025-12-04T09:38:39.1618812Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1619000Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1619222Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1619559Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1619859Z res = mod(**inputs) 2025-12-04T09:38:39.1620202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1620562Z outputs = self.bert( 2025-12-04T09:38:39.1620903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1621261Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1621617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1621981Z layer_outputs = layer_module( 2025-12-04T09:38:39.1622305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1622652Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1623019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1623381Z return func(*args, **kwargs) 2025-12-04T09:38:39.1623721Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:38:39.1624095Z layer_output = apply_chunking_to_forward( 2025-12-04T09:38:39.1624481Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:38:39.1624858Z return forward_fn(*input_tensors) 2025-12-04T09:38:39.1625232Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:38:39.1625665Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:38:39.1626071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:38:39.1626463Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:38:39.1626862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:39.1627196Z return self.act(input) 2025-12-04T09:38:39.1627301Z 2025-12-04T09:38:39.1627384Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1627573Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1627766Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1627959Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1628143Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1628335Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1628526Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1628708Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1628927Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1629268Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1629612Z res = mod(**inputs) 2025-12-04T09:38:39.1629953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1630316Z outputs = self.bert( 2025-12-04T09:38:39.1630656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1631016Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1631383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1631753Z layer_outputs = layer_module( 2025-12-04T09:38:39.1632097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1632450Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1632812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1633317Z return func(*args, **kwargs) 2025-12-04T09:38:39.1633707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:38:39.1634165Z self_attention_outputs = self.attention( 2025-12-04T09:38:39.1634552Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1634929Z return func(*args, **kwargs) 2025-12-04T09:38:39.1635345Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:38:39.1635721Z self_outputs = self.self( 2025-12-04T09:38:39.1636087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1636513Z return func(*args, **kwargs) 2025-12-04T09:38:39.1636881Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:38:39.1637322Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:39.1637503Z 2025-12-04T09:38:39.1637588Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1637788Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1638021Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1638381Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1638707Z res = mod(**inputs) 2025-12-04T09:38:39.1639060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1639440Z outputs = self.bert( 2025-12-04T09:38:39.1639798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1640180Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1640634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1641020Z layer_outputs = layer_module( 2025-12-04T09:38:39.1641372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1641730Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1642111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1642486Z return func(*args, **kwargs) 2025-12-04T09:38:39.1642852Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:38:39.1643237Z layer_output = apply_chunking_to_forward( 2025-12-04T09:38:39.1643650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:38:39.1644081Z return forward_fn(*input_tensors) 2025-12-04T09:38:39.1644477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:38:39.1644929Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:38:39.1645343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:38:39.1645755Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:38:39.1646120Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:39.1646490Z return self.act(input) 2025-12-04T09:38:39.1646592Z 2025-12-04T09:38:39.1646673Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1646860Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1647056Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1647249Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1647437Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1647621Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1647809Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1647997Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1648206Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1648540Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1648838Z res = mod(**inputs) 2025-12-04T09:38:39.1649160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1649514Z outputs = self.bert( 2025-12-04T09:38:39.1649847Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1650209Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1650556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1650911Z layer_outputs = layer_module( 2025-12-04T09:38:39.1651238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1651578Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1651923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1652271Z return func(*args, **kwargs) 2025-12-04T09:38:39.1652612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:38:39.1652976Z self_attention_outputs = self.attention( 2025-12-04T09:38:39.1653330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1653708Z return func(*args, **kwargs) 2025-12-04T09:38:39.1654058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:38:39.1654420Z self_outputs = self.self( 2025-12-04T09:38:39.1654762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1655115Z return func(*args, **kwargs) 2025-12-04T09:38:39.1655448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:38:39.1655860Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:39.1656038Z 2025-12-04T09:38:39.1656112Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1656307Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1656554Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1656900Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1657207Z res = mod(**inputs) 2025-12-04T09:38:39.1657549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1657911Z outputs = self.bert( 2025-12-04T09:38:39.1658254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1658621Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1658973Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1659342Z layer_outputs = layer_module( 2025-12-04T09:38:39.1659674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1660016Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1660381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1660738Z return func(*args, **kwargs) 2025-12-04T09:38:39.1661089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:38:39.1661458Z layer_output = apply_chunking_to_forward( 2025-12-04T09:38:39.1661850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:38:39.1662232Z return forward_fn(*input_tensors) 2025-12-04T09:38:39.1662619Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:38:39.1663048Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:38:39.1663461Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:38:39.1663860Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:38:39.1664217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:39.1664547Z return self.act(input) 2025-12-04T09:38:39.1664658Z 2025-12-04T09:38:39.1664731Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1664929Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1665117Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1665307Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1665497Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1665681Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1665872Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1666062Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1666280Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1666670Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1666981Z res = mod(**inputs) 2025-12-04T09:38:39.1667325Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1667680Z outputs = self.bert( 2025-12-04T09:38:39.1668024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1668393Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1668746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1669110Z layer_outputs = layer_module( 2025-12-04T09:38:39.1669443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1669830Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1670196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1670560Z return func(*args, **kwargs) 2025-12-04T09:38:39.1670912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:38:39.1671286Z self_attention_outputs = self.attention( 2025-12-04T09:38:39.1671640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1671995Z return func(*args, **kwargs) 2025-12-04T09:38:39.1672341Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:38:39.1672694Z self_outputs = self.self( 2025-12-04T09:38:39.1673040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1673533Z return func(*args, **kwargs) 2025-12-04T09:38:39.1673898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:38:39.1674351Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:39.1674540Z 2025-12-04T09:38:39.1674617Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1674819Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1675036Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1675380Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1675694Z res = mod(**inputs) 2025-12-04T09:38:39.1676050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1676420Z outputs = self.bert( 2025-12-04T09:38:39.1676775Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1677154Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1677512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1677885Z layer_outputs = layer_module( 2025-12-04T09:38:39.1678226Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1678581Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1678945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1679314Z return func(*args, **kwargs) 2025-12-04T09:38:39.1679671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:38:39.1680063Z layer_output = apply_chunking_to_forward( 2025-12-04T09:38:39.1680493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:38:39.1680874Z return forward_fn(*input_tensors) 2025-12-04T09:38:39.1681262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:38:39.1681691Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:38:39.1682099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:38:39.1682501Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:38:39.1682862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:39.1683181Z return self.act(input) 2025-12-04T09:38:39.1683327Z 2025-12-04T09:38:39.1683401Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1683605Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1683797Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1683995Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1684191Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1684387Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1684573Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1684765Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1684990Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1685332Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1685647Z res = mod(**inputs) 2025-12-04T09:38:39.1685991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1686359Z outputs = self.bert( 2025-12-04T09:38:39.1686712Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1687093Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1687460Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1687837Z layer_outputs = layer_module( 2025-12-04T09:38:39.1688167Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1688511Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1688871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1689219Z return func(*args, **kwargs) 2025-12-04T09:38:39.1689568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:38:39.1689944Z self_attention_outputs = self.attention( 2025-12-04T09:38:39.1690302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1690657Z return func(*args, **kwargs) 2025-12-04T09:38:39.1691009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:38:39.1691370Z self_outputs = self.self( 2025-12-04T09:38:39.1691703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1692053Z return func(*args, **kwargs) 2025-12-04T09:38:39.1692398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:38:39.1692806Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:39.1692987Z 2025-12-04T09:38:39.1693116Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1693344Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1693571Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1693914Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1694225Z res = mod(**inputs) 2025-12-04T09:38:39.1694567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1694937Z outputs = self.bert( 2025-12-04T09:38:39.1695290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1695657Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1696023Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1696395Z layer_outputs = layer_module( 2025-12-04T09:38:39.1696759Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1697096Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1697459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1697818Z return func(*args, **kwargs) 2025-12-04T09:38:39.1698163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:38:39.1698541Z layer_output = apply_chunking_to_forward( 2025-12-04T09:38:39.1698927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:38:39.1699305Z return forward_fn(*input_tensors) 2025-12-04T09:38:39.1699685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:38:39.1700130Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:38:39.1700539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:38:39.1700942Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:38:39.1701299Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:39.1701632Z return self.act(input) 2025-12-04T09:38:39.1701736Z 2025-12-04T09:38:39.1701817Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1702007Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1702201Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1702394Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1702590Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1702774Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1702968Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1703163Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1703377Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1703723Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1704033Z res = mod(**inputs) 2025-12-04T09:38:39.1704368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1704732Z outputs = self.bert( 2025-12-04T09:38:39.1705078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1705447Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1705944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1706319Z layer_outputs = layer_module( 2025-12-04T09:38:39.1706747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1707087Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1707447Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1707806Z return func(*args, **kwargs) 2025-12-04T09:38:39.1708158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:38:39.1708526Z self_attention_outputs = self.attention( 2025-12-04T09:38:39.1708893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1709252Z return func(*args, **kwargs) 2025-12-04T09:38:39.1709603Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:38:39.1710014Z self_outputs = self.self( 2025-12-04T09:38:39.1710360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1710713Z return func(*args, **kwargs) 2025-12-04T09:38:39.1711052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:38:39.1711466Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:39.1711647Z 2025-12-04T09:38:39.1711721Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1711918Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1712132Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1712475Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1712783Z res = mod(**inputs) 2025-12-04T09:38:39.1713219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1713612Z outputs = self.bert( 2025-12-04T09:38:39.1713978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1714343Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1714705Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1715078Z layer_outputs = layer_module( 2025-12-04T09:38:39.1715421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1715771Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1716148Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1716523Z return func(*args, **kwargs) 2025-12-04T09:38:39.1716893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:38:39.1717274Z layer_output = apply_chunking_to_forward( 2025-12-04T09:38:39.1717668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:38:39.1718053Z return forward_fn(*input_tensors) 2025-12-04T09:38:39.1718450Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:38:39.1718885Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:38:39.1719304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:38:39.1719716Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:38:39.1720082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:39.1720463Z return self.act(input) 2025-12-04T09:38:39.1720582Z 2025-12-04T09:38:39.1720656Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1720858Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1721047Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1721242Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1721438Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1721622Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1721817Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1722011Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1722228Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1722578Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1722892Z res = mod(**inputs) 2025-12-04T09:38:39.1723242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1723648Z outputs = self.bert( 2025-12-04T09:38:39.1723997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1724371Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1724731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1725100Z layer_outputs = layer_module( 2025-12-04T09:38:39.1725436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1725788Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1726151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1726513Z return func(*args, **kwargs) 2025-12-04T09:38:39.1726877Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:38:39.1727252Z self_attention_outputs = self.attention( 2025-12-04T09:38:39.1727627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1727979Z return func(*args, **kwargs) 2025-12-04T09:38:39.1728327Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:38:39.1728678Z self_outputs = self.self( 2025-12-04T09:38:39.1729032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1729384Z return func(*args, **kwargs) 2025-12-04T09:38:39.1729729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:38:39.1730136Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:38:39.1730315Z 2025-12-04T09:38:39.1730389Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1730613Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1730830Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1731177Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1731481Z res = mod(**inputs) 2025-12-04T09:38:39.1731820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:38:39.1732170Z outputs = self.bert( 2025-12-04T09:38:39.1732510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:38:39.1732875Z encoder_outputs = self.encoder( 2025-12-04T09:38:39.1733224Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:38:39.1733638Z layer_outputs = layer_module( 2025-12-04T09:38:39.1733974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:38:39.1734319Z return super().__call__(*args, **kwargs) 2025-12-04T09:38:39.1734678Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:38:39.1735034Z return func(*args, **kwargs) 2025-12-04T09:38:39.1735386Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:38:39.1735754Z layer_output = apply_chunking_to_forward( 2025-12-04T09:38:39.1736149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:38:39.1736575Z return forward_fn(*input_tensors) 2025-12-04T09:38:39.1736963Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:38:39.1737399Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:38:39.1737815Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:38:39.1738225Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:38:39.1738598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:38:39.1738929Z return self.act(input) 2025-12-04T09:38:39.1739041Z 2025-12-04T09:38:39.1739116Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1739318Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1739510Z cudagraph partition due to non gpu ops 2025-12-04T09:38:39.1739740Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:38:39.1740096Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:38:39.1740414Z res = mod(**inputs) 2025-12-04T09:38:39.1740758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1300, in forward 2025-12-04T09:38:39.1741252Z masked_lm_loss = loss_fct(prediction_scores.view(-1, self.config.vocab_size), labels.view(-1)) 2025-12-04T09:38:39.1741482Z 2025-12-04T09:38:48.7110024Z Compilation time (from dynamo_timed): 17.495120601 2025-12-04T09:38:48.7186678Z pass 2025-12-04T09:38:48.7189747Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:38:48.7190516Z TIMING: _recursive_pre_grad_passes:0.02859 _recursive_joint_graph_passes:0.62498 _recursive_post_grad_passes:0.05241 async_compile.wait:0.71333 code_gen:9.51253 inductor_compile:11.15109 backend_compile:14.82911 gc:0.00015 entire_frame_compile:17.49512 total_wall_time:17.49512 2025-12-04T09:38:48.7191442Z STATS: call_* op count: 289 | FakeTensorMode.__torch_dispatch__:18896 | FakeTensor.__torch_dispatch__:3504 | ProxyTorchDispatchMode.__torch_dispatch__:2792 2025-12-04T09:38:48.7191916Z Dynamo produced 1 graphs covering 289 ops with 0 graph breaks (0 unique) 2025-12-04T09:38:51.0860222Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:38:51.0861128Z import pynvml # type: ignore[import] 2025-12-04T09:38:54.1967878Z 2025-12-04T09:39:09.4648922Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:39:09.4649499Z loading model: 0it [00:15, ?it/s] 2025-12-04T09:39:09.4683325Z cpu eval BlenderbotForCausalLM 2025-12-04T09:39:09.4860277Z Compilation time (from dynamo_timed): 0 2025-12-04T09:39:09.4860836Z pass_due_to_skip 2025-12-04T09:39:09.4861540Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:39:09.4861996Z TIMING: total_wall_time:0 2025-12-04T09:39:09.4862708Z STATS: call_* op count: 0 2025-12-04T09:39:09.4863043Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2025-12-04T09:39:11.4401876Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:39:11.4403708Z import pynvml # type: ignore[import] 2025-12-04T09:39:14.6359879Z 2025-12-04T09:39:21.8707763Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:39:21.8708985Z loading model: 0it [00:07, ?it/s] 2025-12-04T09:39:21.8731106Z cpu eval DebertaV2ForMaskedLM 2025-12-04T09:39:21.8910899Z Compilation time (from dynamo_timed): 0 2025-12-04T09:39:21.8913003Z pass_due_to_skip 2025-12-04T09:39:21.8913602Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:39:21.8913940Z TIMING: total_wall_time:0 2025-12-04T09:39:21.8914129Z STATS: call_* op count: 0 2025-12-04T09:39:21.8914374Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2025-12-04T09:39:23.7494429Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:39:23.7495976Z import pynvml # type: ignore[import] 2025-12-04T09:39:26.9189804Z 2025-12-04T09:39:27.5158512Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:39:27.5162779Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:39:27.5173003Z cpu eval DistilBertForMaskedLM 2025-12-04T09:39:27.6623033Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:39:27.7151287Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:39:27.7672129Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:39:32.8794213Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8798592Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8802790Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8806866Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8810962Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8815658Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8818838Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8824176Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8828706Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8829041Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8829283Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8829482Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8829682Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8829920Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:32.8830286Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:32.8830612Z res = mod(**inputs) 2025-12-04T09:39:32.8831031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:32.8831449Z dlbrt_output = self.distilbert( 2025-12-04T09:39:32.8831851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:32.8832258Z return self.transformer( 2025-12-04T09:39:32.8833170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:32.8833649Z layer_outputs = layer_module( 2025-12-04T09:39:32.8834029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:32.8834458Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:32.8834879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 466, in forward 2025-12-04T09:39:32.8835287Z sa_output = self.attention( 2025-12-04T09:39:32.8835676Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 392, in forward 2025-12-04T09:39:32.8836181Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:32.8836361Z 2025-12-04T09:39:32.8836446Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8836746Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8836985Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:32.8837343Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:32.8837655Z res = mod(**inputs) 2025-12-04T09:39:32.8838070Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:32.8838519Z dlbrt_output = self.distilbert( 2025-12-04T09:39:32.8838911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:32.8839315Z return self.transformer( 2025-12-04T09:39:32.8839705Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:32.8840100Z layer_outputs = layer_module( 2025-12-04T09:39:32.8840446Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:32.8840801Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:32.8841203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 484, in forward 2025-12-04T09:39:32.8841635Z ffn_output = self.ffn(sa_output) # (bs, seq_length, dim) 2025-12-04T09:39:32.8842071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 418, in forward 2025-12-04T09:39:32.8842597Z return apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, input) 2025-12-04T09:39:32.8843104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:39:32.8843485Z return forward_fn(*input_tensors) 2025-12-04T09:39:32.8843889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 422, in ff_chunk 2025-12-04T09:39:32.8844304Z x = self.activation(x) 2025-12-04T09:39:32.8844621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:39:32.8844961Z return self.act(input) 2025-12-04T09:39:32.8845071Z 2025-12-04T09:39:32.8845148Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8845347Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8845539Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8845734Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8845925Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8846112Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8846307Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8846500Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8846716Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:32.8847219Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:32.8847535Z res = mod(**inputs) 2025-12-04T09:39:32.8847918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:32.8848326Z dlbrt_output = self.distilbert( 2025-12-04T09:39:32.8848721Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:32.8849120Z return self.transformer( 2025-12-04T09:39:32.8849512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:32.8849905Z layer_outputs = layer_module( 2025-12-04T09:39:32.8850231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:32.8850619Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:32.8851015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 466, in forward 2025-12-04T09:39:32.8851408Z sa_output = self.attention( 2025-12-04T09:39:32.8851779Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 392, in forward 2025-12-04T09:39:32.8852226Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:32.8852407Z 2025-12-04T09:39:32.8852481Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8852673Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8852886Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:32.8853225Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:32.8853530Z res = mod(**inputs) 2025-12-04T09:39:32.8853895Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:32.8854286Z dlbrt_output = self.distilbert( 2025-12-04T09:39:32.8854672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:32.8855058Z return self.transformer( 2025-12-04T09:39:32.8855425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:32.8855815Z layer_outputs = layer_module( 2025-12-04T09:39:32.8856147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:32.8856485Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:32.8856879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 484, in forward 2025-12-04T09:39:32.8857308Z ffn_output = self.ffn(sa_output) # (bs, seq_length, dim) 2025-12-04T09:39:32.8857734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 418, in forward 2025-12-04T09:39:32.8858240Z return apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, input) 2025-12-04T09:39:32.8858737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:39:32.8859119Z return forward_fn(*input_tensors) 2025-12-04T09:39:32.8859506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 422, in ff_chunk 2025-12-04T09:39:32.8859890Z x = self.activation(x) 2025-12-04T09:39:32.8860202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:39:32.8860532Z return self.act(input) 2025-12-04T09:39:32.8860635Z 2025-12-04T09:39:32.8860757Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8860958Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8861150Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8861339Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8861522Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8861709Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8861896Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8862076Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8862293Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:32.8862632Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:32.8862931Z res = mod(**inputs) 2025-12-04T09:39:32.8863297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:32.8863718Z dlbrt_output = self.distilbert( 2025-12-04T09:39:32.8864110Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:32.8864489Z return self.transformer( 2025-12-04T09:39:32.8864866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:32.8865256Z layer_outputs = layer_module( 2025-12-04T09:39:32.8865591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:32.8865928Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:32.8866326Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 466, in forward 2025-12-04T09:39:32.8866714Z sa_output = self.attention( 2025-12-04T09:39:32.8867090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 392, in forward 2025-12-04T09:39:32.8867565Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:32.8867743Z 2025-12-04T09:39:32.8867818Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8868016Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8868235Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:32.8868586Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:32.8868892Z res = mod(**inputs) 2025-12-04T09:39:32.8869253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:32.8869644Z dlbrt_output = self.distilbert( 2025-12-04T09:39:32.8870027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:32.8870427Z return self.transformer( 2025-12-04T09:39:32.8870837Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:32.8871228Z layer_outputs = layer_module( 2025-12-04T09:39:32.8871560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:32.8871894Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:32.8872289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 484, in forward 2025-12-04T09:39:32.8872714Z ffn_output = self.ffn(sa_output) # (bs, seq_length, dim) 2025-12-04T09:39:32.8873237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 418, in forward 2025-12-04T09:39:32.8873779Z return apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, input) 2025-12-04T09:39:32.8874347Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:39:32.8874749Z return forward_fn(*input_tensors) 2025-12-04T09:39:32.8875145Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 422, in ff_chunk 2025-12-04T09:39:32.8875552Z x = self.activation(x) 2025-12-04T09:39:32.8875882Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:39:32.8876224Z return self.act(input) 2025-12-04T09:39:32.8876331Z 2025-12-04T09:39:32.8876414Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8876614Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8876816Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8877017Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8877244Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8877449Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8877649Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8877841Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8878072Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:32.8878432Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:32.8878759Z res = mod(**inputs) 2025-12-04T09:39:32.8879140Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:32.8879552Z dlbrt_output = self.distilbert( 2025-12-04T09:39:32.8879957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:32.8880363Z return self.transformer( 2025-12-04T09:39:32.8880762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:32.8881174Z layer_outputs = layer_module( 2025-12-04T09:39:32.8881525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:32.8881886Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:32.8882299Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 466, in forward 2025-12-04T09:39:32.8882711Z sa_output = self.attention( 2025-12-04T09:39:32.8883108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 392, in forward 2025-12-04T09:39:32.8883554Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:32.8883732Z 2025-12-04T09:39:32.8883806Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8884005Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8884220Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:32.8884559Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:32.8884866Z res = mod(**inputs) 2025-12-04T09:39:32.8885227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:32.8885640Z dlbrt_output = self.distilbert( 2025-12-04T09:39:32.8886030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:32.8886419Z return self.transformer( 2025-12-04T09:39:32.8886787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:32.8887177Z layer_outputs = layer_module( 2025-12-04T09:39:32.8887514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:32.8887883Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:32.8888275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 484, in forward 2025-12-04T09:39:32.8888707Z ffn_output = self.ffn(sa_output) # (bs, seq_length, dim) 2025-12-04T09:39:32.8889126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 418, in forward 2025-12-04T09:39:32.8889632Z return apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, input) 2025-12-04T09:39:32.8890127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:39:32.8890502Z return forward_fn(*input_tensors) 2025-12-04T09:39:32.8890896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 422, in ff_chunk 2025-12-04T09:39:32.8891315Z x = self.activation(x) 2025-12-04T09:39:32.8891630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:39:32.8891952Z return self.act(input) 2025-12-04T09:39:32.8892055Z 2025-12-04T09:39:32.8892133Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8892325Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8892514Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8892703Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8892889Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8893079Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8893268Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8893451Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8893672Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:32.8894019Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:32.8894324Z res = mod(**inputs) 2025-12-04T09:39:32.8894683Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:32.8895075Z dlbrt_output = self.distilbert( 2025-12-04T09:39:32.8895460Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:32.8895846Z return self.transformer( 2025-12-04T09:39:32.8896221Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:32.8896612Z layer_outputs = layer_module( 2025-12-04T09:39:32.8896944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:32.8897284Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:32.8897683Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 466, in forward 2025-12-04T09:39:32.8898076Z sa_output = self.attention( 2025-12-04T09:39:32.8898450Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 392, in forward 2025-12-04T09:39:32.8898898Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:32.8899077Z 2025-12-04T09:39:32.8899150Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8899348Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8899560Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:32.8899892Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:32.8900201Z res = mod(**inputs) 2025-12-04T09:39:32.8900608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:32.8900995Z dlbrt_output = self.distilbert( 2025-12-04T09:39:32.8901381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:32.8901783Z return self.transformer( 2025-12-04T09:39:32.8902152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:32.8902541Z layer_outputs = layer_module( 2025-12-04T09:39:32.8902874Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:32.8903221Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:32.8903608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 484, in forward 2025-12-04T09:39:32.8904069Z ffn_output = self.ffn(sa_output) # (bs, seq_length, dim) 2025-12-04T09:39:32.8904493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 418, in forward 2025-12-04T09:39:32.8905003Z return apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, input) 2025-12-04T09:39:32.8905488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:39:32.8906102Z return forward_fn(*input_tensors) 2025-12-04T09:39:32.8906495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 422, in ff_chunk 2025-12-04T09:39:32.8906874Z x = self.activation(x) 2025-12-04T09:39:32.8907184Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:39:32.8907512Z return self.act(input) 2025-12-04T09:39:32.8907616Z 2025-12-04T09:39:32.8907696Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8907888Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8908084Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8908273Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8908457Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8908648Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8908838Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8909019Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8909235Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:32.8909570Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:32.8909869Z res = mod(**inputs) 2025-12-04T09:39:32.8910223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:32.8910608Z dlbrt_output = self.distilbert( 2025-12-04T09:39:32.8910991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:32.8911368Z return self.transformer( 2025-12-04T09:39:32.8911740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:32.8912123Z layer_outputs = layer_module( 2025-12-04T09:39:32.8912452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:32.8912782Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:32.8913269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 466, in forward 2025-12-04T09:39:32.8913704Z sa_output = self.attention( 2025-12-04T09:39:32.8914120Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 392, in forward 2025-12-04T09:39:32.8914713Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:32.8914906Z 2025-12-04T09:39:32.8914985Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8915198Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8915427Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:32.8915789Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:32.8916169Z res = mod(**inputs) 2025-12-04T09:39:32.8916567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:32.8916977Z dlbrt_output = self.distilbert( 2025-12-04T09:39:32.8917393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:32.8917863Z return self.transformer( 2025-12-04T09:39:32.8918266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:32.8918690Z layer_outputs = layer_module( 2025-12-04T09:39:32.8919044Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:32.8919421Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:32.8919840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 484, in forward 2025-12-04T09:39:32.8920317Z ffn_output = self.ffn(sa_output) # (bs, seq_length, dim) 2025-12-04T09:39:32.8920780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 418, in forward 2025-12-04T09:39:32.8921337Z return apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, input) 2025-12-04T09:39:32.8921867Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:39:32.8922274Z return forward_fn(*input_tensors) 2025-12-04T09:39:32.8922695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 422, in ff_chunk 2025-12-04T09:39:32.8923124Z x = self.activation(x) 2025-12-04T09:39:32.8923427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:39:32.8923750Z return self.act(input) 2025-12-04T09:39:32.8923849Z 2025-12-04T09:39:32.8923930Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8924119Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8924317Z cudagraph partition due to non gpu ops 2025-12-04T09:39:32.8924537Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:32.8924869Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:32.8925171Z res = mod(**inputs) 2025-12-04T09:39:32.8925535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 831, in forward 2025-12-04T09:39:32.8926056Z mlm_loss = self.mlm_loss_fct(prediction_logits.view(-1, prediction_logits.size(-1)), labels.view(-1)) 2025-12-04T09:39:32.8926287Z 2025-12-04T09:39:41.4877040Z Compilation time (from dynamo_timed): 13.260166345 2025-12-04T09:39:41.4882271Z pass 2025-12-04T09:39:41.4882848Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:39:41.4883867Z TIMING: _recursive_pre_grad_passes:0.01619 _recursive_joint_graph_passes:0.24134 _recursive_post_grad_passes:0.03682 async_compile.wait:0.73274 code_gen:8.46321 inductor_compile:9.90428 backend_compile:11.69195 gc:0.00012 entire_frame_compile:13.26017 total_wall_time:13.26017 2025-12-04T09:39:41.4886454Z STATS: call_* op count: 153 | FakeTensorMode.__torch_dispatch__:10141 | FakeTensor.__torch_dispatch__:1896 | ProxyTorchDispatchMode.__torch_dispatch__:1495 2025-12-04T09:39:41.4886954Z Dynamo produced 1 graphs covering 153 ops with 0 graph breaks (0 unique) 2025-12-04T09:39:43.7543995Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:39:43.7544759Z import pynvml # type: ignore[import] 2025-12-04T09:39:46.9333294Z 2025-12-04T09:39:47.8448404Z loading model: 0it [00:00, ?it/s]`loss_type=None` was set in the config but it is unrecognized. Using the default loss: `ForCausalLMLoss`. 2025-12-04T09:39:47.8453333Z WARNING:transformers.modeling_utils:`loss_type=None` was set in the config but it is unrecognized. Using the default loss: `ForCausalLMLoss`. 2025-12-04T09:39:47.8743651Z 2025-12-04T09:39:47.8748036Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:39:47.8748366Z cpu eval DistillGPT2 2025-12-04T09:39:48.3207179Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:39:48.5206115Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:39:48.7018790Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:39:55.3699146Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:55.3699982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:55.3700536Z transformer_outputs = self.transformer( 2025-12-04T09:39:55.3701609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:39:55.3702307Z causal_mask = create_causal_mask( 2025-12-04T09:39:55.3702749Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:39:55.3703386Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:39:55.3704173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:39:55.3704667Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:39:55.3705197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 655, in find_packed_sequence_indices 2025-12-04T09:39:55.3705904Z first_dummy_value = position_ids[:, :1] - 1 # We just need the diff on this first value to be 1 2025-12-04T09:39:55.3710396Z 2025-12-04T09:39:55.3710838Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3711245Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3711544Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:55.3712020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:55.3712456Z transformer_outputs = self.transformer( 2025-12-04T09:39:55.3712919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:39:55.3713438Z causal_mask = create_causal_mask( 2025-12-04T09:39:55.3713823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:39:55.3714332Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:39:55.3714866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:39:55.3715651Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:39:55.3716107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 656, in find_packed_sequence_indices 2025-12-04T09:39:55.3716590Z position_diff = torch.diff(position_ids, prepend=first_dummy_value, dim=-1) 2025-12-04T09:39:55.3716798Z 2025-12-04T09:39:55.3716887Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3717120Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:55.3717543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:55.3717945Z transformer_outputs = self.transformer( 2025-12-04T09:39:55.3718342Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:39:55.3718859Z causal_mask = create_causal_mask( 2025-12-04T09:39:55.3719227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:39:55.3719718Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:39:55.3720222Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:39:55.3720655Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:39:55.3721108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 657, in find_packed_sequence_indices 2025-12-04T09:39:55.3721536Z packed_sequence_mask = (position_diff != 1).cumsum(-1) 2025-12-04T09:39:55.3721696Z 2025-12-04T09:39:55.3721807Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:55.3722211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:55.3722607Z transformer_outputs = self.transformer( 2025-12-04T09:39:55.3722993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:39:55.3723372Z causal_mask = create_causal_mask( 2025-12-04T09:39:55.3723791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:39:55.3724326Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:39:55.3724831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:39:55.3725318Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:39:55.3725762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 657, in find_packed_sequence_indices 2025-12-04T09:39:55.3726189Z packed_sequence_mask = (position_diff != 1).cumsum(-1) 2025-12-04T09:39:55.3726346Z 2025-12-04T09:39:55.3726457Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:55.3726857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:55.3727247Z transformer_outputs = self.transformer( 2025-12-04T09:39:55.3727625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:55.3727990Z outputs = block( 2025-12-04T09:39:55.3728308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:55.3728670Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:55.3729049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:55.3729447Z return func(*args, **kwargs) 2025-12-04T09:39:55.3729808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:39:55.3730194Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:39:55.3730568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:55.3730924Z return func(*args, **kwargs) 2025-12-04T09:39:55.3731276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:39:55.3731667Z attn_output, attn_weights = attention_interface( 2025-12-04T09:39:55.3732095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:39:55.3732590Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:55.3732772Z 2025-12-04T09:39:55.3732854Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3733061Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3733255Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3733455Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3733681Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:55.3734088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:55.3734466Z transformer_outputs = self.transformer( 2025-12-04T09:39:55.3734845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:55.3735208Z outputs = block( 2025-12-04T09:39:55.3735520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:55.3735877Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:55.3736252Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:55.3736618Z return func(*args, **kwargs) 2025-12-04T09:39:55.3736971Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:39:55.3737356Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:39:55.3737761Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:55.3738126Z return func(*args, **kwargs) 2025-12-04T09:39:55.3738479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:39:55.3738876Z attn_output, attn_weights = attention_interface( 2025-12-04T09:39:55.3739312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:39:55.3739781Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:55.3739956Z 2025-12-04T09:39:55.3740033Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3740241Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3740469Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:55.3740868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:55.3741251Z transformer_outputs = self.transformer( 2025-12-04T09:39:55.3741626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:55.3741990Z outputs = block( 2025-12-04T09:39:55.3742301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:55.3742658Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:55.3743063Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:55.3743429Z return func(*args, **kwargs) 2025-12-04T09:39:55.3743775Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:39:55.3744169Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:39:55.3744558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:39:55.3744916Z hidden_states = self.act(hidden_states) 2025-12-04T09:39:55.3745258Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:39:55.3745704Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:39:55.3745962Z 2025-12-04T09:39:55.3746046Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3746238Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3746431Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3746621Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3746804Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3746997Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3747217Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:55.3747615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:55.3747997Z transformer_outputs = self.transformer( 2025-12-04T09:39:55.3748368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:55.3748727Z outputs = block( 2025-12-04T09:39:55.3749040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:55.3749399Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:55.3749766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:55.3750122Z return func(*args, **kwargs) 2025-12-04T09:39:55.3750468Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:39:55.3750850Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:39:55.3751232Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:55.3751595Z return func(*args, **kwargs) 2025-12-04T09:39:55.3751954Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:39:55.3752357Z attn_output, attn_weights = attention_interface( 2025-12-04T09:39:55.3752801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:39:55.3753369Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:55.3753567Z 2025-12-04T09:39:55.3753647Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3753860Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3754107Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:55.3754525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:55.3754918Z transformer_outputs = self.transformer( 2025-12-04T09:39:55.3755304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:55.3755663Z outputs = block( 2025-12-04T09:39:55.3756058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:55.3756425Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:55.3756799Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:55.3757159Z return func(*args, **kwargs) 2025-12-04T09:39:55.3757525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:39:55.3757933Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:39:55.3758329Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:39:55.3758711Z hidden_states = self.act(hidden_states) 2025-12-04T09:39:55.3759061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:39:55.3759558Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:39:55.3759787Z 2025-12-04T09:39:55.3759866Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3760073Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3760277Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3760474Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3760662Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3760857Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3761083Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:55.3761489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:55.3761881Z transformer_outputs = self.transformer( 2025-12-04T09:39:55.3762257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:55.3762620Z outputs = block( 2025-12-04T09:39:55.3762939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:55.3763296Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:55.3763669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:55.3764029Z return func(*args, **kwargs) 2025-12-04T09:39:55.3764394Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:39:55.3764784Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:39:55.3765163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:55.3765572Z return func(*args, **kwargs) 2025-12-04T09:39:55.3765922Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:39:55.3766313Z attn_output, attn_weights = attention_interface( 2025-12-04T09:39:55.3766738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:39:55.3767196Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:55.3767375Z 2025-12-04T09:39:55.3767449Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3767648Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3767862Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:55.3768261Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:55.3768641Z transformer_outputs = self.transformer( 2025-12-04T09:39:55.3769004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:55.3769371Z outputs = block( 2025-12-04T09:39:55.3769715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:55.3770066Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:55.3770422Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:55.3770779Z return func(*args, **kwargs) 2025-12-04T09:39:55.3771138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:39:55.3771530Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:39:55.3771910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:39:55.3772281Z hidden_states = self.act(hidden_states) 2025-12-04T09:39:55.3772658Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:39:55.3773091Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:39:55.3773324Z 2025-12-04T09:39:55.3773398Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3773599Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3773793Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3773979Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3774170Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3774363Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3774575Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:55.3774975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:55.3775357Z transformer_outputs = self.transformer( 2025-12-04T09:39:55.3775737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:55.3776092Z outputs = block( 2025-12-04T09:39:55.3776405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:55.3776756Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:55.3777112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:55.3777470Z return func(*args, **kwargs) 2025-12-04T09:39:55.3777825Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:39:55.3778205Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:39:55.3778570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:55.3778931Z return func(*args, **kwargs) 2025-12-04T09:39:55.3779288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:39:55.3779672Z attn_output, attn_weights = attention_interface( 2025-12-04T09:39:55.3780099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:39:55.3780558Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:55.3780732Z 2025-12-04T09:39:55.3780813Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3781005Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3781227Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:55.3781628Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:55.3782009Z transformer_outputs = self.transformer( 2025-12-04T09:39:55.3782411Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:55.3782771Z outputs = block( 2025-12-04T09:39:55.3783082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:55.3783424Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:55.3783785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:55.3784145Z return func(*args, **kwargs) 2025-12-04T09:39:55.3784500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:39:55.3784886Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:39:55.3785278Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:39:55.3785696Z hidden_states = self.act(hidden_states) 2025-12-04T09:39:55.3786031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:39:55.3786465Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:39:55.3786692Z 2025-12-04T09:39:55.3786767Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3786964Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3787150Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3787343Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3787537Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3787719Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3787942Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:55.3788341Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:55.3788724Z transformer_outputs = self.transformer( 2025-12-04T09:39:55.3789093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:55.3789452Z outputs = block( 2025-12-04T09:39:55.3789765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:55.3790105Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:55.3790465Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:55.3790822Z return func(*args, **kwargs) 2025-12-04T09:39:55.3791176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:39:55.3791547Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:39:55.3791919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:55.3792282Z return func(*args, **kwargs) 2025-12-04T09:39:55.3792634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:39:55.3793053Z attn_output, attn_weights = attention_interface( 2025-12-04T09:39:55.3793578Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:39:55.3794056Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:55.3794236Z 2025-12-04T09:39:55.3794310Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3794521Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3794744Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:55.3795143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:55.3795520Z transformer_outputs = self.transformer( 2025-12-04T09:39:55.3795960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:55.3796332Z outputs = block( 2025-12-04T09:39:55.3796642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:55.3797002Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:55.3797377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:55.3797745Z return func(*args, **kwargs) 2025-12-04T09:39:55.3798090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:39:55.3798481Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:39:55.3798919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:39:55.3799291Z hidden_states = self.act(hidden_states) 2025-12-04T09:39:55.3799663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:39:55.3800109Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:39:55.3800338Z 2025-12-04T09:39:55.3800421Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3800617Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3800822Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3801023Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3801222Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3801419Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3801647Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:55.3802062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:55.3802444Z transformer_outputs = self.transformer( 2025-12-04T09:39:55.3802827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:55.3803198Z outputs = block( 2025-12-04T09:39:55.3803517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:55.3803869Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:55.3804241Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:55.3804612Z return func(*args, **kwargs) 2025-12-04T09:39:55.3804967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:39:55.3805364Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:39:55.3805936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:55.3806314Z return func(*args, **kwargs) 2025-12-04T09:39:55.3806669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:39:55.3807067Z attn_output, attn_weights = attention_interface( 2025-12-04T09:39:55.3807507Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:39:55.3807976Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:55.3808156Z 2025-12-04T09:39:55.3808233Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3808438Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3808664Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:55.3810255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:55.3810649Z transformer_outputs = self.transformer( 2025-12-04T09:39:55.3811024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:55.3811385Z outputs = block( 2025-12-04T09:39:55.3811689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:55.3812039Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:55.3812402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:55.3812756Z return func(*args, **kwargs) 2025-12-04T09:39:55.3813112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:39:55.3813561Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:39:55.3813950Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:39:55.3814312Z hidden_states = self.act(hidden_states) 2025-12-04T09:39:55.3814647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:39:55.3815082Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:39:55.3815302Z 2025-12-04T09:39:55.3815384Z cudagraph partition due to non gpu ops 2025-12-04T09:39:55.3815600Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:55.3816000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1092, in forward 2025-12-04T09:39:55.3816408Z logits = self.lm_head(hidden_states[:, slice_indices, :]) 2025-12-04T09:39:55.3816569Z 2025-12-04T09:40:04.4045084Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:04.4049502Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 67, in ForCausalLMLoss 2025-12-04T09:40:04.4054828Z loss = fixed_cross_entropy(logits, shift_labels, num_items_in_batch, ignore_index, **kwargs) 2025-12-04T09:40:04.4060001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 36, in fixed_cross_entropy 2025-12-04T09:40:04.4061901Z loss = nn.functional.cross_entropy(source, target, ignore_index=ignore_index, reduction=reduction) 2025-12-04T09:40:04.4062267Z 2025-12-04T09:40:05.4244950Z Compilation time (from dynamo_timed): 16.096894669 2025-12-04T09:40:05.4422689Z pass 2025-12-04T09:40:05.4429386Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:40:05.4432577Z TIMING: gc:0.00315 entire_frame_compile:16.09689 _recursive_pre_grad_passes:0.02339 _recursive_joint_graph_passes:0.22006 _recursive_post_grad_passes:0.04077 async_compile.wait:1.37412 code_gen:9.5803 inductor_compile:10.76095 backend_compile:12.63577 total_wall_time:16.09689 2025-12-04T09:40:05.4433849Z STATS: call_* op count: 311 | FakeTensorMode.__torch_dispatch__:9916 | FakeTensor.__torch_dispatch__:1939 | ProxyTorchDispatchMode.__torch_dispatch__:980 2025-12-04T09:40:05.4434347Z Dynamo produced 2 graphs covering 311 ops with 2 graph breaks (1 unique) 2025-12-04T09:40:07.9173672Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:40:07.9174502Z import pynvml # type: ignore[import] 2025-12-04T09:40:11.0886981Z 2025-12-04T09:40:11.0897355Z loading model: 0it [00:00, ?it/s]If you want to use `ElectraForCausalLM` as a standalone, add `is_decoder=True.` 2025-12-04T09:40:11.0898808Z WARNING:transformers.models.electra.modeling_electra:If you want to use `ElectraForCausalLM` as a standalone, add `is_decoder=True.` 2025-12-04T09:40:11.3186464Z 2025-12-04T09:40:11.3186948Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:40:11.3200768Z cpu eval ElectraForCausalLM 2025-12-04T09:40:11.4892138Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:40:11.5639909Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:40:11.6378702Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:40:20.6746675Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6747055Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:20.6747471Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:20.6748303Z res = mod(**inputs) 2025-12-04T09:40:20.6748731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:20.6749188Z outputs = self.electra( 2025-12-04T09:40:20.6749695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 787, in forward 2025-12-04T09:40:20.6750175Z hidden_states = self.embeddings_project(hidden_states) 2025-12-04T09:40:20.6750354Z 2025-12-04T09:40:20.6750437Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6750651Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6750856Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6751063Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6751278Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6751497Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6751695Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6751907Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6752127Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6752322Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6752573Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:20.6752980Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:20.6753450Z res = mod(**inputs) 2025-12-04T09:40:20.6753839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:20.6754254Z outputs = self.electra( 2025-12-04T09:40:20.6754706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:20.6755112Z hidden_states = self.encoder( 2025-12-04T09:40:20.6755521Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:20.6755999Z layer_outputs = layer_module( 2025-12-04T09:40:20.6756373Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:20.6756776Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:20.6757177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:20.6757578Z return func(*args, **kwargs) 2025-12-04T09:40:20.6757984Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:20.6758414Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:20.6758813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:20.6759227Z return forward_fn(*input_tensors) 2025-12-04T09:40:20.6759780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:20.6760272Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:20.6760721Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:20.6761169Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:20.6761565Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:20.6761912Z return self.act(input) 2025-12-04T09:40:20.6762020Z 2025-12-04T09:40:20.6762094Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6762293Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6762489Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6762678Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6762919Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6763115Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6763301Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6763500Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6763695Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6763887Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6764076Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6764302Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:20.6764666Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:20.6764967Z res = mod(**inputs) 2025-12-04T09:40:20.6765327Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:20.6765706Z outputs = self.electra( 2025-12-04T09:40:20.6766071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:20.6766473Z hidden_states = self.encoder( 2025-12-04T09:40:20.6766841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:20.6767210Z layer_outputs = layer_module( 2025-12-04T09:40:20.6767537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:20.6767900Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:20.6768274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:20.6768644Z return func(*args, **kwargs) 2025-12-04T09:40:20.6769009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:20.6769412Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:20.6769807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:20.6770185Z return forward_fn(*input_tensors) 2025-12-04T09:40:20.6770597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:20.6771059Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:20.6771486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:20.6771901Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:20.6772281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:20.6772605Z return self.act(input) 2025-12-04T09:40:20.6772712Z 2025-12-04T09:40:20.6772796Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6772986Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6773224Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6773415Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6773599Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6773787Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6773974Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6774152Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6774340Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6774526Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6774705Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6774922Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:20.6775257Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:20.6775557Z res = mod(**inputs) 2025-12-04T09:40:20.6775937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:20.6776311Z outputs = self.electra( 2025-12-04T09:40:20.6776665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:20.6777027Z hidden_states = self.encoder( 2025-12-04T09:40:20.6777390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:20.6777755Z layer_outputs = layer_module( 2025-12-04T09:40:20.6778085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:20.6778417Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:20.6778777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:20.6779131Z return func(*args, **kwargs) 2025-12-04T09:40:20.6779493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:20.6779867Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:20.6780239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:20.6780643Z return forward_fn(*input_tensors) 2025-12-04T09:40:20.6781033Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:20.6781489Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:20.6781908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:20.6782320Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:20.6782685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:20.6783017Z return self.act(input) 2025-12-04T09:40:20.6783122Z 2025-12-04T09:40:20.6783203Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6783395Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6783591Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6783784Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6783967Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6784161Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6784355Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6784548Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6784735Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6784927Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6785116Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6785330Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:20.6785716Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:20.6786037Z res = mod(**inputs) 2025-12-04T09:40:20.6786395Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:20.6786776Z outputs = self.electra( 2025-12-04T09:40:20.6787144Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:20.6787526Z hidden_states = self.encoder( 2025-12-04T09:40:20.6787898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:20.6788283Z layer_outputs = layer_module( 2025-12-04T09:40:20.6788620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:20.6789013Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:20.6789382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:20.6789753Z return func(*args, **kwargs) 2025-12-04T09:40:20.6790124Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:20.6790518Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:20.6790912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:20.6791297Z return forward_fn(*input_tensors) 2025-12-04T09:40:20.6791710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:20.6792162Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:20.6792595Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:20.6793018Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:20.6793493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:20.6793825Z return self.act(input) 2025-12-04T09:40:20.6793940Z 2025-12-04T09:40:20.6794015Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6794217Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6794410Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6794609Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6794809Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6795000Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6795199Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6795400Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6795597Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6795788Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6796130Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6796414Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:20.6797039Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:20.6797468Z res = mod(**inputs) 2025-12-04T09:40:20.6797893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:20.6798395Z outputs = self.electra( 2025-12-04T09:40:20.6798798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:20.6799237Z hidden_states = self.encoder( 2025-12-04T09:40:20.6799708Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:20.6800130Z layer_outputs = layer_module( 2025-12-04T09:40:20.6800568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:20.6801024Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:20.6801448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:20.6801828Z return func(*args, **kwargs) 2025-12-04T09:40:20.6802288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:20.6802740Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:20.6803196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:20.6803612Z return forward_fn(*input_tensors) 2025-12-04T09:40:20.6804117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:20.6804623Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:20.6818362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:20.6818828Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:20.6819206Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:20.6819541Z return self.act(input) 2025-12-04T09:40:20.6819661Z 2025-12-04T09:40:20.6819744Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6819953Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6820141Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6820339Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6820533Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6820741Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6820932Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6821127Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6821323Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6821511Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6821708Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6821937Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:20.6822282Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:20.6822602Z res = mod(**inputs) 2025-12-04T09:40:20.6822970Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:20.6823358Z outputs = self.electra( 2025-12-04T09:40:20.6823721Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:20.6824105Z hidden_states = self.encoder( 2025-12-04T09:40:20.6824484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:20.6824854Z layer_outputs = layer_module( 2025-12-04T09:40:20.6825196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:20.6825550Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:20.6825920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:20.6826276Z return func(*args, **kwargs) 2025-12-04T09:40:20.6826642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:20.6827032Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:20.6827560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:20.6827943Z return forward_fn(*input_tensors) 2025-12-04T09:40:20.6828355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:20.6828813Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:20.6829228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:20.6829642Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:20.6830011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:20.6830345Z return self.act(input) 2025-12-04T09:40:20.6830451Z 2025-12-04T09:40:20.6830527Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6830812Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6831009Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6831197Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6831395Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6831587Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6831773Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6831972Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6832169Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6832364Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6832550Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6832774Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:20.6833219Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:20.6833549Z res = mod(**inputs) 2025-12-04T09:40:20.6833934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:20.6834347Z outputs = self.electra( 2025-12-04T09:40:20.6834730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:20.6835101Z hidden_states = self.encoder( 2025-12-04T09:40:20.6835489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:20.6835889Z layer_outputs = layer_module( 2025-12-04T09:40:20.6836236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:20.6836611Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:20.6836992Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:20.6837374Z return func(*args, **kwargs) 2025-12-04T09:40:20.6837754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:20.6838164Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:20.6838566Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:20.6838963Z return forward_fn(*input_tensors) 2025-12-04T09:40:20.6839374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:20.6839841Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:20.6840277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:20.6840700Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:20.6841081Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:20.6841428Z return self.act(input) 2025-12-04T09:40:20.6841568Z 2025-12-04T09:40:20.6841655Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6841851Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6842053Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6842254Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6842497Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6842700Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6842895Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6843096Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6843296Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6843489Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6843689Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6843921Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:20.6844308Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:20.6844612Z res = mod(**inputs) 2025-12-04T09:40:20.6844967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:20.6845343Z outputs = self.electra( 2025-12-04T09:40:20.6845692Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:20.6846066Z hidden_states = self.encoder( 2025-12-04T09:40:20.6846442Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:20.6846814Z layer_outputs = layer_module( 2025-12-04T09:40:20.6847134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:20.6847476Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:20.6847840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:20.6848190Z return func(*args, **kwargs) 2025-12-04T09:40:20.6848550Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:20.6848937Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:20.6849315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:20.6849680Z return forward_fn(*input_tensors) 2025-12-04T09:40:20.6850094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:20.6850541Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:20.6850953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:20.6851357Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:20.6851717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:20.6852038Z return self.act(input) 2025-12-04T09:40:20.6852143Z 2025-12-04T09:40:20.6852218Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6852415Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6852607Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6852796Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6852980Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6853171Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6853356Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6853537Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6853724Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6853918Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6854100Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6854354Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:20.6854696Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:20.6855002Z res = mod(**inputs) 2025-12-04T09:40:20.6855359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:20.6855735Z outputs = self.electra( 2025-12-04T09:40:20.6856093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:20.6856459Z hidden_states = self.encoder( 2025-12-04T09:40:20.6856830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:20.6857237Z layer_outputs = layer_module( 2025-12-04T09:40:20.6857570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:20.6857915Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:20.6858282Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:20.6858644Z return func(*args, **kwargs) 2025-12-04T09:40:20.6858999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:20.6859388Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:20.6859772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:20.6860145Z return forward_fn(*input_tensors) 2025-12-04T09:40:20.6860539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:20.6860992Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:20.6861405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:20.6861812Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:20.6862167Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:20.6862492Z return self.act(input) 2025-12-04T09:40:20.6862596Z 2025-12-04T09:40:20.6862677Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6862865Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6863058Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6863246Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6863427Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6863621Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6863818Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6864012Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6864197Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6864389Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6864579Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6864793Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:20.6865141Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:20.6865453Z res = mod(**inputs) 2025-12-04T09:40:20.6865806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:20.6866182Z outputs = self.electra( 2025-12-04T09:40:20.6866543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:20.6866921Z hidden_states = self.encoder( 2025-12-04T09:40:20.6867314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:20.6867692Z layer_outputs = layer_module( 2025-12-04T09:40:20.6868027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:20.6868362Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:20.6868727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:20.6869083Z return func(*args, **kwargs) 2025-12-04T09:40:20.6869443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:20.6869819Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:20.6870199Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:20.6870604Z return forward_fn(*input_tensors) 2025-12-04T09:40:20.6871004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:20.6871445Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:20.6871863Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:20.6872271Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:20.6872624Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:20.6872956Z return self.act(input) 2025-12-04T09:40:20.6873147Z 2025-12-04T09:40:20.6873228Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6873436Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6873635Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6873840Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6874039Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6874237Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6874431Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6874625Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6874813Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6875012Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6875209Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6875431Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:20.6875774Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:20.6876089Z res = mod(**inputs) 2025-12-04T09:40:20.6876466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:20.6876851Z outputs = self.electra( 2025-12-04T09:40:20.6877224Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:20.6877613Z hidden_states = self.encoder( 2025-12-04T09:40:20.6877990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:20.6878365Z layer_outputs = layer_module( 2025-12-04T09:40:20.6878703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:20.6879060Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:20.6879425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:20.6879800Z return func(*args, **kwargs) 2025-12-04T09:40:20.6880171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:20.6880622Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:20.6881004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:20.6881392Z return forward_fn(*input_tensors) 2025-12-04T09:40:20.6881804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:20.6882267Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:20.6882681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:20.6883098Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:20.6883467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:20.6883825Z return self.act(input) 2025-12-04T09:40:20.6883936Z 2025-12-04T09:40:20.6884013Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6884217Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6884414Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6884601Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6884795Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6884988Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6885174Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6885368Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6885563Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6885750Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6885952Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6886170Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:20.6886516Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:20.6886833Z res = mod(**inputs) 2025-12-04T09:40:20.6887204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:20.6887574Z outputs = self.electra( 2025-12-04T09:40:20.6887937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:20.6888310Z hidden_states = self.encoder( 2025-12-04T09:40:20.6888667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:20.6889041Z layer_outputs = layer_module( 2025-12-04T09:40:20.6889377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:20.6889724Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:20.6890079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:20.6890443Z return func(*args, **kwargs) 2025-12-04T09:40:20.6890804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:20.6891188Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:20.6891568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:20.6891940Z return forward_fn(*input_tensors) 2025-12-04T09:40:20.6892338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:20.6892777Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:20.6893192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:20.6893601Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:20.6893990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:20.6894310Z return self.act(input) 2025-12-04T09:40:20.6894421Z 2025-12-04T09:40:20.6894495Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6894689Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6894876Z cudagraph partition due to non gpu ops 2025-12-04T09:40:20.6895095Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:20.6895436Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:20.6895744Z res = mod(**inputs) 2025-12-04T09:40:20.6896093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1554, in forward 2025-12-04T09:40:20.6896472Z lm_loss = self.loss_function( 2025-12-04T09:40:20.6896866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 67, in ForCausalLMLoss 2025-12-04T09:40:20.6897316Z loss = fixed_cross_entropy(logits, shift_labels, num_items_in_batch, ignore_index, **kwargs) 2025-12-04T09:40:20.6897779Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 36, in fixed_cross_entropy 2025-12-04T09:40:20.6898265Z loss = nn.functional.cross_entropy(source, target, ignore_index=ignore_index, reduction=reduction) 2025-12-04T09:40:20.6898504Z 2025-12-04T09:40:30.1618328Z Compilation time (from dynamo_timed): 18.016353273 2025-12-04T09:40:30.1722499Z pass 2025-12-04T09:40:30.1722842Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:40:30.1723602Z TIMING: _recursive_pre_grad_passes:0.03107 _recursive_joint_graph_passes:0.42655 _recursive_post_grad_passes:0.05416 async_compile.wait:0.80084 code_gen:9.20447 inductor_compile:11.17772 backend_compile:15.20192 gc:0.00032 entire_frame_compile:18.01635 total_wall_time:18.01635 2025-12-04T09:40:30.1724527Z STATS: call_* op count: 377 | FakeTensorMode.__torch_dispatch__:20521 | FakeTensor.__torch_dispatch__:3510 | ProxyTorchDispatchMode.__torch_dispatch__:3375 2025-12-04T09:40:30.1725003Z Dynamo produced 1 graphs covering 377 ops with 0 graph breaks (0 unique) 2025-12-04T09:40:32.5654228Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:40:32.5658719Z import pynvml # type: ignore[import] 2025-12-04T09:40:35.7497461Z 2025-12-04T09:40:36.9405161Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:40:36.9410147Z loading model: 0it [00:01, ?it/s] 2025-12-04T09:40:36.9415259Z cpu eval GPT2ForSequenceClassification 2025-12-04T09:40:37.6721427Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:40:37.9970908Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:40:38.3202617Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:40:46.0318155Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0319813Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0320279Z res = mod(**inputs) 2025-12-04T09:40:46.0325386Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0327028Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0327624Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:40:46.0332934Z causal_mask = create_causal_mask( 2025-12-04T09:40:46.0335455Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:40:46.0336134Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:40:46.0339390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:40:46.0340029Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:40:46.0340558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 655, in find_packed_sequence_indices 2025-12-04T09:40:46.0343364Z first_dummy_value = position_ids[:, :1] - 1 # We just need the diff on this first value to be 1 2025-12-04T09:40:46.0343695Z 2025-12-04T09:40:46.0343894Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0344389Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0344895Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0345237Z res = mod(**inputs) 2025-12-04T09:40:46.0345628Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1394, in forward 2025-12-04T09:40:46.0346060Z last_non_pad_token = (token_indices * non_pad_mask).argmax(-1) 2025-12-04T09:40:46.0346408Z 2025-12-04T09:40:46.0346492Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0346726Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0347079Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0347394Z res = mod(**inputs) 2025-12-04T09:40:46.0347753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0348147Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0348544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:40:46.0348933Z causal_mask = create_causal_mask( 2025-12-04T09:40:46.0349305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:40:46.0349796Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:40:46.0350323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:40:46.0350771Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:40:46.0351217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 656, in find_packed_sequence_indices 2025-12-04T09:40:46.0351685Z position_diff = torch.diff(position_ids, prepend=first_dummy_value, dim=-1) 2025-12-04T09:40:46.0351908Z 2025-12-04T09:40:46.0351985Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0352219Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0352568Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0352946Z res = mod(**inputs) 2025-12-04T09:40:46.0353458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0353916Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0354346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:40:46.0354728Z causal_mask = create_causal_mask( 2025-12-04T09:40:46.0355102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:40:46.0355653Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:40:46.0356154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:40:46.0356589Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:40:46.0357029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 657, in find_packed_sequence_indices 2025-12-04T09:40:46.0357482Z packed_sequence_mask = (position_diff != 1).cumsum(-1) 2025-12-04T09:40:46.0357651Z 2025-12-04T09:40:46.0357754Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0358115Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0358429Z res = mod(**inputs) 2025-12-04T09:40:46.0358814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0359196Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0359572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:40:46.0359930Z causal_mask = create_causal_mask( 2025-12-04T09:40:46.0360289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:40:46.0360760Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:40:46.0361245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:40:46.0361665Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:40:46.0362092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 657, in find_packed_sequence_indices 2025-12-04T09:40:46.0362523Z packed_sequence_mask = (position_diff != 1).cumsum(-1) 2025-12-04T09:40:46.0362680Z 2025-12-04T09:40:46.0362790Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0363133Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0363453Z res = mod(**inputs) 2025-12-04T09:40:46.0363812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0364173Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0364537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0364884Z outputs = block( 2025-12-04T09:40:46.0365190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0365527Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0365887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0366241Z return func(*args, **kwargs) 2025-12-04T09:40:46.0366586Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:46.0366950Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:46.0367314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0367661Z return func(*args, **kwargs) 2025-12-04T09:40:46.0367996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:46.0368373Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:46.0368819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:46.0369284Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:46.0369459Z 2025-12-04T09:40:46.0369536Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0369741Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0369937Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0370119Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0370340Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0370678Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0370981Z res = mod(**inputs) 2025-12-04T09:40:46.0371317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0371693Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0372091Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0372431Z outputs = block( 2025-12-04T09:40:46.0372737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0373081Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0373436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0373778Z return func(*args, **kwargs) 2025-12-04T09:40:46.0374129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:46.0374497Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:46.0374853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0375204Z return func(*args, **kwargs) 2025-12-04T09:40:46.0375549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:46.0375923Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:46.0376332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:46.0376782Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:46.0376957Z 2025-12-04T09:40:46.0377032Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0377227Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0377438Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0377769Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0378067Z res = mod(**inputs) 2025-12-04T09:40:46.0378401Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0378771Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0379134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0379478Z outputs = block( 2025-12-04T09:40:46.0379778Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0380116Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0380536Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0380894Z return func(*args, **kwargs) 2025-12-04T09:40:46.0381244Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:46.0381642Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:46.0382062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:46.0382430Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:46.0382774Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:46.0383230Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:46.0383465Z 2025-12-04T09:40:46.0383550Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0383749Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0383956Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0384148Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0384331Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0384523Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0384774Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0385123Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0385445Z res = mod(**inputs) 2025-12-04T09:40:46.0385804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0386196Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0386576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0386949Z outputs = block( 2025-12-04T09:40:46.0387276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0387634Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0388016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0388393Z return func(*args, **kwargs) 2025-12-04T09:40:46.0388769Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:46.0389157Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:46.0389550Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0389921Z return func(*args, **kwargs) 2025-12-04T09:40:46.0390288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:46.0390685Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:46.0391128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:46.0391609Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:46.0391792Z 2025-12-04T09:40:46.0391870Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0392082Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0392316Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0392673Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0392990Z res = mod(**inputs) 2025-12-04T09:40:46.0393501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0393976Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0394396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0394854Z outputs = block( 2025-12-04T09:40:46.0395181Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0395551Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0396031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0396397Z return func(*args, **kwargs) 2025-12-04T09:40:46.0396755Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:46.0397149Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:46.0397541Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:46.0397914Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:46.0398259Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:46.0398708Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:46.0399681Z 2025-12-04T09:40:46.0399759Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0399974Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0400181Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0400373Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0400573Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0400775Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0400998Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0401360Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0401683Z res = mod(**inputs) 2025-12-04T09:40:46.0402037Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0402433Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0402821Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0403200Z outputs = block( 2025-12-04T09:40:46.0403521Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0403887Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0404273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0404651Z return func(*args, **kwargs) 2025-12-04T09:40:46.0405018Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:46.0405418Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:46.0406128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0406559Z return func(*args, **kwargs) 2025-12-04T09:40:46.0406941Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:46.0407351Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:46.0407803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:46.0408280Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:46.0408472Z 2025-12-04T09:40:46.0408547Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0408754Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0408980Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0409337Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0409657Z res = mod(**inputs) 2025-12-04T09:40:46.0410014Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0410487Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0410881Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0411252Z outputs = block( 2025-12-04T09:40:46.0411571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0411938Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0412320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0412694Z return func(*args, **kwargs) 2025-12-04T09:40:46.0413055Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:46.0413471Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:46.0413930Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:46.0414300Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:46.0414629Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:46.0415060Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:46.0415277Z 2025-12-04T09:40:46.0415360Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0415549Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0415743Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0415934Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0416126Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0416313Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0416537Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0416885Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0417187Z res = mod(**inputs) 2025-12-04T09:40:46.0417530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0417904Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0418275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0418622Z outputs = block( 2025-12-04T09:40:46.0418929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0419275Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0419627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0419981Z return func(*args, **kwargs) 2025-12-04T09:40:46.0420341Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:46.0420722Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:46.0421084Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0421434Z return func(*args, **kwargs) 2025-12-04T09:40:46.0421787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:46.0422165Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:46.0422593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:46.0423060Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:46.0423234Z 2025-12-04T09:40:46.0423318Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0423511Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0423771Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0424126Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0424445Z res = mod(**inputs) 2025-12-04T09:40:46.0424803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0425197Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0425582Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0426030Z outputs = block( 2025-12-04T09:40:46.0426381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0426729Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0427146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0427506Z return func(*args, **kwargs) 2025-12-04T09:40:46.0427872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:46.0428275Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:46.0428662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:46.0429039Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:46.0429377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:46.0429819Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:46.0430041Z 2025-12-04T09:40:46.0430120Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0430320Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0430519Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0430709Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0430900Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0431091Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0431315Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0431655Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0431965Z res = mod(**inputs) 2025-12-04T09:40:46.0432315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0432693Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0433139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0433577Z outputs = block( 2025-12-04T09:40:46.0433933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0434327Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0434712Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0435081Z return func(*args, **kwargs) 2025-12-04T09:40:46.0435437Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:46.0435877Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:46.0436317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0436774Z return func(*args, **kwargs) 2025-12-04T09:40:46.0437171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:46.0437678Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:46.0438179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:46.0438709Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:46.0438905Z 2025-12-04T09:40:46.0438988Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0439207Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0439459Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0439837Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0440211Z res = mod(**inputs) 2025-12-04T09:40:46.0440602Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0441154Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0441612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0442001Z outputs = block( 2025-12-04T09:40:46.0442312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0442656Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0443015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0443366Z return func(*args, **kwargs) 2025-12-04T09:40:46.0443729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:46.0444123Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:46.0444502Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:46.0444877Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:46.0445210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:46.0445642Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:46.0445862Z 2025-12-04T09:40:46.0445936Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0446135Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0446328Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0446512Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0446701Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0446890Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0447109Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0447445Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0447761Z res = mod(**inputs) 2025-12-04T09:40:46.0448107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0448478Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0448850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0449208Z outputs = block( 2025-12-04T09:40:46.0449519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0449861Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0450223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0450585Z return func(*args, **kwargs) 2025-12-04T09:40:46.0450930Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:46.0451347Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:46.0451713Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0452060Z return func(*args, **kwargs) 2025-12-04T09:40:46.0452398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:46.0452775Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:46.0453194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:46.0453642Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:46.0453814Z 2025-12-04T09:40:46.0453888Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0454113Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0454331Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0454666Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0454973Z res = mod(**inputs) 2025-12-04T09:40:46.0455319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0455693Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0456057Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0456413Z outputs = block( 2025-12-04T09:40:46.0456719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0457056Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0457418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0457778Z return func(*args, **kwargs) 2025-12-04T09:40:46.0458131Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:46.0458524Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:46.0458907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:46.0459275Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:46.0459600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:46.0460035Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:46.0460261Z 2025-12-04T09:40:46.0460333Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0460536Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0460723Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0460916Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0461107Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0461294Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0461519Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0461860Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0462166Z res = mod(**inputs) 2025-12-04T09:40:46.0462501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0462881Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0463256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0463604Z outputs = block( 2025-12-04T09:40:46.0463918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0464308Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0464670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0465020Z return func(*args, **kwargs) 2025-12-04T09:40:46.0465369Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:46.0465747Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:46.0466117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0466469Z return func(*args, **kwargs) 2025-12-04T09:40:46.0466818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:46.0467244Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:46.0467677Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:46.0468146Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:46.0468329Z 2025-12-04T09:40:46.0468404Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0468602Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0468820Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0469172Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0469488Z res = mod(**inputs) 2025-12-04T09:40:46.0469830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0470278Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0470660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0471031Z outputs = block( 2025-12-04T09:40:46.0471339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0471695Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0472066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0472426Z return func(*args, **kwargs) 2025-12-04T09:40:46.0472834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:46.0473338Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:46.0473764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:46.0474182Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:46.0474548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:46.0475041Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:46.0475268Z 2025-12-04T09:40:46.0475351Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0475548Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0475749Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0475948Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0476141Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0476338Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0476562Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0476901Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0477214Z res = mod(**inputs) 2025-12-04T09:40:46.0477607Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0477997Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0478373Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0478744Z outputs = block( 2025-12-04T09:40:46.0479063Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0479419Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0479779Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0480151Z return func(*args, **kwargs) 2025-12-04T09:40:46.0480514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:46.0480930Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:46.0481316Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0481679Z return func(*args, **kwargs) 2025-12-04T09:40:46.0482039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:46.0482426Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:46.0482862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:46.0483326Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:46.0483522Z 2025-12-04T09:40:46.0483594Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0483787Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0484001Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0484346Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0484651Z res = mod(**inputs) 2025-12-04T09:40:46.0484996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0485363Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0485733Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0486085Z outputs = block( 2025-12-04T09:40:46.0486384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0486735Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0487093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0487454Z return func(*args, **kwargs) 2025-12-04T09:40:46.0487798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:46.0488195Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:46.0488580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:46.0488945Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:46.0489270Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:46.0489702Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:46.0489921Z 2025-12-04T09:40:46.0490001Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0490189Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0490380Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0490574Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0490800Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0490990Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0491236Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0491577Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0491881Z res = mod(**inputs) 2025-12-04T09:40:46.0492228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0492603Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0492974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0493332Z outputs = block( 2025-12-04T09:40:46.0493647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0494036Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0494392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0494747Z return func(*args, **kwargs) 2025-12-04T09:40:46.0495101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:46.0495481Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:46.0495845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0496204Z return func(*args, **kwargs) 2025-12-04T09:40:46.0496555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:46.0496937Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:46.0497371Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:46.0497828Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:46.0497999Z 2025-12-04T09:40:46.0498080Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0498269Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0498489Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0498824Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0499120Z res = mod(**inputs) 2025-12-04T09:40:46.0499458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0499832Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0500196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0500548Z outputs = block( 2025-12-04T09:40:46.0500854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0501200Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0501551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0501908Z return func(*args, **kwargs) 2025-12-04T09:40:46.0502255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:46.0502644Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:46.0503020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:46.0503388Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:46.0503730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:46.0504195Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:46.0504416Z 2025-12-04T09:40:46.0504488Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0504686Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0504879Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0505063Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0505255Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0505445Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0505841Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0506208Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0506517Z res = mod(**inputs) 2025-12-04T09:40:46.0506861Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0507306Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0507677Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0508032Z outputs = block( 2025-12-04T09:40:46.0508343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0508683Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0509045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0509404Z return func(*args, **kwargs) 2025-12-04T09:40:46.0509752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:46.0510133Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:46.0510510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0510868Z return func(*args, **kwargs) 2025-12-04T09:40:46.0511215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:46.0511604Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:46.0512032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:46.0512483Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:46.0512667Z 2025-12-04T09:40:46.0512741Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0512944Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0513237Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0513601Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0513932Z res = mod(**inputs) 2025-12-04T09:40:46.0514307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0514679Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0515050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0515408Z outputs = block( 2025-12-04T09:40:46.0515719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0516060Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0516423Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0516784Z return func(*args, **kwargs) 2025-12-04T09:40:46.0517144Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:46.0517587Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:46.0517969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:46.0518329Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:46.0518649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:46.0519071Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:46.0519290Z 2025-12-04T09:40:46.0519364Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0519561Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0519748Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0519940Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0520165Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0520351Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0520572Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0520912Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0521217Z res = mod(**inputs) 2025-12-04T09:40:46.0521552Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0521928Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0522293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0522646Z outputs = block( 2025-12-04T09:40:46.0522957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0523307Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0523675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0524025Z return func(*args, **kwargs) 2025-12-04T09:40:46.0524379Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:46.0524760Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:46.0525122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0525475Z return func(*args, **kwargs) 2025-12-04T09:40:46.0525826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:46.0526208Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:46.0526626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:46.0527093Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:46.0527264Z 2025-12-04T09:40:46.0527346Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0527544Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0527757Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0528097Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0528405Z res = mod(**inputs) 2025-12-04T09:40:46.0528740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0529116Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0529489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0529843Z outputs = block( 2025-12-04T09:40:46.0530186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0530534Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0530891Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0531239Z return func(*args, **kwargs) 2025-12-04T09:40:46.0531593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:46.0531991Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:46.0532380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:46.0532737Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:46.0533078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:46.0533545Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:46.0533769Z 2025-12-04T09:40:46.0533852Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0534047Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0534244Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0534439Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0534624Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0534817Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0535040Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0535374Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0535682Z res = mod(**inputs) 2025-12-04T09:40:46.0536024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0536401Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0536766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0537126Z outputs = block( 2025-12-04T09:40:46.0537436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0537777Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0538142Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0538503Z return func(*args, **kwargs) 2025-12-04T09:40:46.0538851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:46.0539225Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:46.0539593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0539952Z return func(*args, **kwargs) 2025-12-04T09:40:46.0540297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:46.0540689Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:46.0541116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:46.0541573Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:46.0541744Z 2025-12-04T09:40:46.0541818Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0542015Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0542237Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0542576Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0542880Z res = mod(**inputs) 2025-12-04T09:40:46.0543273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:46.0543652Z transformer_outputs = self.transformer( 2025-12-04T09:40:46.0544018Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:46.0544373Z outputs = block( 2025-12-04T09:40:46.0544685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:46.0545031Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:46.0545388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:46.0545750Z return func(*args, **kwargs) 2025-12-04T09:40:46.0546099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:46.0546546Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:46.0546933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:46.0547304Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:46.0547634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:46.0548059Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:46.0548289Z 2025-12-04T09:40:46.0548363Z cudagraph partition due to non gpu ops 2025-12-04T09:40:46.0548588Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0548938Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0549241Z res = mod(**inputs) 2025-12-04T09:40:46.0549587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1379, in forward 2025-12-04T09:40:46.0549973Z logits = self.score(hidden_states) 2025-12-04T09:40:46.0550093Z 2025-12-04T09:40:46.0550191Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0550522Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0550828Z res = mod(**inputs) 2025-12-04T09:40:46.0551166Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1422, in forward 2025-12-04T09:40:46.0551687Z loss = loss_fct(pooled_logits.view(-1, self.num_labels), labels.view(-1)) 2025-12-04T09:40:46.0551878Z 2025-12-04T09:40:46.0551971Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:46.0552298Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:46.0552596Z res = mod(**inputs) 2025-12-04T09:40:46.0552936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1422, in forward 2025-12-04T09:40:46.0553469Z loss = loss_fct(pooled_logits.view(-1, self.num_labels), labels.view(-1)) 2025-12-04T09:40:46.0553664Z 2025-12-04T09:41:01.2587444Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2588159Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2588546Z res = mod(**inputs) 2025-12-04T09:41:01.2588944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2589366Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2589791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:41:01.2590268Z causal_mask = create_causal_mask( 2025-12-04T09:41:01.2591081Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:41:01.2591635Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:41:01.2592146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:41:01.2592600Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:41:01.2593050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 655, in find_packed_sequence_indices 2025-12-04T09:41:01.2593676Z first_dummy_value = position_ids[:, :1] - 1 # We just need the diff on this first value to be 1 2025-12-04T09:41:01.2593897Z 2025-12-04T09:41:01.2593977Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2594212Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2594683Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2595021Z res = mod(**inputs) 2025-12-04T09:41:01.2595372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1394, in forward 2025-12-04T09:41:01.2595799Z last_non_pad_token = (token_indices * non_pad_mask).argmax(-1) 2025-12-04T09:41:01.2595966Z 2025-12-04T09:41:01.2596055Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2596283Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2596634Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2596948Z res = mod(**inputs) 2025-12-04T09:41:01.2597293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2597673Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2598065Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:41:01.2598439Z causal_mask = create_causal_mask( 2025-12-04T09:41:01.2598801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:41:01.2599274Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:41:01.2599772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:41:01.2600206Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:41:01.2600635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 656, in find_packed_sequence_indices 2025-12-04T09:41:01.2601098Z position_diff = torch.diff(position_ids, prepend=first_dummy_value, dim=-1) 2025-12-04T09:41:01.2601313Z 2025-12-04T09:41:01.2601391Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2601725Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2602065Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2602379Z res = mod(**inputs) 2025-12-04T09:41:01.2602723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2603108Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2603478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:41:01.2603852Z causal_mask = create_causal_mask( 2025-12-04T09:41:01.2604217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:41:01.2604694Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:41:01.2605228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:41:01.2605884Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:41:01.2606339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 657, in find_packed_sequence_indices 2025-12-04T09:41:01.2606763Z packed_sequence_mask = (position_diff != 1).cumsum(-1) 2025-12-04T09:41:01.2606929Z 2025-12-04T09:41:01.2607031Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2607402Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2607711Z res = mod(**inputs) 2025-12-04T09:41:01.2608045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2608510Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2608889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:41:01.2609262Z causal_mask = create_causal_mask( 2025-12-04T09:41:01.2609637Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:41:01.2610122Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:41:01.2610626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:41:01.2611050Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:41:01.2611482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 657, in find_packed_sequence_indices 2025-12-04T09:41:01.2611907Z packed_sequence_mask = (position_diff != 1).cumsum(-1) 2025-12-04T09:41:01.2612064Z 2025-12-04T09:41:01.2612173Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2612507Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2612819Z res = mod(**inputs) 2025-12-04T09:41:01.2613164Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2613553Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2613931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2614295Z outputs = block( 2025-12-04T09:41:01.2614618Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2614968Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2615352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2615725Z return func(*args, **kwargs) 2025-12-04T09:41:01.2616083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:01.2616460Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:01.2616834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2617187Z return func(*args, **kwargs) 2025-12-04T09:41:01.2617528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:01.2617915Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:01.2618344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:01.2618858Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:01.2619033Z 2025-12-04T09:41:01.2619110Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2619314Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2619510Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2619703Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2619913Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2620253Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2620558Z res = mod(**inputs) 2025-12-04T09:41:01.2620893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2621266Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2621639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2622038Z outputs = block( 2025-12-04T09:41:01.2622340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2622686Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2623047Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2623393Z return func(*args, **kwargs) 2025-12-04T09:41:01.2623743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:01.2624117Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:01.2624483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2624827Z return func(*args, **kwargs) 2025-12-04T09:41:01.2625182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:01.2625567Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:01.2625988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:01.2626443Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:01.2626620Z 2025-12-04T09:41:01.2626694Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2626890Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2627103Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2627439Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2627741Z res = mod(**inputs) 2025-12-04T09:41:01.2628071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2628447Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2628820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2629175Z outputs = block( 2025-12-04T09:41:01.2629476Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2629823Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2630182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2630534Z return func(*args, **kwargs) 2025-12-04T09:41:01.2630877Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:01.2631270Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:01.2631698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:01.2632062Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:01.2632410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:01.2632858Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:01.2633161Z 2025-12-04T09:41:01.2633260Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2633465Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2633674Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2633879Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2634080Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2634296Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2634527Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2634937Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2635251Z res = mod(**inputs) 2025-12-04T09:41:01.2635613Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2635986Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2636363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2636733Z outputs = block( 2025-12-04T09:41:01.2637058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2637430Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2637804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2638176Z return func(*args, **kwargs) 2025-12-04T09:41:01.2638555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:01.2638941Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:01.2639330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2639701Z return func(*args, **kwargs) 2025-12-04T09:41:01.2640093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:01.2640498Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:01.2640939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:01.2641407Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:01.2641596Z 2025-12-04T09:41:01.2641679Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2641889Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2642126Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2642478Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2642803Z res = mod(**inputs) 2025-12-04T09:41:01.2643162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2643556Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2643933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2644303Z outputs = block( 2025-12-04T09:41:01.2644628Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2644982Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2645399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2645768Z return func(*args, **kwargs) 2025-12-04T09:41:01.2646131Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:01.2646519Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:01.2646910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:01.2647271Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:01.2647598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:01.2648019Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:01.2648241Z 2025-12-04T09:41:01.2648353Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2648551Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2648739Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2648929Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2649118Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2649299Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2649517Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2649853Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2650158Z res = mod(**inputs) 2025-12-04T09:41:01.2650493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2650865Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2651233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2651581Z outputs = block( 2025-12-04T09:41:01.2651892Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2652239Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2652600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2652945Z return func(*args, **kwargs) 2025-12-04T09:41:01.2653291Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:01.2653664Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:01.2654022Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2654378Z return func(*args, **kwargs) 2025-12-04T09:41:01.2654729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:01.2655120Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:01.2655540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:01.2655997Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:01.2656170Z 2025-12-04T09:41:01.2656248Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2656442Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2656652Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2656986Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2657290Z res = mod(**inputs) 2025-12-04T09:41:01.2657623Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2658003Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2658407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2658768Z outputs = block( 2025-12-04T09:41:01.2659070Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2659419Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2659781Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2660131Z return func(*args, **kwargs) 2025-12-04T09:41:01.2660485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:01.2660880Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:01.2661268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:01.2661683Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:01.2662038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:01.2662469Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:01.2662688Z 2025-12-04T09:41:01.2662769Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2662960Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2663180Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2663368Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2663552Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2663739Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2663954Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2664283Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2664593Z res = mod(**inputs) 2025-12-04T09:41:01.2664940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2665318Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2665680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2666037Z outputs = block( 2025-12-04T09:41:01.2666347Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2666686Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2667048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2667406Z return func(*args, **kwargs) 2025-12-04T09:41:01.2667756Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:01.2668131Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:01.2668502Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2668859Z return func(*args, **kwargs) 2025-12-04T09:41:01.2669204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:01.2669593Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:01.2670022Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:01.2670478Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:01.2670651Z 2025-12-04T09:41:01.2670723Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2670922Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2671144Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2671525Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2671830Z res = mod(**inputs) 2025-12-04T09:41:01.2672180Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2672558Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2672930Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2673367Z outputs = block( 2025-12-04T09:41:01.2673689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2674107Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2674460Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2674857Z return func(*args, **kwargs) 2025-12-04T09:41:01.2675212Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:01.2675600Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:01.2675997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:01.2676376Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:01.2676714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:01.2677144Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:01.2677379Z 2025-12-04T09:41:01.2677453Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2677655Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2677855Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2678044Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2678239Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2678431Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2678646Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2678998Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2679313Z res = mod(**inputs) 2025-12-04T09:41:01.2679651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2680034Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2680407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2680766Z outputs = block( 2025-12-04T09:41:01.2681077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2681436Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2681807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2682165Z return func(*args, **kwargs) 2025-12-04T09:41:01.2682526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:01.2682912Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:01.2683289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2683644Z return func(*args, **kwargs) 2025-12-04T09:41:01.2684000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:01.2684396Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:01.2684884Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:01.2685345Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:01.2685531Z 2025-12-04T09:41:01.2685608Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2685810Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2686031Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2686379Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2686691Z res = mod(**inputs) 2025-12-04T09:41:01.2687040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2687421Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2687829Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2688177Z outputs = block( 2025-12-04T09:41:01.2688475Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2688810Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2689162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2689509Z return func(*args, **kwargs) 2025-12-04T09:41:01.2689842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:01.2690222Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:01.2690599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:01.2690961Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:01.2691286Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:01.2691702Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:01.2691915Z 2025-12-04T09:41:01.2691995Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2692181Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2692372Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2692560Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2692746Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2692924Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2693136Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2693465Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2693753Z res = mod(**inputs) 2025-12-04T09:41:01.2694090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2694454Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2694805Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2695148Z outputs = block( 2025-12-04T09:41:01.2695445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2695779Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2696120Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2696463Z return func(*args, **kwargs) 2025-12-04T09:41:01.2696803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:01.2697175Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:01.2697559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2697916Z return func(*args, **kwargs) 2025-12-04T09:41:01.2698265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:01.2698640Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:01.2699069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:01.2699528Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:01.2699701Z 2025-12-04T09:41:01.2699783Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2699974Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2700197Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2700576Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2700871Z res = mod(**inputs) 2025-12-04T09:41:01.2701211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2701586Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2701951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2702306Z outputs = block( 2025-12-04T09:41:01.2702614Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2702962Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2703322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2703672Z return func(*args, **kwargs) 2025-12-04T09:41:01.2704019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:01.2704405Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:01.2704785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:01.2705152Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:01.2705485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:01.2706074Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:01.2706302Z 2025-12-04T09:41:01.2706377Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2706576Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2706771Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2706956Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2707150Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2707344Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2707563Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2707895Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2708199Z res = mod(**inputs) 2025-12-04T09:41:01.2708537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2708901Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2709269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2709621Z outputs = block( 2025-12-04T09:41:01.2709928Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2710269Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2710699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2711051Z return func(*args, **kwargs) 2025-12-04T09:41:01.2711396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:01.2711772Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:01.2712141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2712490Z return func(*args, **kwargs) 2025-12-04T09:41:01.2712833Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:01.2713274Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:01.2713765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:01.2714239Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:01.2714412Z 2025-12-04T09:41:01.2714486Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2714682Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2714904Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2715240Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2715548Z res = mod(**inputs) 2025-12-04T09:41:01.2715890Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2716268Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2716634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2716996Z outputs = block( 2025-12-04T09:41:01.2717305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2717645Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2718009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2718361Z return func(*args, **kwargs) 2025-12-04T09:41:01.2718712Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:01.2719095Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:01.2719482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:01.2719852Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:01.2720177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:01.2720614Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:01.2720839Z 2025-12-04T09:41:01.2720913Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2721110Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2721294Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2721484Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2721674Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2721856Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2722075Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2722412Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2722717Z res = mod(**inputs) 2025-12-04T09:41:01.2723053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2723469Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2723835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2724182Z outputs = block( 2025-12-04T09:41:01.2724489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2724831Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2725189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2725533Z return func(*args, **kwargs) 2025-12-04T09:41:01.2725887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:01.2726262Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:01.2726656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2727008Z return func(*args, **kwargs) 2025-12-04T09:41:01.2727359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:01.2727742Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:01.2728158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:01.2728616Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:01.2728795Z 2025-12-04T09:41:01.2728871Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2729069Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2729280Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2729622Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2729933Z res = mod(**inputs) 2025-12-04T09:41:01.2730272Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2730647Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2731015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2731372Z outputs = block( 2025-12-04T09:41:01.2731676Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2732021Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2732384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2732729Z return func(*args, **kwargs) 2025-12-04T09:41:01.2733081Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:01.2733474Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:01.2733856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:01.2734216Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:01.2734553Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:01.2734983Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:01.2735202Z 2025-12-04T09:41:01.2735281Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2735471Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2735666Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2735856Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2736043Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2736237Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2736509Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2736846Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2737154Z res = mod(**inputs) 2025-12-04T09:41:01.2737499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2737876Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2738236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2738592Z outputs = block( 2025-12-04T09:41:01.2738898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2739240Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2739639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2739993Z return func(*args, **kwargs) 2025-12-04T09:41:01.2740344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:01.2740716Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:01.2741083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2741433Z return func(*args, **kwargs) 2025-12-04T09:41:01.2741785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:01.2742161Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:01.2742586Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:01.2743049Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:01.2743221Z 2025-12-04T09:41:01.2743294Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2743493Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2743716Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2744060Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2744362Z res = mod(**inputs) 2025-12-04T09:41:01.2744704Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2745080Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2745444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2745799Z outputs = block( 2025-12-04T09:41:01.2746112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2746460Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2746813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2747173Z return func(*args, **kwargs) 2025-12-04T09:41:01.2747527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:01.2747915Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:01.2748306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:01.2748673Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:01.2749009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:01.2749463Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:01.2749693Z 2025-12-04T09:41:01.2749766Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2749962Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2750151Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2750335Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2750524Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2750714Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2750926Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2751267Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2751577Z res = mod(**inputs) 2025-12-04T09:41:01.2751914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2752322Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2752689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2753123Z outputs = block( 2025-12-04T09:41:01.2753457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2753824Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2754208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2754566Z return func(*args, **kwargs) 2025-12-04T09:41:01.2754908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:01.2755286Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:01.2755664Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2756014Z return func(*args, **kwargs) 2025-12-04T09:41:01.2756378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:01.2756760Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:01.2757183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:01.2757630Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:01.2757816Z 2025-12-04T09:41:01.2757887Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2758082Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2758297Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2758633Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2758941Z res = mod(**inputs) 2025-12-04T09:41:01.2759284Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2759650Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2760017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2760371Z outputs = block( 2025-12-04T09:41:01.2760673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2761020Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2761380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2761734Z return func(*args, **kwargs) 2025-12-04T09:41:01.2762079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:01.2762474Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:01.2762900Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:01.2763273Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:01.2763598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:01.2764028Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:01.2764249Z 2025-12-04T09:41:01.2764337Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2764526Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2764721Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2764916Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2765107Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2765326Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2765549Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2765894Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2766194Z res = mod(**inputs) 2025-12-04T09:41:01.2766540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2766917Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2767288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2767633Z outputs = block( 2025-12-04T09:41:01.2767943Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2768292Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2768645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2769006Z return func(*args, **kwargs) 2025-12-04T09:41:01.2769358Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:01.2769735Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:01.2770095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2770449Z return func(*args, **kwargs) 2025-12-04T09:41:01.2770798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:01.2771175Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:01.2771599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:01.2772062Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:01.2772232Z 2025-12-04T09:41:01.2772313Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2772505Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2772725Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2773063Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2773363Z res = mod(**inputs) 2025-12-04T09:41:01.2773706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2774080Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2774449Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2774795Z outputs = block( 2025-12-04T09:41:01.2775105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2775491Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2775860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2776205Z return func(*args, **kwargs) 2025-12-04T09:41:01.2776558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:01.2776950Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:01.2777331Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:01.2777698Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:01.2778037Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:01.2778466Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:01.2778739Z 2025-12-04T09:41:01.2778816Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2779016Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2779212Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2779394Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2779587Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2779776Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2779996Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2780324Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2780630Z res = mod(**inputs) 2025-12-04T09:41:01.2780978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2781365Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2781745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2782104Z outputs = block( 2025-12-04T09:41:01.2782412Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2782753Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2783114Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2783465Z return func(*args, **kwargs) 2025-12-04T09:41:01.2783811Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:01.2784184Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:01.2784551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2784904Z return func(*args, **kwargs) 2025-12-04T09:41:01.2785254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:01.2785636Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:01.2786066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:01.2786520Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:01.2786692Z 2025-12-04T09:41:01.2786790Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2786984Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2787203Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2787542Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2787840Z res = mod(**inputs) 2025-12-04T09:41:01.2788182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:01.2788590Z transformer_outputs = self.transformer( 2025-12-04T09:41:01.2788965Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:01.2789325Z outputs = block( 2025-12-04T09:41:01.2789638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:01.2789992Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:01.2790348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:01.2790705Z return func(*args, **kwargs) 2025-12-04T09:41:01.2791054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:01.2791443Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:01.2791882Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:01.2792249Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:01.2792578Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:01.2793039Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:01.2793345Z 2025-12-04T09:41:01.2793424Z cudagraph partition due to non gpu ops 2025-12-04T09:41:01.2793658Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2794015Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2794327Z res = mod(**inputs) 2025-12-04T09:41:01.2794687Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1379, in forward 2025-12-04T09:41:01.2795067Z logits = self.score(hidden_states) 2025-12-04T09:41:01.2795190Z 2025-12-04T09:41:01.2795289Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2795638Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2795955Z res = mod(**inputs) 2025-12-04T09:41:01.2796306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1422, in forward 2025-12-04T09:41:01.2796745Z loss = loss_fct(pooled_logits.view(-1, self.num_labels), labels.view(-1)) 2025-12-04T09:41:01.2796945Z 2025-12-04T09:41:01.2797041Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:01.2797385Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:01.2797689Z res = mod(**inputs) 2025-12-04T09:41:01.2798038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1422, in forward 2025-12-04T09:41:01.2798483Z loss = loss_fct(pooled_logits.view(-1, self.num_labels), labels.view(-1)) 2025-12-04T09:41:01.2798672Z 2025-12-04T09:41:02.9722914Z Compilation time (from dynamo_timed): 23.88553277 2025-12-04T09:41:02.9723401Z pass 2025-12-04T09:41:02.9723799Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:41:02.9725205Z TIMING: _recursive_pre_grad_passes:0.06934 _recursive_joint_graph_passes:0.76218 _recursive_post_grad_passes:0.10136 async_compile.wait:0.67418 code_gen:11.02786 inductor_compile:13.60933 backend_compile:19.68252 gc:0.00095 entire_frame_compile:23.88553 total_wall_time:23.88553 2025-12-04T09:41:02.9730552Z STATS: call_* op count: 1126 | FakeTensorMode.__torch_dispatch__:31994 | FakeTensor.__torch_dispatch__:7032 | ProxyTorchDispatchMode.__torch_dispatch__:3409 2025-12-04T09:41:02.9734721Z Dynamo produced 2 graphs covering 1126 ops with 0 graph breaks (0 unique) 2025-12-04T09:41:05.6529237Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:41:05.6530142Z import pynvml # type: ignore[import] 2025-12-04T09:41:08.7654677Z 2025-12-04T09:41:09.5509423Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:41:09.5512544Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:41:09.5514562Z cpu eval GoogleFnet 2025-12-04T09:41:09.9919363Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:41:10.1518159Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:41:10.3079678Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:41:15.9391392Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9392005Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9392352Z res = mod(**inputs) 2025-12-04T09:41:15.9392747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9393304Z outputs = self.fnet( 2025-12-04T09:41:15.9393676Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9394052Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9394423Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9394801Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9395159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9395530Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9395906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9396290Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9396680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9397057Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9397421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9397817Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9397977Z 2025-12-04T09:41:15.9398078Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9398430Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9398741Z res = mod(**inputs) 2025-12-04T09:41:15.9399086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9399463Z outputs = self.fnet( 2025-12-04T09:41:15.9399806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9400168Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9400530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9400974Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9401323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9401683Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9402120Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9402870Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9403273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9403657Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9404039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9404474Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9404637Z 2025-12-04T09:41:15.9404739Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9405095Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9405408Z res = mod(**inputs) 2025-12-04T09:41:15.9406001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9406482Z outputs = self.fnet( 2025-12-04T09:41:15.9406831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9407206Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9407560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9407932Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9408273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9408623Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9408986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9409373Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9409751Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9410138Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9410501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9410885Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9411028Z 2025-12-04T09:41:15.9411125Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9411464Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9411768Z res = mod(**inputs) 2025-12-04T09:41:15.9412095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9412450Z outputs = self.fnet( 2025-12-04T09:41:15.9412790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9413153Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9413499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9413870Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9414220Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9414557Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9414920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9415307Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9415684Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9416044Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9416459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9416849Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9416994Z 2025-12-04T09:41:15.9417100Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9417437Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9417750Z res = mod(**inputs) 2025-12-04T09:41:15.9418096Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9418459Z outputs = self.fnet( 2025-12-04T09:41:15.9418804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9419220Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9419583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9419964Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9420320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9420677Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9421043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9421439Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9421830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9422210Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9422573Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9422978Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9423135Z 2025-12-04T09:41:15.9423234Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9423582Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9423888Z res = mod(**inputs) 2025-12-04T09:41:15.9424231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9424596Z outputs = self.fnet( 2025-12-04T09:41:15.9424932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9425313Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9425682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9426066Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9426414Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9426767Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9427139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9427532Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9427915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9428294Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9428666Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9429058Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9429221Z 2025-12-04T09:41:15.9429322Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9429706Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9430062Z res = mod(**inputs) 2025-12-04T09:41:15.9430484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9430855Z outputs = self.fnet( 2025-12-04T09:41:15.9431196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9431557Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9431916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9432333Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9432700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9433188Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9433593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9434000Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9434398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9434791Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9435167Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9435563Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9435711Z 2025-12-04T09:41:15.9435819Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9436160Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9436474Z res = mod(**inputs) 2025-12-04T09:41:15.9436810Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9437162Z outputs = self.fnet( 2025-12-04T09:41:15.9437501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9437864Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9438215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9438583Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9438931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9439275Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9439639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9440025Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9440407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9440775Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9441132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9441522Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9441665Z 2025-12-04T09:41:15.9441769Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9442134Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9442440Z res = mod(**inputs) 2025-12-04T09:41:15.9442777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9443164Z outputs = self.fnet( 2025-12-04T09:41:15.9443498Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9443860Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9444211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9444581Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9444923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9445270Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9445633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9446042Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9446435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9446807Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9447172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9447555Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9447706Z 2025-12-04T09:41:15.9447803Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9448139Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9448443Z res = mod(**inputs) 2025-12-04T09:41:15.9448772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9449125Z outputs = self.fnet( 2025-12-04T09:41:15.9449469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9449824Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9450181Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9450552Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9450899Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9451235Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9451599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9451982Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9452354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9452723Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9453090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9453477Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9453621Z 2025-12-04T09:41:15.9453718Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9454054Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9454354Z res = mod(**inputs) 2025-12-04T09:41:15.9454689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9455040Z outputs = self.fnet( 2025-12-04T09:41:15.9455374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9455740Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9456142Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9456517Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9456864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9457207Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9457564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9457949Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9458327Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9458685Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9459049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9459471Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9459616Z 2025-12-04T09:41:15.9459722Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9460051Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9460362Z res = mod(**inputs) 2025-12-04T09:41:15.9460697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9461053Z outputs = self.fnet( 2025-12-04T09:41:15.9461379Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9461745Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9462138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9462515Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9462866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9463273Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9463648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9464025Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9464404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9464771Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9465136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9465518Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9465673Z 2025-12-04T09:41:15.9465779Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9466019Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9466348Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9466653Z res = mod(**inputs) 2025-12-04T09:41:15.9466986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9467344Z outputs = self.fnet( 2025-12-04T09:41:15.9467671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 511, in forward 2025-12-04T09:41:15.9468037Z embedding_output = self.embeddings( 2025-12-04T09:41:15.9468410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 141, in forward 2025-12-04T09:41:15.9468779Z embeddings = self.projection(embeddings) 2025-12-04T09:41:15.9468920Z 2025-12-04T09:41:15.9468994Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9469281Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9469625Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9469922Z res = mod(**inputs) 2025-12-04T09:41:15.9470258Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9470619Z outputs = self.fnet( 2025-12-04T09:41:15.9470952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9471319Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9471677Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9472056Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9472426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9472767Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9473178Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9473573Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9473972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9474347Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9474730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9475104Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9475253Z 2025-12-04T09:41:15.9475348Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9475694Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9476004Z res = mod(**inputs) 2025-12-04T09:41:15.9476338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9476702Z outputs = self.fnet( 2025-12-04T09:41:15.9477046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9477412Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9477788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9478171Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9478533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9478883Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9479260Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9479660Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9480053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9480427Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9480799Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9481198Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9481351Z 2025-12-04T09:41:15.9481449Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9481796Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9482112Z res = mod(**inputs) 2025-12-04T09:41:15.9482501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9482870Z outputs = self.fnet( 2025-12-04T09:41:15.9483222Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9483596Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9483952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9484338Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9484695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9485052Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9485417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9485851Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9486244Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9486625Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9487000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9487389Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9487533Z 2025-12-04T09:41:15.9487639Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9487970Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9488273Z res = mod(**inputs) 2025-12-04T09:41:15.9488608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9488972Z outputs = self.fnet( 2025-12-04T09:41:15.9489302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9489668Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9490024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9490401Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9490741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9491079Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9491439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9491816Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9492201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9492574Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9492938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9493321Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9493472Z 2025-12-04T09:41:15.9493547Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9493954Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9494289Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9494586Z res = mod(**inputs) 2025-12-04T09:41:15.9494918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9495276Z outputs = self.fnet( 2025-12-04T09:41:15.9495638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9496005Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9496365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9496742Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9497086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9497434Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9497797Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:15.9498163Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:15.9498552Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:15.9498971Z return forward_fn(*input_tensors) 2025-12-04T09:41:15.9499363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:15.9499791Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:15.9500197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:15.9500601Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:15.9500965Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:15.9501403Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:15.9501636Z 2025-12-04T09:41:15.9501711Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9501911Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9502132Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9502474Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9502777Z res = mod(**inputs) 2025-12-04T09:41:15.9503115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9503468Z outputs = self.fnet( 2025-12-04T09:41:15.9503805Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9504169Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9504517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9504895Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9505242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9505604Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9506099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9506487Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9506868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9507234Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9507591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9507980Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9508128Z 2025-12-04T09:41:15.9508234Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9508566Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9508882Z res = mod(**inputs) 2025-12-04T09:41:15.9509288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9509650Z outputs = self.fnet( 2025-12-04T09:41:15.9509982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9510347Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9510705Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9511072Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9511420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9511764Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9512168Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9512543Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9512927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9513379Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9513777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9514243Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9514397Z 2025-12-04T09:41:15.9514495Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9514854Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9515173Z res = mod(**inputs) 2025-12-04T09:41:15.9515532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9515920Z outputs = self.fnet( 2025-12-04T09:41:15.9516279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9516654Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9517031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9517440Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9517800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9518166Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9518554Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9518962Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9519362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9519754Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9520141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9520551Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9520708Z 2025-12-04T09:41:15.9520813Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9521172Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9521496Z res = mod(**inputs) 2025-12-04T09:41:15.9521891Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9522257Z outputs = self.fnet( 2025-12-04T09:41:15.9522627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9522990Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9523334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9523710Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9524062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9524415Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9524778Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9525179Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9525572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9525988Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9526363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9526763Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9526912Z 2025-12-04T09:41:15.9526997Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9527219Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9527562Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9527874Z res = mod(**inputs) 2025-12-04T09:41:15.9528211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9528581Z outputs = self.fnet( 2025-12-04T09:41:15.9528924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9529297Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9529660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9530049Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9530412Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9530755Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9531111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:15.9531485Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:15.9531868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:15.9532243Z return forward_fn(*input_tensors) 2025-12-04T09:41:15.9532639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:15.9533068Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:15.9533467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:15.9533860Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:15.9534223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:15.9534660Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:15.9534878Z 2025-12-04T09:41:15.9534960Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9535165Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9535385Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9535728Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9536067Z res = mod(**inputs) 2025-12-04T09:41:15.9536410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9536768Z outputs = self.fnet( 2025-12-04T09:41:15.9537111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9537469Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9537825Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9538200Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9538542Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9538887Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9539288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9539676Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9540058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9540437Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9540809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9541207Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9541355Z 2025-12-04T09:41:15.9541454Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9541803Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9542115Z res = mod(**inputs) 2025-12-04T09:41:15.9542457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9542821Z outputs = self.fnet( 2025-12-04T09:41:15.9543160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9543527Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9543880Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9544259Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9544617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9544969Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9545354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9545763Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9546163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9546547Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9546930Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9547341Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9547495Z 2025-12-04T09:41:15.9547607Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9547953Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9548271Z res = mod(**inputs) 2025-12-04T09:41:15.9548624Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9549008Z outputs = self.fnet( 2025-12-04T09:41:15.9549386Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9549752Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9550107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9550476Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9550826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9551168Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9551528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9551911Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9552289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9552694Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9553066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9553541Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9553699Z 2025-12-04T09:41:15.9553799Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9554151Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9554447Z res = mod(**inputs) 2025-12-04T09:41:15.9554783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9555141Z outputs = self.fnet( 2025-12-04T09:41:15.9555475Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9555847Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9556209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9556589Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9556927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9557270Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9557634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9558009Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9558396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9558766Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9559133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9559517Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9559668Z 2025-12-04T09:41:15.9559743Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9559963Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9560298Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9560593Z res = mod(**inputs) 2025-12-04T09:41:15.9560927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9561284Z outputs = self.fnet( 2025-12-04T09:41:15.9561610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9561973Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9562370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9562747Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9563089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9563437Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9563805Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:15.9564172Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:15.9564561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:15.9564943Z return forward_fn(*input_tensors) 2025-12-04T09:41:15.9565330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:15.9565810Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:15.9566208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:15.9566605Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:15.9566969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:15.9567397Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:15.9567628Z 2025-12-04T09:41:15.9567705Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9567908Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9568119Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9568463Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9568780Z res = mod(**inputs) 2025-12-04T09:41:15.9569123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9569472Z outputs = self.fnet( 2025-12-04T09:41:15.9569810Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9570173Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9570523Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9570902Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9571253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9571597Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9571956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9572350Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9572734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9573104Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9573460Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9573850Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9573993Z 2025-12-04T09:41:15.9574099Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9574429Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9574735Z res = mod(**inputs) 2025-12-04T09:41:15.9575067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9575428Z outputs = self.fnet( 2025-12-04T09:41:15.9575785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9576148Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9576502Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9576876Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9577213Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9577553Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9577916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9578294Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9578711Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9579076Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9579432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9579806Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9579956Z 2025-12-04T09:41:15.9580051Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9580384Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9580684Z res = mod(**inputs) 2025-12-04T09:41:15.9581007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9581357Z outputs = self.fnet( 2025-12-04T09:41:15.9581687Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9582042Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9582392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9582760Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9583101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9583433Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9583787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9584162Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9584522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9584892Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9585253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9585632Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9585773Z 2025-12-04T09:41:15.9585870Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9586204Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9586506Z res = mod(**inputs) 2025-12-04T09:41:15.9586833Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9587179Z outputs = self.fnet( 2025-12-04T09:41:15.9587510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9587864Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9588239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9588619Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9588967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9589314Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9589668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9590056Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9590438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9590803Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9591169Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9591592Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9591736Z 2025-12-04T09:41:15.9591820Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9592037Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9592374Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9592678Z res = mod(**inputs) 2025-12-04T09:41:15.9593025Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9593514Z outputs = self.fnet( 2025-12-04T09:41:15.9593876Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9594271Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9594612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9594987Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9595332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9595673Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9596023Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:15.9596393Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:15.9596767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:15.9597132Z return forward_fn(*input_tensors) 2025-12-04T09:41:15.9597516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:15.9597933Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:15.9598329Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:15.9598711Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:15.9599068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:15.9599489Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:15.9599702Z 2025-12-04T09:41:15.9599784Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9599977Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9600201Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9600536Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9600839Z res = mod(**inputs) 2025-12-04T09:41:15.9601171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9601561Z outputs = self.fnet( 2025-12-04T09:41:15.9601907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9602265Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9602621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9602995Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9603333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9603681Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9604045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9604468Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9604845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9605216Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9605583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9606113Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9606261Z 2025-12-04T09:41:15.9606362Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9606700Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9607005Z res = mod(**inputs) 2025-12-04T09:41:15.9607328Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9607678Z outputs = self.fnet( 2025-12-04T09:41:15.9608017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9608375Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9608715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9609079Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9609421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9609751Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9610105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9610480Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9610852Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9611207Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9611568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9611949Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9612091Z 2025-12-04T09:41:15.9612193Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9612517Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9612815Z res = mod(**inputs) 2025-12-04T09:41:15.9613144Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9613486Z outputs = self.fnet( 2025-12-04T09:41:15.9613817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9614178Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9615204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9615593Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9615944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9616286Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9616653Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9617033Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9617417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9617789Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9618148Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9618589Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9618740Z 2025-12-04T09:41:15.9618837Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9619171Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9619469Z res = mod(**inputs) 2025-12-04T09:41:15.9619803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9620157Z outputs = self.fnet( 2025-12-04T09:41:15.9620485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9620848Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9621203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9621582Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9621924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9622265Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9622628Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9623010Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9623382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9623753Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9624119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9624498Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9624653Z 2025-12-04T09:41:15.9624729Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9624953Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9625286Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9625585Z res = mod(**inputs) 2025-12-04T09:41:15.9625922Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9626276Z outputs = self.fnet( 2025-12-04T09:41:15.9626601Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9626959Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9627313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9627687Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9628059Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9628412Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9628779Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:15.9629151Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:15.9629532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:15.9629915Z return forward_fn(*input_tensors) 2025-12-04T09:41:15.9630305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:15.9630731Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:15.9631133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:15.9631579Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:15.9631971Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:15.9632409Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:15.9632644Z 2025-12-04T09:41:15.9632723Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9632933Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9633223Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9633571Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9633895Z res = mod(**inputs) 2025-12-04T09:41:15.9634265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9634650Z outputs = self.fnet( 2025-12-04T09:41:15.9635032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9635422Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9635806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9636215Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9636599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9636976Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9637365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9637783Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9638200Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9638605Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9638992Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9639420Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9639577Z 2025-12-04T09:41:15.9639691Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9640056Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9640378Z res = mod(**inputs) 2025-12-04T09:41:15.9640736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9641123Z outputs = self.fnet( 2025-12-04T09:41:15.9641482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9641878Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9642302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9642712Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9643081Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9643452Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9643845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9644258Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9644672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9645072Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9645509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9645904Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9646062Z 2025-12-04T09:41:15.9646163Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9646510Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9646830Z res = mod(**inputs) 2025-12-04T09:41:15.9647163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9647527Z outputs = self.fnet( 2025-12-04T09:41:15.9647873Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9648237Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9648602Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9648995Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9649355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9649701Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9650078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9650476Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9650860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9651240Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9651612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9652018Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9652167Z 2025-12-04T09:41:15.9652270Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9652618Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9652929Z res = mod(**inputs) 2025-12-04T09:41:15.9653271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9653630Z outputs = self.fnet( 2025-12-04T09:41:15.9653974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9654349Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9654707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9655094Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9655482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9655843Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9656210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9656610Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9657001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9657382Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9657752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9658154Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9658303Z 2025-12-04T09:41:15.9658389Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9658673Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9659017Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9659328Z res = mod(**inputs) 2025-12-04T09:41:15.9659671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9660031Z outputs = self.fnet( 2025-12-04T09:41:15.9660375Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9660745Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9661102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9661487Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9661849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9662205Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9662571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:15.9662957Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:15.9663350Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:15.9663734Z return forward_fn(*input_tensors) 2025-12-04T09:41:15.9664129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:15.9664574Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:15.9664969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:15.9665362Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:15.9665728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:15.9666159Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:15.9666380Z 2025-12-04T09:41:15.9666463Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9666658Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9666884Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9667224Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9667527Z res = mod(**inputs) 2025-12-04T09:41:15.9667867Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9668222Z outputs = self.fnet( 2025-12-04T09:41:15.9668574Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9668961Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9669310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9669676Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9670015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9670343Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9670698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9671076Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9671439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9671842Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9672216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9672611Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9672758Z 2025-12-04T09:41:15.9672859Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9673282Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9673609Z res = mod(**inputs) 2025-12-04T09:41:15.9673949Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9674322Z outputs = self.fnet( 2025-12-04T09:41:15.9674681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9675050Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9675410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9675835Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9676201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9676560Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9676924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9677308Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9677694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9678054Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9678424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9678821Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9678967Z 2025-12-04T09:41:15.9679070Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9679403Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9679708Z res = mod(**inputs) 2025-12-04T09:41:15.9680049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9680407Z outputs = self.fnet( 2025-12-04T09:41:15.9680738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9681107Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9681466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9681839Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9682243Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9682590Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9682959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9683343Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9683727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9684099Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9684459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9684849Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9685037Z 2025-12-04T09:41:15.9685136Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9685478Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9685778Z res = mod(**inputs) 2025-12-04T09:41:15.9686117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9686470Z outputs = self.fnet( 2025-12-04T09:41:15.9702068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9702480Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9702852Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9703240Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9703598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9703967Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9704329Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9704719Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9705103Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9705465Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9706024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9706424Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9706578Z 2025-12-04T09:41:15.9706672Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9706897Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9707251Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9707573Z res = mod(**inputs) 2025-12-04T09:41:15.9707908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9708292Z outputs = self.fnet( 2025-12-04T09:41:15.9708642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9709020Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9709375Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9709760Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9710118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9710474Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9710975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:15.9711364Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:15.9711760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:15.9712144Z return forward_fn(*input_tensors) 2025-12-04T09:41:15.9712550Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:15.9712997Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:15.9713487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:15.9713907Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:15.9714371Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:15.9714882Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:15.9715107Z 2025-12-04T09:41:15.9715194Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9715391Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9715621Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9715969Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9716282Z res = mod(**inputs) 2025-12-04T09:41:15.9716637Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9717005Z outputs = self.fnet( 2025-12-04T09:41:15.9717356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9717725Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9718096Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9718479Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9718825Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9719175Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9719547Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9719943Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9720324Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9720703Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9721083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9721480Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9721629Z 2025-12-04T09:41:15.9721730Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9722082Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9722396Z res = mod(**inputs) 2025-12-04T09:41:15.9722732Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9723094Z outputs = self.fnet( 2025-12-04T09:41:15.9723435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9723799Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9724153Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9724569Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9724928Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9725285Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9725673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9726059Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9726438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9726799Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9727163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9727599Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9727744Z 2025-12-04T09:41:15.9727852Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9728186Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9728492Z res = mod(**inputs) 2025-12-04T09:41:15.9728827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9729183Z outputs = self.fnet( 2025-12-04T09:41:15.9729509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9729874Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9730225Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9730604Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9730956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9731303Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9731665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9732042Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9732422Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9732791Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9733147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9733534Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9733684Z 2025-12-04T09:41:15.9733781Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9734126Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9734428Z res = mod(**inputs) 2025-12-04T09:41:15.9734763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9735124Z outputs = self.fnet( 2025-12-04T09:41:15.9735463Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9735824Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9736179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9736556Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9736895Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9737240Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9737640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9738029Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9738399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9738767Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9739131Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9739518Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9739662Z 2025-12-04T09:41:15.9739739Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9739962Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9740299Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9740637Z res = mod(**inputs) 2025-12-04T09:41:15.9740976Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9741341Z outputs = self.fnet( 2025-12-04T09:41:15.9741675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9742031Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9742384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9742763Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9743101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9743446Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9743812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:15.9744194Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:15.9744575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:15.9744959Z return forward_fn(*input_tensors) 2025-12-04T09:41:15.9745347Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:15.9745776Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:15.9746167Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:15.9746560Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:15.9746924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:15.9747359Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:15.9747590Z 2025-12-04T09:41:15.9747665Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9747864Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9748083Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9748416Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9748722Z res = mod(**inputs) 2025-12-04T09:41:15.9749061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9749413Z outputs = self.fnet( 2025-12-04T09:41:15.9749754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9750114Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9750508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9750881Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9751226Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9751574Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9751944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9752330Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9752713Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9753151Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9753538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9753970Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9754129Z 2025-12-04T09:41:15.9754231Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9754589Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9754885Z res = mod(**inputs) 2025-12-04T09:41:15.9755230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9755596Z outputs = self.fnet( 2025-12-04T09:41:15.9755932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9756300Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9756662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9757040Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9757386Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9757731Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9758098Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9758485Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9758859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9759226Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9759594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9759976Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9760128Z 2025-12-04T09:41:15.9760230Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9760580Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9760888Z res = mod(**inputs) 2025-12-04T09:41:15.9761217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9761575Z outputs = self.fnet( 2025-12-04T09:41:15.9761912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9762269Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9762625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9762997Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9763345Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9763687Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9764085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9764483Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9764873Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9765246Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9765620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9766019Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9766170Z 2025-12-04T09:41:15.9766271Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9766623Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9766974Z res = mod(**inputs) 2025-12-04T09:41:15.9767318Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9767679Z outputs = self.fnet( 2025-12-04T09:41:15.9768034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9768401Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9768756Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9769122Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9769472Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9769818Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9770174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9770565Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9770947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9771313Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9771669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9772060Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9772204Z 2025-12-04T09:41:15.9772287Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9772503Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9772845Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9773148Z res = mod(**inputs) 2025-12-04T09:41:15.9773488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9773846Z outputs = self.fnet( 2025-12-04T09:41:15.9774090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9774160Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9774395Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9774483Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9774694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9774775Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9775007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:15.9775088Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:15.9775385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:15.9775467Z return forward_fn(*input_tensors) 2025-12-04T09:41:15.9775730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:15.9775845Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:15.9776079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:15.9776182Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:15.9776392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:15.9776563Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:15.9776596Z 2025-12-04T09:41:15.9776681Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9776755Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9776854Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9777054Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9777114Z res = mod(**inputs) 2025-12-04T09:41:15.9777352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9777421Z outputs = self.fnet( 2025-12-04T09:41:15.9777655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9777732Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9777968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9778049Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9778270Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9778343Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9778576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9778674Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9778908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9778990Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9779222Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9779318Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9779324Z 2025-12-04T09:41:15.9779429Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9779618Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9779684Z res = mod(**inputs) 2025-12-04T09:41:15.9779917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9779978Z outputs = self.fnet( 2025-12-04T09:41:15.9780218Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9780286Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9780517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9780602Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9780809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9780919Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9781153Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9781241Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9781481Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9781555Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9781794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9781890Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9781893Z 2025-12-04T09:41:15.9781988Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9782217Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9782281Z res = mod(**inputs) 2025-12-04T09:41:15.9782515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9782584Z outputs = self.fnet( 2025-12-04T09:41:15.9782819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9782892Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9783126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9783205Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9783421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9783493Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9783734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9783830Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9784068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9784150Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9784387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9784480Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9784483Z 2025-12-04T09:41:15.9784588Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9784773Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9784841Z res = mod(**inputs) 2025-12-04T09:41:15.9785080Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9785145Z outputs = self.fnet( 2025-12-04T09:41:15.9785387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9785454Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9785692Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9785780Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9785990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9786069Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9786304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9786394Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9786668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9786744Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9786988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9787083Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9787087Z 2025-12-04T09:41:15.9787161Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9787265Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9787451Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9787510Z res = mod(**inputs) 2025-12-04T09:41:15.9787754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9787845Z outputs = self.fnet( 2025-12-04T09:41:15.9788084Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9788151Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9788382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9788469Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9788675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9788746Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9788983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:15.9789061Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:15.9789318Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:15.9789396Z return forward_fn(*input_tensors) 2025-12-04T09:41:15.9789660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:15.9789773Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:15.9790012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:15.9790123Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:15.9790327Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:15.9790497Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:15.9790501Z 2025-12-04T09:41:15.9790585Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9790659Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9790765Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9790950Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9791012Z res = mod(**inputs) 2025-12-04T09:41:15.9791256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9791319Z outputs = self.fnet( 2025-12-04T09:41:15.9791550Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9791627Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9791858Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9791944Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9792185Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9792259Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9792497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9792587Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9792849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9792935Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9793247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9793360Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9793364Z 2025-12-04T09:41:15.9793467Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9793713Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9793786Z res = mod(**inputs) 2025-12-04T09:41:15.9794031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9794104Z outputs = self.fnet( 2025-12-04T09:41:15.9794358Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9794425Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9794674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9794755Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9794975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9795062Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9795311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9795410Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9795656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9795734Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9795983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9796080Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9796084Z 2025-12-04T09:41:15.9796189Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9796381Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9796448Z res = mod(**inputs) 2025-12-04T09:41:15.9796703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9796769Z outputs = self.fnet( 2025-12-04T09:41:15.9797015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9797094Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9797340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9797429Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9797648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9797722Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9797974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9798115Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9798363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9798446Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9798690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9798795Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9798798Z 2025-12-04T09:41:15.9798896Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9799094Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9799165Z res = mod(**inputs) 2025-12-04T09:41:15.9799415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9799519Z outputs = self.fnet( 2025-12-04T09:41:15.9799830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9799902Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9800160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9800241Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9800463Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9800545Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9800797Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9800896Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9801147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9801226Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9801480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9801577Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9801581Z 2025-12-04T09:41:15.9801666Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9801766Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9801966Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9802036Z res = mod(**inputs) 2025-12-04T09:41:15.9802283Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9802348Z outputs = self.fnet( 2025-12-04T09:41:15.9802607Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9802678Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9802932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9803015Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9803235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9803318Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9803565Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:15.9803647Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:15.9803916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:15.9803995Z return forward_fn(*input_tensors) 2025-12-04T09:41:15.9804311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:15.9804423Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:15.9804661Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:15.9804776Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:15.9804986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:15.9805168Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:15.9805171Z 2025-12-04T09:41:15.9805248Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9805330Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9805463Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9805648Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9805881Z res = mod(**inputs) 2025-12-04T09:41:15.9806129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9806192Z outputs = self.fnet( 2025-12-04T09:41:15.9806435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9806503Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9806735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9806823Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9807033Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9807118Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9807354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9807446Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9807686Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9807760Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9807993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9808098Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9808101Z 2025-12-04T09:41:15.9808198Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9808396Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9808461Z res = mod(**inputs) 2025-12-04T09:41:15.9808699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9808770Z outputs = self.fnet( 2025-12-04T09:41:15.9809004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9809077Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9809310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9809388Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9809604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9809675Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9809911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9810064Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9810302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9810382Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9810616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9810711Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9810714Z 2025-12-04T09:41:15.9810816Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9811000Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9811064Z res = mod(**inputs) 2025-12-04T09:41:15.9811302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9811413Z outputs = self.fnet( 2025-12-04T09:41:15.9811654Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9811722Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9811953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9812041Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9812250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9812329Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9812561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9812653Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9812897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9812972Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9813201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9813304Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9813307Z 2025-12-04T09:41:15.9813401Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9813596Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9813655Z res = mod(**inputs) 2025-12-04T09:41:15.9813888Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9813958Z outputs = self.fnet( 2025-12-04T09:41:15.9814193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9814269Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9814501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9814579Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9814792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9814864Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9815101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:15.9815196Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:15.9815427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:15.9815510Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:15.9815798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:15.9815893Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:15.9815896Z 2025-12-04T09:41:15.9815978Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9816077Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9816261Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9816328Z res = mod(**inputs) 2025-12-04T09:41:15.9816560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:15.9816626Z outputs = self.fnet( 2025-12-04T09:41:15.9816861Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:15.9816965Z encoder_outputs = self.encoder( 2025-12-04T09:41:15.9817212Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:15.9817290Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:15.9817508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:15.9817580Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:15.9817815Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:15.9817899Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:15.9818149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:15.9818221Z return forward_fn(*input_tensors) 2025-12-04T09:41:15.9818500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:15.9818605Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:15.9818845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:15.9818944Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:15.9819146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:15.9819324Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:15.9819328Z 2025-12-04T09:41:15.9819401Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9819479Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9819547Z cudagraph partition due to non gpu ops 2025-12-04T09:41:15.9819644Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:15.9819844Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:15.9819904Z res = mod(**inputs) 2025-12-04T09:41:15.9820140Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 685, in forward 2025-12-04T09:41:15.9820323Z masked_lm_loss = loss_fct(prediction_scores.view(-1, self.config.vocab_size), labels.view(-1)) 2025-12-04T09:41:15.9820327Z 2025-12-04T09:41:25.6251719Z Compilation time (from dynamo_timed): 14.372665826 2025-12-04T09:41:25.6338750Z pass 2025-12-04T09:41:25.6340810Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:41:25.6341683Z TIMING: _recursive_pre_grad_passes:0.02016 _recursive_joint_graph_passes:0.20112 _recursive_post_grad_passes:0.05453 async_compile.wait:0.76037 code_gen:9.12311 inductor_compile:10.49057 backend_compile:12.54392 gc:0.00099 entire_frame_compile:14.37267 total_wall_time:14.37267 2025-12-04T09:41:25.6346796Z STATS: call_* op count: 232 | FakeTensorMode.__torch_dispatch__:11067 | FakeTensor.__torch_dispatch__:2950 | ProxyTorchDispatchMode.__torch_dispatch__:1323 2025-12-04T09:41:25.6347906Z Dynamo produced 1 graphs covering 232 ops with 0 graph breaks (0 unique) 2025-12-04T09:41:27.8291864Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:41:27.8292645Z import pynvml # type: ignore[import] 2025-12-04T09:41:31.0193446Z 2025-12-04T09:41:32.0110176Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:41:32.0112127Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:41:32.0119680Z cpu eval LayoutLMForMaskedLM 2025-12-04T09:41:32.5987105Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:41:32.8353851Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:41:33.0715960Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:41:42.2096857Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2101986Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2102725Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2102945Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2103189Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2103388Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2103648Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2103851Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2104077Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2104270Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2104524Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:42.2104909Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:42.2105234Z res = mod(**inputs) 2025-12-04T09:41:42.2105570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2106098Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2106560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:41:42.2106968Z outputs = self.layoutlm( 2025-12-04T09:41:42.2107302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2107664Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2108064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:41:42.2108468Z encoder_outputs = self.encoder( 2025-12-04T09:41:42.2108817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2109166Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2109559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:41:42.2109947Z layer_outputs = layer_module( 2025-12-04T09:41:42.2110296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:42.2110657Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:42.2111067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:41:42.2111472Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:42.2112236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:42.2112642Z return forward_fn(*input_tensors) 2025-12-04T09:41:42.2113065Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:41:42.2113670Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:41:42.2114126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:41:42.2114573Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:42.2114961Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:41:42.2115308Z return self.act(input) 2025-12-04T09:41:42.2115423Z 2025-12-04T09:41:42.2115512Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2115829Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2116028Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2116228Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2116426Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2116618Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2116816Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2117015Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2117206Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2117406Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2117605Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2117827Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:42.2118187Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:42.2118515Z res = mod(**inputs) 2025-12-04T09:41:42.2118840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2119195Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2119589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:41:42.2119988Z outputs = self.layoutlm( 2025-12-04T09:41:42.2120308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2120658Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2121052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:41:42.2121454Z encoder_outputs = self.encoder( 2025-12-04T09:41:42.2121787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2122135Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2122531Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:41:42.2122923Z layer_outputs = layer_module( 2025-12-04T09:41:42.2123263Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:42.2123624Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:42.2124017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:41:42.2124420Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:42.2124817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:42.2125213Z return forward_fn(*input_tensors) 2025-12-04T09:41:42.2125634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:41:42.2126145Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:41:42.2126590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:41:42.2127009Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:42.2127376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:41:42.2127698Z return self.act(input) 2025-12-04T09:41:42.2127813Z 2025-12-04T09:41:42.2127888Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2128085Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2128269Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2128459Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2128654Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2128842Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2129070Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2129261Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2129450Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2129633Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2129826Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2130047Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:42.2130382Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:42.2130690Z res = mod(**inputs) 2025-12-04T09:41:42.2131000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2131323Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2131700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:41:42.2132077Z outputs = self.layoutlm( 2025-12-04T09:41:42.2132402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2132725Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2133147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:41:42.2133532Z encoder_outputs = self.encoder( 2025-12-04T09:41:42.2133857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2134199Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2134666Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:41:42.2135051Z layer_outputs = layer_module( 2025-12-04T09:41:42.2135378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:42.2135730Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:42.2136125Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:41:42.2136535Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:42.2136931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:42.2137308Z return forward_fn(*input_tensors) 2025-12-04T09:41:42.2137710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:41:42.2138169Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:41:42.2138600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:41:42.2139018Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:42.2139415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:41:42.2139744Z return self.act(input) 2025-12-04T09:41:42.2139848Z 2025-12-04T09:41:42.2139930Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2140119Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2140315Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2140509Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2140699Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2140882Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2141072Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2141259Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2141443Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2141632Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2141821Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2142091Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:42.2142440Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:42.2142754Z res = mod(**inputs) 2025-12-04T09:41:42.2143059Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2143399Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2143780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:41:42.2144163Z outputs = self.layoutlm( 2025-12-04T09:41:42.2144479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2144824Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2145207Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:41:42.2145602Z encoder_outputs = self.encoder( 2025-12-04T09:41:42.2145931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2146275Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2146658Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:41:42.2147045Z layer_outputs = layer_module( 2025-12-04T09:41:42.2147382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:42.2147733Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:42.2148121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:41:42.2148510Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:42.2148900Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:42.2149288Z return forward_fn(*input_tensors) 2025-12-04T09:41:42.2149694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:41:42.2150157Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:41:42.2150586Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:41:42.2151008Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:42.2151367Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:41:42.2151695Z return self.act(input) 2025-12-04T09:41:42.2151806Z 2025-12-04T09:41:42.2151882Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2152085Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2152272Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2152497Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2152688Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2152872Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2153065Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2153343Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2153530Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2153729Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2153930Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2154151Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:42.2154506Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:42.2154824Z res = mod(**inputs) 2025-12-04T09:41:42.2155149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2155525Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2155926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:41:42.2156339Z outputs = self.layoutlm( 2025-12-04T09:41:42.2156674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2157016Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2157404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:41:42.2157796Z encoder_outputs = self.encoder( 2025-12-04T09:41:42.2158128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2158476Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2158869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:41:42.2159257Z layer_outputs = layer_module( 2025-12-04T09:41:42.2159591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:42.2159944Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:42.2160339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:41:42.2160737Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:42.2161141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:42.2161526Z return forward_fn(*input_tensors) 2025-12-04T09:41:42.2161947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:41:42.2162411Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:41:42.2162855Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:41:42.2163287Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:42.2163665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:41:42.2163997Z return self.act(input) 2025-12-04T09:41:42.2164108Z 2025-12-04T09:41:42.2164183Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2164386Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2164578Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2164775Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2164968Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2165154Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2165352Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2165548Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2165774Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2165967Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2166170Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2166399Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:42.2166747Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:42.2167072Z res = mod(**inputs) 2025-12-04T09:41:42.2167399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2167746Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2168143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:41:42.2168545Z outputs = self.layoutlm( 2025-12-04T09:41:42.2168910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2169251Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2169647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:41:42.2170028Z encoder_outputs = self.encoder( 2025-12-04T09:41:42.2170356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2170683Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2171060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:41:42.2171437Z layer_outputs = layer_module( 2025-12-04T09:41:42.2171764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:42.2172111Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:42.2172492Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:41:42.2172878Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:42.2173254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:42.2173629Z return forward_fn(*input_tensors) 2025-12-04T09:41:42.2174038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:41:42.2174485Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:41:42.2174897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:41:42.2175311Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:42.2175672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:41:42.2175989Z return self.act(input) 2025-12-04T09:41:42.2176100Z 2025-12-04T09:41:42.2176174Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2176417Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2176610Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2176799Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2176983Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2177175Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2177363Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2177551Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2177732Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2177923Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2178115Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2178329Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:42.2178702Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:42.2179027Z res = mod(**inputs) 2025-12-04T09:41:42.2179335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2179674Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2180055Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:41:42.2180442Z outputs = self.layoutlm( 2025-12-04T09:41:42.2180758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2181099Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2181485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:41:42.2182108Z encoder_outputs = self.encoder( 2025-12-04T09:41:42.2182442Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2182785Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2183174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:41:42.2183552Z layer_outputs = layer_module( 2025-12-04T09:41:42.2183887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:42.2184233Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:42.2184624Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:41:42.2185017Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:42.2185415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:42.2185797Z return forward_fn(*input_tensors) 2025-12-04T09:41:42.2186203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:41:42.2186663Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:41:42.2187091Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:41:42.2187508Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:42.2187864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:41:42.2188191Z return self.act(input) 2025-12-04T09:41:42.2188300Z 2025-12-04T09:41:42.2188374Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2188575Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2188759Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2188952Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2189143Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2189326Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2189517Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2189707Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2189891Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2190085Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2190274Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2190485Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:42.2190828Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:42.2191138Z res = mod(**inputs) 2025-12-04T09:41:42.2191455Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2191791Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2192226Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:41:42.2192612Z outputs = self.layoutlm( 2025-12-04T09:41:42.2192927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2193376Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2193778Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:41:42.2194178Z encoder_outputs = self.encoder( 2025-12-04T09:41:42.2194509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2194853Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2195286Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:41:42.2195677Z layer_outputs = layer_module( 2025-12-04T09:41:42.2196023Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:42.2196392Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:42.2196795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:41:42.2197204Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:42.2197608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:42.2198003Z return forward_fn(*input_tensors) 2025-12-04T09:41:42.2198430Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:41:42.2198907Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:41:42.2199355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:41:42.2199799Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:42.2200179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:41:42.2200522Z return self.act(input) 2025-12-04T09:41:42.2200637Z 2025-12-04T09:41:42.2200715Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2200923Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2201119Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2201322Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2201525Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2201719Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2201919Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2202121Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2202321Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2202512Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2202712Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2202938Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:42.2203293Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:42.2203620Z res = mod(**inputs) 2025-12-04T09:41:42.2203949Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2204298Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2204695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:41:42.2205097Z outputs = self.layoutlm( 2025-12-04T09:41:42.2205456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2205959Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2206358Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:41:42.2206746Z encoder_outputs = self.encoder( 2025-12-04T09:41:42.2207086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2207419Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2207803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:41:42.2208186Z layer_outputs = layer_module( 2025-12-04T09:41:42.2208516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:42.2208945Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:42.2209336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:41:42.2209744Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:42.2210124Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:42.2210507Z return forward_fn(*input_tensors) 2025-12-04T09:41:42.2210919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:41:42.2211371Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:41:42.2211791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:41:42.2212213Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:42.2212583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:41:42.2212900Z return self.act(input) 2025-12-04T09:41:42.2213013Z 2025-12-04T09:41:42.2213086Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2213283Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2213472Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2213656Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2213846Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2214032Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2214216Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2214405Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2214596Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2214778Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2214973Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2215201Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:42.2215545Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:42.2215860Z res = mod(**inputs) 2025-12-04T09:41:42.2216170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2216515Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2216890Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:41:42.2217272Z outputs = self.layoutlm( 2025-12-04T09:41:42.2217594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2217922Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2218297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:41:42.2218682Z encoder_outputs = self.encoder( 2025-12-04T09:41:42.2219064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2219396Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2219777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:41:42.2220157Z layer_outputs = layer_module( 2025-12-04T09:41:42.2220490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:42.2220825Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:42.2221212Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:41:42.2221605Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:42.2222006Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:42.2222383Z return forward_fn(*input_tensors) 2025-12-04T09:41:42.2222787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:41:42.2223233Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:41:42.2223643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:41:42.2224055Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:42.2224409Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:41:42.2224730Z return self.act(input) 2025-12-04T09:41:42.2224835Z 2025-12-04T09:41:42.2224909Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2225108Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2225305Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2225491Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2225682Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2225873Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2226059Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2226250Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2226440Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2226627Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2226812Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2227027Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:42.2227366Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:42.2227661Z res = mod(**inputs) 2025-12-04T09:41:42.2227969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2228306Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2228678Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:41:42.2229057Z outputs = self.layoutlm( 2025-12-04T09:41:42.2229375Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2229708Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2230074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:41:42.2230455Z encoder_outputs = self.encoder( 2025-12-04T09:41:42.2230780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2231109Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2231474Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:41:42.2231881Z layer_outputs = layer_module( 2025-12-04T09:41:42.2232212Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:42.2232551Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:42.2232936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:41:42.2233400Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:42.2233796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:42.2234173Z return forward_fn(*input_tensors) 2025-12-04T09:41:42.2234588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:41:42.2235106Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:41:42.2235531Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:41:42.2235950Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:42.2236324Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:41:42.2236660Z return self.act(input) 2025-12-04T09:41:42.2236766Z 2025-12-04T09:41:42.2236842Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2237043Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2237239Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2237439Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2237625Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2237820Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2238018Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2238208Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2238406Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2238605Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2238794Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2239018Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:42.2239365Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:42.2239675Z res = mod(**inputs) 2025-12-04T09:41:42.2239993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2240340Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2240728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:41:42.2241110Z outputs = self.layoutlm( 2025-12-04T09:41:42.2241441Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2241790Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2242172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:41:42.2242565Z encoder_outputs = self.encoder( 2025-12-04T09:41:42.2242906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2243251Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2243629Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:41:42.2244019Z layer_outputs = layer_module( 2025-12-04T09:41:42.2244371Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:42.2244728Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:42.2245189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:41:42.2245598Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:42.2245991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:42.2246372Z return forward_fn(*input_tensors) 2025-12-04T09:41:42.2246791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:41:42.2247253Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:41:42.2247690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:41:42.2248118Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:42.2248517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:41:42.2248842Z return self.act(input) 2025-12-04T09:41:42.2248945Z 2025-12-04T09:41:42.2249023Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2249212Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2249411Z cudagraph partition due to non gpu ops 2025-12-04T09:41:42.2249629Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:42.2249964Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:42.2250271Z res = mod(**inputs) 2025-12-04T09:41:42.2250579Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:41:42.2250918Z output = func(self, *args, **kwargs) 2025-12-04T09:41:42.2251291Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 740, in forward 2025-12-04T09:41:42.2251676Z masked_lm_loss = loss_fct( 2025-12-04T09:41:42.2251787Z 2025-12-04T09:41:51.5515369Z Compilation time (from dynamo_timed): 17.859035477 2025-12-04T09:41:51.5583666Z pass 2025-12-04T09:41:51.5586328Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:41:51.5587215Z TIMING: _recursive_pre_grad_passes:0.03397 _recursive_joint_graph_passes:0.42888 _recursive_post_grad_passes:0.05439 async_compile.wait:0.67053 code_gen:8.94874 inductor_compile:10.92689 backend_compile:15.0169 gc:0.00042 entire_frame_compile:17.85904 total_wall_time:17.85904 2025-12-04T09:41:51.5589044Z STATS: call_* op count: 432 | FakeTensorMode.__torch_dispatch__:20793 | FakeTensor.__torch_dispatch__:3620 | ProxyTorchDispatchMode.__torch_dispatch__:3441 2025-12-04T09:41:51.5589522Z Dynamo produced 1 graphs covering 432 ops with 0 graph breaks (0 unique) 2025-12-04T09:41:53.8686942Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:41:53.8687925Z import pynvml # type: ignore[import] 2025-12-04T09:41:56.9692084Z 2025-12-04T09:42:02.2722460Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:42:02.2727909Z loading model: 0it [00:05, ?it/s] 2025-12-04T09:42:02.2747645Z cpu eval M2M100ForConditionalGeneration 2025-12-04T09:42:03.0315795Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:42:03.3242443Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:42:03.6719547Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:42:22.2514632Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2521714Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2522118Z res = mod(**inputs) 2025-12-04T09:42:22.2522518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2522901Z outputs = self.model( 2025-12-04T09:42:22.2523275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2523661Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2524044Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 849, in forward 2025-12-04T09:42:22.2524459Z embed_pos = self.embed_positions(input_ids, inputs_embeds) 2025-12-04T09:42:22.2524858Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context 2025-12-04T09:42:22.2525327Z return func(*args, **kwargs) 2025-12-04T09:42:22.2525697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 149, in forward 2025-12-04T09:42:22.2526201Z position_ids = create_position_ids_from_input_ids(input_ids, self.padding_idx, past_key_values_length).to( 2025-12-04T09:42:22.2526763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 81, in create_position_ids_from_input_ids 2025-12-04T09:42:22.2527208Z mask = input_ids.ne(padding_idx).int() 2025-12-04T09:42:22.2527344Z 2025-12-04T09:42:22.2527440Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2527643Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2527844Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2528042Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2528231Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2528431Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2528630Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2528818Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2529015Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2529216Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2529400Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2529593Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2529816Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2530165Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2530474Z res = mod(**inputs) 2025-12-04T09:42:22.2530824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2531197Z outputs = self.model( 2025-12-04T09:42:22.2531541Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2531928Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2532297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 849, in forward 2025-12-04T09:42:22.2532711Z embed_pos = self.embed_positions(input_ids, inputs_embeds) 2025-12-04T09:42:22.2533096Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context 2025-12-04T09:42:22.2533446Z return func(*args, **kwargs) 2025-12-04T09:42:22.2533807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 149, in forward 2025-12-04T09:42:22.2534321Z position_ids = create_position_ids_from_input_ids(input_ids, self.padding_idx, past_key_values_length).to( 2025-12-04T09:42:22.2534888Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 82, in create_position_ids_from_input_ids 2025-12-04T09:42:22.2535496Z incremental_indices = (torch.cumsum(mask, dim=1).type_as(mask) + past_key_values_length) * mask 2025-12-04T09:42:22.2535725Z 2025-12-04T09:42:22.2535835Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2536183Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2536496Z res = mod(**inputs) 2025-12-04T09:42:22.2536856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2537233Z outputs = self.model( 2025-12-04T09:42:22.2537590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2537970Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2538346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 849, in forward 2025-12-04T09:42:22.2538813Z embed_pos = self.embed_positions(input_ids, inputs_embeds) 2025-12-04T09:42:22.2539204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context 2025-12-04T09:42:22.2539578Z return func(*args, **kwargs) 2025-12-04T09:42:22.2539947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 149, in forward 2025-12-04T09:42:22.2540454Z position_ids = create_position_ids_from_input_ids(input_ids, self.padding_idx, past_key_values_length).to( 2025-12-04T09:42:22.2541158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 82, in create_position_ids_from_input_ids 2025-12-04T09:42:22.2541696Z incremental_indices = (torch.cumsum(mask, dim=1).type_as(mask) + past_key_values_length) * mask 2025-12-04T09:42:22.2541929Z 2025-12-04T09:42:22.2542020Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2542220Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2542425Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2542623Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2542822Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2543011Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2543206Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2543430Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2543773Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2544090Z res = mod(**inputs) 2025-12-04T09:42:22.2544446Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2544805Z outputs = self.model( 2025-12-04T09:42:22.2545156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2545533Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2545901Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2546268Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2546611Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2546964Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2547339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:42:22.2547729Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:42:22.2548114Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2548481Z return func(*args, **kwargs) 2025-12-04T09:42:22.2548883Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2549295Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2549737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2550212Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2550393Z 2025-12-04T09:42:22.2550469Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2550672Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2550896Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2551243Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2551558Z res = mod(**inputs) 2025-12-04T09:42:22.2551953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2552329Z outputs = self.model( 2025-12-04T09:42:22.2552678Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2553199Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2553719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2554120Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2554484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2554865Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2555250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:42:22.2555693Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2555881Z 2025-12-04T09:42:22.2555965Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2556184Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2556395Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2556598Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2556810Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2557015Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2557213Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2557423Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2557663Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2558024Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2558357Z res = mod(**inputs) 2025-12-04T09:42:22.2558730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2559133Z outputs = self.model( 2025-12-04T09:42:22.2559508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2559921Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2560314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2560714Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2561070Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2561447Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2561846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:42:22.2562259Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:42:22.2562718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2563112Z return func(*args, **kwargs) 2025-12-04T09:42:22.2563497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2563916Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2564378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2564891Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2565083Z 2025-12-04T09:42:22.2565166Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2565362Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2565585Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2565927Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2566293Z res = mod(**inputs) 2025-12-04T09:42:22.2566657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2567033Z outputs = self.model( 2025-12-04T09:42:22.2567392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2567765Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2568136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2568515Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2568912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2569373Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2569759Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:42:22.2570180Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2570347Z 2025-12-04T09:42:22.2570425Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2570631Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2570834Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2571022Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2571217Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2571415Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2571601Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2571796Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2572019Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2572371Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2572714Z res = mod(**inputs) 2025-12-04T09:42:22.2573078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2573456Z outputs = self.model( 2025-12-04T09:42:22.2573807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2574187Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2574556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2574932Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2575266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2575625Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2576004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:42:22.2576444Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:42:22.2576818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2577178Z return func(*args, **kwargs) 2025-12-04T09:42:22.2577542Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2577930Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2578364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2578823Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2579000Z 2025-12-04T09:42:22.2579087Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2579285Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2579557Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2579905Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2580211Z res = mod(**inputs) 2025-12-04T09:42:22.2580568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2580946Z outputs = self.model( 2025-12-04T09:42:22.2581317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2581680Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2582044Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2582409Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2582744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2583090Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2583463Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:42:22.2583876Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2584041Z 2025-12-04T09:42:22.2584114Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2584310Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2584503Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2584692Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2584875Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2585063Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2585253Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2585436Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2585651Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2585996Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2586300Z res = mod(**inputs) 2025-12-04T09:42:22.2586647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2587018Z outputs = self.model( 2025-12-04T09:42:22.2587373Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2587749Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2588119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2588499Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2588838Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2589200Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2589616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:42:22.2590013Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:42:22.2590396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2590769Z return func(*args, **kwargs) 2025-12-04T09:42:22.2591141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2591541Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2591969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2592438Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2592646Z 2025-12-04T09:42:22.2592730Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2592931Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2593256Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2593631Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2593965Z res = mod(**inputs) 2025-12-04T09:42:22.2594337Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2594738Z outputs = self.model( 2025-12-04T09:42:22.2595097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2595468Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2595841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2596233Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2596583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2596933Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2597315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:42:22.2597741Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2597909Z 2025-12-04T09:42:22.2597991Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2598203Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2598399Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2598595Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2598784Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2598980Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2599178Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2599367Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2599595Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2599942Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2600258Z res = mod(**inputs) 2025-12-04T09:42:22.2600611Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2600994Z outputs = self.model( 2025-12-04T09:42:22.2601349Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2601723Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2602099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2602486Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2602872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2603221Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2603780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:42:22.2604190Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:42:22.2604573Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2604946Z return func(*args, **kwargs) 2025-12-04T09:42:22.2605315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2605846Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2606305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2606895Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2607083Z 2025-12-04T09:42:22.2607160Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2607372Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2607600Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2607966Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2608297Z res = mod(**inputs) 2025-12-04T09:42:22.2608647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2609024Z outputs = self.model( 2025-12-04T09:42:22.2609383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2609768Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2610135Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2610515Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2610854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2611206Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2611589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:42:22.2612009Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2612175Z 2025-12-04T09:42:22.2612255Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2612449Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2612646Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2612840Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2613032Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2613229Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2613426Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2613619Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2613835Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2614183Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2614501Z res = mod(**inputs) 2025-12-04T09:42:22.2614852Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2615229Z outputs = self.model( 2025-12-04T09:42:22.2615591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2615975Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2616352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2616787Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2617123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2617463Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2617834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:42:22.2618215Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:42:22.2618589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2618947Z return func(*args, **kwargs) 2025-12-04T09:42:22.2619303Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2620585Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2621016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2621479Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2621663Z 2025-12-04T09:42:22.2621740Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2621941Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2622161Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2622508Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2622824Z res = mod(**inputs) 2025-12-04T09:42:22.2623173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2623553Z outputs = self.model( 2025-12-04T09:42:22.2623915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2624291Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2624652Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2625031Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2625371Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2625724Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2626090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:42:22.2626511Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2626681Z 2025-12-04T09:42:22.2626763Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2626957Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2627156Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2627354Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2627550Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2627739Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2627934Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2628126Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2628343Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2628689Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2629003Z res = mod(**inputs) 2025-12-04T09:42:22.2629346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2629723Z outputs = self.model( 2025-12-04T09:42:22.2630077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2630466Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2630871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2631242Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2631578Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2631914Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2632282Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:42:22.2632666Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:42:22.2633046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2633500Z return func(*args, **kwargs) 2025-12-04T09:42:22.2633954Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2634374Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2634828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2635283Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2635467Z 2025-12-04T09:42:22.2635542Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2635747Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2635975Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2636332Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2636653Z res = mod(**inputs) 2025-12-04T09:42:22.2637016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2637404Z outputs = self.model( 2025-12-04T09:42:22.2637773Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2638165Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2638539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2638926Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2639280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2639642Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2640024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:42:22.2640457Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2640628Z 2025-12-04T09:42:22.2640716Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2640921Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2641116Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2641316Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2641514Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2641707Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2641906Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2642107Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2642330Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2642688Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2643007Z res = mod(**inputs) 2025-12-04T09:42:22.2643370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2643757Z outputs = self.model( 2025-12-04T09:42:22.2644165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2644560Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2644937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2645324Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2645656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2646004Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2646368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:42:22.2646754Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:42:22.2647132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2647527Z return func(*args, **kwargs) 2025-12-04T09:42:22.2647889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2648280Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2648704Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2649152Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2649331Z 2025-12-04T09:42:22.2649403Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2649601Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2649819Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2650151Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2650454Z res = mod(**inputs) 2025-12-04T09:42:22.2650803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2651172Z outputs = self.model( 2025-12-04T09:42:22.2651522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2651891Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2652251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2652611Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2652944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2653289Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2653652Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:42:22.2654068Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2654237Z 2025-12-04T09:42:22.2654312Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2654507Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2654690Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2654879Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2655069Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2655249Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2655438Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2655626Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2655833Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2656168Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2656472Z res = mod(**inputs) 2025-12-04T09:42:22.2656856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2657223Z outputs = self.model( 2025-12-04T09:42:22.2657569Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2657941Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2658306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2658665Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2658999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2659343Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2659706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:42:22.2660133Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:42:22.2660515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2660879Z return func(*args, **kwargs) 2025-12-04T09:42:22.2661235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2661630Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2662061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2662515Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2662688Z 2025-12-04T09:42:22.2662761Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2662956Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2663178Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2663512Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2663817Z res = mod(**inputs) 2025-12-04T09:42:22.2664165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2664534Z outputs = self.model( 2025-12-04T09:42:22.2664877Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2665249Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2665612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2665972Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2666304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2666655Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2667033Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:42:22.2667439Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2667608Z 2025-12-04T09:42:22.2667682Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2667877Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2668062Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2668254Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2668448Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2668629Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2668821Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2669009Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2669224Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2669556Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2669915Z res = mod(**inputs) 2025-12-04T09:42:22.2670257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2670611Z outputs = self.model( 2025-12-04T09:42:22.2670973Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2671345Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2671716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2672068Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2672393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2672786Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2673247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:42:22.2673656Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:42:22.2674048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2674419Z return func(*args, **kwargs) 2025-12-04T09:42:22.2674776Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2675182Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2675624Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2676092Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2676276Z 2025-12-04T09:42:22.2676353Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2676562Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2676794Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2677134Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2677452Z res = mod(**inputs) 2025-12-04T09:42:22.2677809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2678190Z outputs = self.model( 2025-12-04T09:42:22.2678544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2678925Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2679297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2679668Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2680014Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2680366Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2680745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:42:22.2681158Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2681329Z 2025-12-04T09:42:22.2681402Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2681605Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2681802Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2681993Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2682187Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2682382Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2682569Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2682765Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2683028Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2683371Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2683686Z res = mod(**inputs) 2025-12-04T09:42:22.2684042Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2684422Z outputs = self.model( 2025-12-04T09:42:22.2684770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2685150Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2685519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2685889Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2686269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2686609Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2686971Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:42:22.2687340Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:42:22.2687706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2688055Z return func(*args, **kwargs) 2025-12-04T09:42:22.2688396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2688774Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2689190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2689639Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2689809Z 2025-12-04T09:42:22.2689882Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2690080Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2690299Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2690634Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2690931Z res = mod(**inputs) 2025-12-04T09:42:22.2691270Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2691631Z outputs = self.model( 2025-12-04T09:42:22.2691968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2692330Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2692695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2693055Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2693378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2693716Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2694079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:42:22.2694476Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2694642Z 2025-12-04T09:42:22.2694715Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2694917Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2695112Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2695295Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2695490Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2695680Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2695890Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2696079Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2696297Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2696626Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2696929Z res = mod(**inputs) 2025-12-04T09:42:22.2697274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2697644Z outputs = self.model( 2025-12-04T09:42:22.2697985Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2698352Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2698709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2699127Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2699451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2699797Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2700165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:42:22.2700545Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:42:22.2700919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2701278Z return func(*args, **kwargs) 2025-12-04T09:42:22.2701631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2702009Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2702440Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2702892Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2703060Z 2025-12-04T09:42:22.2703139Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2703332Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2703549Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2703886Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2704187Z res = mod(**inputs) 2025-12-04T09:42:22.2704531Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2704927Z outputs = self.model( 2025-12-04T09:42:22.2705270Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:42:22.2705642Z encoder_outputs = self.encoder( 2025-12-04T09:42:22.2706186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:42:22.2706555Z layer_outputs = encoder_layer( 2025-12-04T09:42:22.2706881Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2707226Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2707600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:42:22.2708008Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2708168Z 2025-12-04T09:42:22.2708241Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2708443Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2708645Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2708833Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2709103Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2709299Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2709484Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2709679Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2709904Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2710253Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2710555Z res = mod(**inputs) 2025-12-04T09:42:22.2710908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2711280Z outputs = self.model( 2025-12-04T09:42:22.2711627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2712071Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2712435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2712808Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2713176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2713543Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2713928Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2714293Z return func(*args, **kwargs) 2025-12-04T09:42:22.2714681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:42:22.2715073Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:22.2715458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2715818Z return func(*args, **kwargs) 2025-12-04T09:42:22.2716183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2716578Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2717005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2717454Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2717635Z 2025-12-04T09:42:22.2717709Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2717903Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2718092Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2718279Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2718473Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2718662Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2718850Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2719036Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2719255Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2719583Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2719889Z res = mod(**inputs) 2025-12-04T09:42:22.2720238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2720596Z outputs = self.model( 2025-12-04T09:42:22.2720947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2721316Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2721679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2722080Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2722423Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2722773Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2723136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2723489Z return func(*args, **kwargs) 2025-12-04T09:42:22.2723849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:42:22.2724252Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:42:22.2724639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2724996Z return func(*args, **kwargs) 2025-12-04T09:42:22.2725386Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2725767Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2726175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2726615Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2726781Z 2025-12-04T09:42:22.2726864Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2727062Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2727271Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2727605Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2727906Z res = mod(**inputs) 2025-12-04T09:42:22.2728239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2728603Z outputs = self.model( 2025-12-04T09:42:22.2728945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2729324Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2729674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2730032Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2730358Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2730686Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2731040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2731386Z return func(*args, **kwargs) 2025-12-04T09:42:22.2731736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:42:22.2732126Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2732291Z 2025-12-04T09:42:22.2732364Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2732560Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2732742Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2732934Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2733125Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2733313Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2733494Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2733684Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2734039Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2734366Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2734673Z res = mod(**inputs) 2025-12-04T09:42:22.2735051Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2735416Z outputs = self.model( 2025-12-04T09:42:22.2735767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2736136Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2736501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2736859Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2737199Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2737547Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2737908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2738296Z return func(*args, **kwargs) 2025-12-04T09:42:22.2738653Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:42:22.2739050Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:22.2739427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2739781Z return func(*args, **kwargs) 2025-12-04T09:42:22.2740135Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2740525Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2740941Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2741398Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2741568Z 2025-12-04T09:42:22.2741652Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2741850Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2742035Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2742230Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2742420Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2742602Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2742791Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2742981Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2743193Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2743538Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2743844Z res = mod(**inputs) 2025-12-04T09:42:22.2744192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2744553Z outputs = self.model( 2025-12-04T09:42:22.2744902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2745273Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2745629Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2745997Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2746330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2746674Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2747030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2747386Z return func(*args, **kwargs) 2025-12-04T09:42:22.2747777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:42:22.2748173Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:42:22.2748562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2748918Z return func(*args, **kwargs) 2025-12-04T09:42:22.2749271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2749654Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2750083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2750541Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2750710Z 2025-12-04T09:42:22.2750826Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2751018Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2751241Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2751580Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2751883Z res = mod(**inputs) 2025-12-04T09:42:22.2752235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2752606Z outputs = self.model( 2025-12-04T09:42:22.2752957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2753438Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2753809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2754188Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2754528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2754888Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2755254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2755607Z return func(*args, **kwargs) 2025-12-04T09:42:22.2755957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:42:22.2756364Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2756529Z 2025-12-04T09:42:22.2756612Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2756809Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2756997Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2757189Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2757386Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2757574Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2757771Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2757965Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2758181Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2758526Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2758839Z res = mod(**inputs) 2025-12-04T09:42:22.2759184Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2759554Z outputs = self.model( 2025-12-04T09:42:22.2759908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2760281Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2760639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2761057Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2761410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2761760Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2762125Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2762488Z return func(*args, **kwargs) 2025-12-04T09:42:22.2762850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:42:22.2763248Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:22.2763649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2764065Z return func(*args, **kwargs) 2025-12-04T09:42:22.2764432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2764829Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2765265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2765732Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2765907Z 2025-12-04T09:42:22.2765989Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2766187Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2766388Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2766589Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2766776Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2766977Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2767173Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2767369Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2767601Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2767955Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2768273Z res = mod(**inputs) 2025-12-04T09:42:22.2768611Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2768979Z outputs = self.model( 2025-12-04T09:42:22.2769328Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2769691Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2770055Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2770425Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2770766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2771102Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2771465Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2771824Z return func(*args, **kwargs) 2025-12-04T09:42:22.2772170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:42:22.2772573Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:42:22.2772960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2773317Z return func(*args, **kwargs) 2025-12-04T09:42:22.2773670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2774067Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2774528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2774983Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2775155Z 2025-12-04T09:42:22.2775227Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2775423Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2775641Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2775965Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2776267Z res = mod(**inputs) 2025-12-04T09:42:22.2776614Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2776976Z outputs = self.model( 2025-12-04T09:42:22.2777366Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2777738Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2778100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2778462Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2778795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2779140Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2779500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2779851Z return func(*args, **kwargs) 2025-12-04T09:42:22.2780210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:42:22.2780627Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2780792Z 2025-12-04T09:42:22.2780872Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2781061Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2781255Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2781448Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2781634Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2781827Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2782018Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2782201Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2782424Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2782765Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2783068Z res = mod(**inputs) 2025-12-04T09:42:22.2783418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2783789Z outputs = self.model( 2025-12-04T09:42:22.2784144Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2784505Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2784872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2785239Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2785570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2785911Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2786274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2786632Z return func(*args, **kwargs) 2025-12-04T09:42:22.2787016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:42:22.2787414Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:22.2787796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2788156Z return func(*args, **kwargs) 2025-12-04T09:42:22.2788506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2788900Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2789327Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2789781Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2789952Z 2025-12-04T09:42:22.2790064Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2790265Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2790468Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2790661Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2790858Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2791054Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2791245Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2791444Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2791671Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2792019Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2792328Z res = mod(**inputs) 2025-12-04T09:42:22.2792689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2793072Z outputs = self.model( 2025-12-04T09:42:22.2793532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2793945Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2794342Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2794743Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2795071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2795447Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2795842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2796225Z return func(*args, **kwargs) 2025-12-04T09:42:22.2796615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:42:22.2797055Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:42:22.2797485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2797865Z return func(*args, **kwargs) 2025-12-04T09:42:22.2798250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2798677Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2799130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2799614Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2799808Z 2025-12-04T09:42:22.2799888Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2800098Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2800330Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2800750Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2801087Z res = mod(**inputs) 2025-12-04T09:42:22.2801458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2801846Z outputs = self.model( 2025-12-04T09:42:22.2802221Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2802620Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2802994Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2803362Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2803694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2804079Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2804433Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2804792Z return func(*args, **kwargs) 2025-12-04T09:42:22.2805147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:42:22.2805558Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2805821Z 2025-12-04T09:42:22.2805898Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2806097Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2806292Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2806476Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2806672Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2806865Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2807156Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2807348Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2807570Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2807914Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2808213Z res = mod(**inputs) 2025-12-04T09:42:22.2808559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2808926Z outputs = self.model( 2025-12-04T09:42:22.2809270Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2809639Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2810005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2810381Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2810714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2811061Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2811426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2811777Z return func(*args, **kwargs) 2025-12-04T09:42:22.2812132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:42:22.2812527Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:22.2812914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2813265Z return func(*args, **kwargs) 2025-12-04T09:42:22.2813618Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2814012Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2814576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2815030Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2815214Z 2025-12-04T09:42:22.2815290Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2815496Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2815685Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2815882Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2816077Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2816269Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2816463Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2816654Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2816875Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2817263Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2817568Z res = mod(**inputs) 2025-12-04T09:42:22.2817917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2818280Z outputs = self.model( 2025-12-04T09:42:22.2818629Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2819003Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2819367Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2819731Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2820062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2820414Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2820775Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2821136Z return func(*args, **kwargs) 2025-12-04T09:42:22.2821491Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:42:22.2821890Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:42:22.2822270Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2822628Z return func(*args, **kwargs) 2025-12-04T09:42:22.2822980Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2823370Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2823793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2824243Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2824411Z 2025-12-04T09:42:22.2824492Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2824682Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2824899Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2825234Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2825536Z res = mod(**inputs) 2025-12-04T09:42:22.2825870Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2826236Z outputs = self.model( 2025-12-04T09:42:22.2826587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2826951Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2827354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2827726Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2828058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2828399Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2828761Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2829116Z return func(*args, **kwargs) 2025-12-04T09:42:22.2829468Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:42:22.2829871Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2830077Z 2025-12-04T09:42:22.2830149Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2830347Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2830532Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2830724Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2830915Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2831099Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2831294Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2831483Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2831699Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2832027Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2832330Z res = mod(**inputs) 2025-12-04T09:42:22.2832673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2833041Z outputs = self.model( 2025-12-04T09:42:22.2833471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2833856Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2834233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2834610Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2834948Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2835308Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2835677Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2836046Z return func(*args, **kwargs) 2025-12-04T09:42:22.2836416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:42:22.2836829Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:22.2837224Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2837594Z return func(*args, **kwargs) 2025-12-04T09:42:22.2837959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2838360Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2838790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2839255Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2839430Z 2025-12-04T09:42:22.2839511Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2839704Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2839906Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2840103Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2840339Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2840532Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2840730Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2840924Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2841140Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2841489Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2841807Z res = mod(**inputs) 2025-12-04T09:42:22.2842154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2842533Z outputs = self.model( 2025-12-04T09:42:22.2842898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2843317Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2843689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2844082Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2844432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2844791Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2845175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2845552Z return func(*args, **kwargs) 2025-12-04T09:42:22.2845919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:42:22.2846335Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:42:22.2846754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2847128Z return func(*args, **kwargs) 2025-12-04T09:42:22.2847492Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2847884Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2848315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2848772Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2848946Z 2025-12-04T09:42:22.2849021Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2849226Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2849454Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2849801Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2850110Z res = mod(**inputs) 2025-12-04T09:42:22.2850464Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2850840Z outputs = self.model( 2025-12-04T09:42:22.2851190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2851566Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2851935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2852316Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2852646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2852997Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2853366Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2853763Z return func(*args, **kwargs) 2025-12-04T09:42:22.2854125Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:42:22.2854552Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2854715Z 2025-12-04T09:42:22.2854796Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2854982Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2855170Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2855358Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2855540Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2855732Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2855920Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2856112Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2856323Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2856706Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2857015Z res = mod(**inputs) 2025-12-04T09:42:22.2857354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2857721Z outputs = self.model( 2025-12-04T09:42:22.2858070Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2858443Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2858685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2858753Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2858972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2859055Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2859289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2859367Z return func(*args, **kwargs) 2025-12-04T09:42:22.2859607Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:42:22.2859710Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:22.2859942Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2860007Z return func(*args, **kwargs) 2025-12-04T09:42:22.2860258Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2860349Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2860636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2860757Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2860760Z 2025-12-04T09:42:22.2860835Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2860914Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2860983Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2861054Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2861134Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2861203Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2861280Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2861350Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2861451Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2861645Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2861708Z res = mod(**inputs) 2025-12-04T09:42:22.2862001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2862075Z outputs = self.model( 2025-12-04T09:42:22.2862317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2862394Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2862637Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2862704Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2862922Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2862995Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2863226Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2863340Z return func(*args, **kwargs) 2025-12-04T09:42:22.2863579Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:42:22.2863689Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:42:22.2863919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2863981Z return func(*args, **kwargs) 2025-12-04T09:42:22.2864233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2864324Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2864600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2864733Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2864736Z 2025-12-04T09:42:22.2864811Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2864890Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2864988Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2865173Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2865242Z res = mod(**inputs) 2025-12-04T09:42:22.2865484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2865557Z outputs = self.model( 2025-12-04T09:42:22.2865795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2865863Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2866109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2866180Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2866392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2866474Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2866703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2866773Z return func(*args, **kwargs) 2025-12-04T09:42:22.2867011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:42:22.2867123Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2867126Z 2025-12-04T09:42:22.2867203Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2867273Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2867346Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2867422Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2867524Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2867602Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2867672Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2867742Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2867844Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2868029Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2868089Z res = mod(**inputs) 2025-12-04T09:42:22.2868338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2868400Z outputs = self.model( 2025-12-04T09:42:22.2868648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2868747Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2868988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2869062Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2869274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2869349Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2869585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2869649Z return func(*args, **kwargs) 2025-12-04T09:42:22.2869894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:42:22.2869986Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:22.2870221Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2870296Z return func(*args, **kwargs) 2025-12-04T09:42:22.2870541Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2870637Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2870910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2871030Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2871033Z 2025-12-04T09:42:22.2871114Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2871184Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2871253Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2871331Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2871405Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2871481Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2871553Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2871624Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2871728Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2871911Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2871971Z res = mod(**inputs) 2025-12-04T09:42:22.2872219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2872283Z outputs = self.model( 2025-12-04T09:42:22.2872524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2872598Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2872839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2872950Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2873240Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2873320Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2873562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2873629Z return func(*args, **kwargs) 2025-12-04T09:42:22.2873890Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:42:22.2873995Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:42:22.2874237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2874313Z return func(*args, **kwargs) 2025-12-04T09:42:22.2874592Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2874684Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2874969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2875089Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2875092Z 2025-12-04T09:42:22.2875173Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2875246Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2875346Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2875539Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2875599Z res = mod(**inputs) 2025-12-04T09:42:22.2875850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2875920Z outputs = self.model( 2025-12-04T09:42:22.2876161Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2876236Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2876475Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2876542Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2876763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2876836Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2877071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2877134Z return func(*args, **kwargs) 2025-12-04T09:42:22.2877377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:42:22.2877496Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2877499Z 2025-12-04T09:42:22.2877570Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2877639Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2877718Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2877787Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2877862Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2877932Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2878001Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2878077Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2878173Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2878357Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2878427Z res = mod(**inputs) 2025-12-04T09:42:22.2878712Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2878785Z outputs = self.model( 2025-12-04T09:42:22.2879027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2879096Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2879345Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2879413Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2879630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2879709Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2879937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2880073Z return func(*args, **kwargs) 2025-12-04T09:42:22.2880315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:42:22.2880407Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:22.2880642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2880704Z return func(*args, **kwargs) 2025-12-04T09:42:22.2880940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2881034Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2881310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2881440Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2881445Z 2025-12-04T09:42:22.2881517Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2881588Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2881665Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2881734Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2881809Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2881879Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2881949Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2882025Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2882122Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2882309Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2882378Z res = mod(**inputs) 2025-12-04T09:42:22.2882622Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2882690Z outputs = self.model( 2025-12-04T09:42:22.2882945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2883013Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2883265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2883331Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2883542Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2883625Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2883857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2883927Z return func(*args, **kwargs) 2025-12-04T09:42:22.2884202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:42:22.2884306Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:42:22.2884543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2884606Z return func(*args, **kwargs) 2025-12-04T09:42:22.2884857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2884953Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2885238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2885366Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2885369Z 2025-12-04T09:42:22.2885475Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2885545Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2885653Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2885837Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2885896Z res = mod(**inputs) 2025-12-04T09:42:22.2886147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2886208Z outputs = self.model( 2025-12-04T09:42:22.2886457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2886524Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2886765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2886839Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2887055Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2887134Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2887361Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2887423Z return func(*args, **kwargs) 2025-12-04T09:42:22.2887669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:42:22.2887780Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2887784Z 2025-12-04T09:42:22.2887854Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2887933Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2888001Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2888077Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2888150Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2888219Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2888297Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2888366Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2888462Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2888657Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2888716Z res = mod(**inputs) 2025-12-04T09:42:22.2888961Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2889031Z outputs = self.model( 2025-12-04T09:42:22.2889271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2889347Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2889590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2889693Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2889912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2889986Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2890225Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2890289Z return func(*args, **kwargs) 2025-12-04T09:42:22.2890526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:42:22.2890627Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:22.2890853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2890954Z return func(*args, **kwargs) 2025-12-04T09:42:22.2891197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2891286Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2891560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2891678Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2891681Z 2025-12-04T09:42:22.2891752Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2891831Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2891898Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2891967Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2892042Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2892110Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2892185Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2892255Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2892352Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2892538Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2892597Z res = mod(**inputs) 2025-12-04T09:42:22.2892833Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2892903Z outputs = self.model( 2025-12-04T09:42:22.2893137Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2893212Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2893443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2893509Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2893727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2893799Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2894023Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2894093Z return func(*args, **kwargs) 2025-12-04T09:42:22.2894325Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:42:22.2894431Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:42:22.2894652Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2894714Z return func(*args, **kwargs) 2025-12-04T09:42:22.2894956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2895046Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2895356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2895476Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2895479Z 2025-12-04T09:42:22.2895555Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2895633Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2895728Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2895909Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2895976Z res = mod(**inputs) 2025-12-04T09:42:22.2896211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2896280Z outputs = self.model( 2025-12-04T09:42:22.2896564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2896633Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2896884Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2896950Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2897163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2897246Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2897476Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2897549Z return func(*args, **kwargs) 2025-12-04T09:42:22.2897788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:42:22.2897904Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2897909Z 2025-12-04T09:42:22.2897991Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2898063Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2898140Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2898212Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2898281Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2898358Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2898428Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2898497Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2898598Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2898783Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2898843Z res = mod(**inputs) 2025-12-04T09:42:22.2899093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2899160Z outputs = self.model( 2025-12-04T09:42:22.2899408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2899475Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2899713Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2899787Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2899997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2900079Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2900308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2900372Z return func(*args, **kwargs) 2025-12-04T09:42:22.2900657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:42:22.2900750Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:22.2900982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2901054Z return func(*args, **kwargs) 2025-12-04T09:42:22.2901297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2905805Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2906115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2906238Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2906242Z 2025-12-04T09:42:22.2906374Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2906448Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2906522Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2906599Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2906667Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2906736Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2906816Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2906885Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2906982Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2907180Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2907275Z res = mod(**inputs) 2025-12-04T09:42:22.2907520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2907590Z outputs = self.model( 2025-12-04T09:42:22.2907838Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2907915Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2908153Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2908218Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2908435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2908508Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2908745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2908809Z return func(*args, **kwargs) 2025-12-04T09:42:22.2909047Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:42:22.2909158Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:42:22.2909400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2909464Z return func(*args, **kwargs) 2025-12-04T09:42:22.2909710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2909798Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2910077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2910199Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2910201Z 2025-12-04T09:42:22.2910272Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2910352Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2910450Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2910699Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2910767Z res = mod(**inputs) 2025-12-04T09:42:22.2911002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2911072Z outputs = self.model( 2025-12-04T09:42:22.2911304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2911372Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2911683Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2911751Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2911967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2912062Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2912297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2912372Z return func(*args, **kwargs) 2025-12-04T09:42:22.2912612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:42:22.2912724Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2912727Z 2025-12-04T09:42:22.2912809Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2912883Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2912960Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2913029Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2913098Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2913232Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2913308Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2913378Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2913485Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2913672Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2913733Z res = mod(**inputs) 2025-12-04T09:42:22.2913989Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2914052Z outputs = self.model( 2025-12-04T09:42:22.2914301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2914373Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2914612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2914688Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2914902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2914985Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2915215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2915282Z return func(*args, **kwargs) 2025-12-04T09:42:22.2915530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:42:22.2915624Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:22.2915855Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2915930Z return func(*args, **kwargs) 2025-12-04T09:42:22.2916170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2916274Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2916588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2916715Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2916719Z 2025-12-04T09:42:22.2916799Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2916869Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2916945Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2917013Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2917116Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2917192Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2917260Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2917330Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2917432Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2917641Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2917702Z res = mod(**inputs) 2025-12-04T09:42:22.2917956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2918021Z outputs = self.model( 2025-12-04T09:42:22.2918269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2918336Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2918580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2918654Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2918865Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2918939Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2919182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2919247Z return func(*args, **kwargs) 2025-12-04T09:42:22.2919494Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:42:22.2919595Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:42:22.2919826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2919900Z return func(*args, **kwargs) 2025-12-04T09:42:22.2920141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:42:22.2920239Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:22.2920518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:22.2920637Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:22.2920640Z 2025-12-04T09:42:22.2920719Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2920790Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2920885Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2921080Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2921141Z res = mod(**inputs) 2025-12-04T09:42:22.2921392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:42:22.2921455Z outputs = self.model( 2025-12-04T09:42:22.2921699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:42:22.2921778Z decoder_outputs = self.decoder( 2025-12-04T09:42:22.2922051Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:42:22.2922119Z layer_outputs = decoder_layer( 2025-12-04T09:42:22.2922335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:22.2922408Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:22.2922644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:22.2922730Z return func(*args, **kwargs) 2025-12-04T09:42:22.2922972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:42:22.2923091Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:22.2923111Z 2025-12-04T09:42:22.2923182Z cudagraph partition due to non gpu ops 2025-12-04T09:42:22.2923287Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2923474Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2923532Z res = mod(**inputs) 2025-12-04T09:42:22.2923783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1419, in forward 2025-12-04T09:42:22.2923857Z lm_logits = self.lm_head(outputs[0]) 2025-12-04T09:42:22.2923861Z 2025-12-04T09:42:22.2923955Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:22.2924155Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:22.2924213Z res = mod(**inputs) 2025-12-04T09:42:22.2924458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1426, in forward 2025-12-04T09:42:22.2924616Z masked_lm_loss = loss_fct(lm_logits.view(-1, self.config.vocab_size), labels.view(-1)) 2025-12-04T09:42:22.2924621Z 2025-12-04T09:42:33.7424604Z Compilation time (from dynamo_timed): 29.326873399 2025-12-04T09:42:33.7516116Z pass 2025-12-04T09:42:33.7517953Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:42:33.7518846Z TIMING: _recursive_pre_grad_passes:0.06911 _recursive_joint_graph_passes:0.73782 _recursive_post_grad_passes:0.09681 async_compile.wait:0.71925 code_gen:11.4007 inductor_compile:14.96281 backend_compile:24.03162 gc:0.00067 entire_frame_compile:29.32687 total_wall_time:29.32687 2025-12-04T09:42:33.7523608Z STATS: call_* op count: 906 | FakeTensorMode.__torch_dispatch__:46361 | FakeTensor.__torch_dispatch__:8011 | ProxyTorchDispatchMode.__torch_dispatch__:6566 2025-12-04T09:42:33.7528131Z Dynamo produced 1 graphs covering 906 ops with 0 graph breaks (0 unique) 2025-12-04T09:42:36.8298113Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:42:36.8298927Z import pynvml # type: ignore[import] 2025-12-04T09:42:39.8991718Z 2025-12-04T09:42:42.0381592Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:42:42.0383146Z loading model: 0it [00:02, ?it/s] 2025-12-04T09:42:42.0403007Z cpu eval MBartForCausalLM 2025-12-04T09:42:43.5964771Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:42:44.1479994Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:42:44.7271463Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:42:52.9374777Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9375366Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9376552Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9381059Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9383290Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9383675Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9383921Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9384223Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9384438Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9384750Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9384950Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9385376Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9385579Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9385774Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9385970Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9386166Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9386463Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9386666Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9386851Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9387089Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9387463Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9387796Z res = mod(**inputs) 2025-12-04T09:42:52.9388186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9388580Z outputs = self.model.decoder( 2025-12-04T09:42:52.9388961Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9389349Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9389701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9390061Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9390438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9390807Z return func(*args, **kwargs) 2025-12-04T09:42:52.9391171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:42:52.9391575Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:52.9391967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9392329Z return func(*args, **kwargs) 2025-12-04T09:42:52.9392687Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:42:52.9393262Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:52.9393720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:52.9394216Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:52.9394405Z 2025-12-04T09:42:52.9394492Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9394806Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9395045Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9395403Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9395732Z res = mod(**inputs) 2025-12-04T09:42:52.9396097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9396536Z outputs = self.model.decoder( 2025-12-04T09:42:52.9396928Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9397318Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9397731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9398115Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9398512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9398885Z return func(*args, **kwargs) 2025-12-04T09:42:52.9399267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:42:52.9399741Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:52.9400134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:52.9400482Z return self.act(input) 2025-12-04T09:42:52.9400621Z 2025-12-04T09:42:52.9400700Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9400908Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9401109Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9401310Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9401512Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9401705Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9401906Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9402106Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9402332Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9402692Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9403016Z res = mod(**inputs) 2025-12-04T09:42:52.9403382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9403771Z outputs = self.model.decoder( 2025-12-04T09:42:52.9404160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9404550Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9404901Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9405302Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9405874Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9406260Z return func(*args, **kwargs) 2025-12-04T09:42:52.9406646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:42:52.9407067Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:52.9407476Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9407850Z return func(*args, **kwargs) 2025-12-04T09:42:52.9408220Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:42:52.9408626Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:52.9409061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:52.9409532Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:52.9409719Z 2025-12-04T09:42:52.9409794Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9409996Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9410217Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9410562Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9410889Z res = mod(**inputs) 2025-12-04T09:42:52.9411315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9411713Z outputs = self.model.decoder( 2025-12-04T09:42:52.9412115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9412544Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9412878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9413238Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9413650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9414020Z return func(*args, **kwargs) 2025-12-04T09:42:52.9414382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:42:52.9414830Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:52.9415217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:52.9415553Z return self.act(input) 2025-12-04T09:42:52.9415667Z 2025-12-04T09:42:52.9415742Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9415949Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9416149Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9416339Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9416536Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9416737Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9416927Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9417122Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9417350Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9417691Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9418010Z res = mod(**inputs) 2025-12-04T09:42:52.9418372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9418759Z outputs = self.model.decoder( 2025-12-04T09:42:52.9419131Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9419516Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9419857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9420209Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9420584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9420954Z return func(*args, **kwargs) 2025-12-04T09:42:52.9421320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:42:52.9421724Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:52.9422118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9422486Z return func(*args, **kwargs) 2025-12-04T09:42:52.9422843Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:42:52.9423242Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:52.9423690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:52.9424149Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:52.9424320Z 2025-12-04T09:42:52.9424393Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9424592Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9424872Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9425205Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9425500Z res = mod(**inputs) 2025-12-04T09:42:52.9425840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9441492Z outputs = self.model.decoder( 2025-12-04T09:42:52.9441948Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9442458Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9442826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9443203Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9443595Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9444008Z return func(*args, **kwargs) 2025-12-04T09:42:52.9444390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:42:52.9444826Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:52.9445210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:52.9445558Z return self.act(input) 2025-12-04T09:42:52.9445679Z 2025-12-04T09:42:52.9445764Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9445976Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9446168Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9446373Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9446574Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9446769Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9446969Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9447171Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9447395Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9447761Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9448086Z res = mod(**inputs) 2025-12-04T09:42:52.9448489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9448878Z outputs = self.model.decoder( 2025-12-04T09:42:52.9449261Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9449642Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9449981Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9450346Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9450725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9451094Z return func(*args, **kwargs) 2025-12-04T09:42:52.9451464Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:42:52.9451887Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:52.9452286Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9452657Z return func(*args, **kwargs) 2025-12-04T09:42:52.9453014Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:42:52.9453427Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:52.9453869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:52.9454377Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:52.9454567Z 2025-12-04T09:42:52.9454644Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9454925Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9455158Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9455506Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9455830Z res = mod(**inputs) 2025-12-04T09:42:52.9456214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9456605Z outputs = self.model.decoder( 2025-12-04T09:42:52.9456966Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9457355Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9457699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9458040Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9458406Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9458764Z return func(*args, **kwargs) 2025-12-04T09:42:52.9459122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:42:52.9459532Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:52.9459912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:52.9460243Z return self.act(input) 2025-12-04T09:42:52.9460348Z 2025-12-04T09:42:52.9460423Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9460622Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9460823Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9461017Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9461197Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9461387Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9461577Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9461763Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9461985Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9462332Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9462634Z res = mod(**inputs) 2025-12-04T09:42:52.9462978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9463348Z outputs = self.model.decoder( 2025-12-04T09:42:52.9463706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9464074Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9464406Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9464754Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9465126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9465517Z return func(*args, **kwargs) 2025-12-04T09:42:52.9465875Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:42:52.9466271Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:52.9466644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9467005Z return func(*args, **kwargs) 2025-12-04T09:42:52.9467394Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:42:52.9467790Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:52.9468217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:52.9468687Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:52.9468864Z 2025-12-04T09:42:52.9468947Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9469164Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9469383Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9469734Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9470043Z res = mod(**inputs) 2025-12-04T09:42:52.9470388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9470783Z outputs = self.model.decoder( 2025-12-04T09:42:52.9471146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9471520Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9471843Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9472187Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9472547Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9472899Z return func(*args, **kwargs) 2025-12-04T09:42:52.9473348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:42:52.9473781Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:52.9474171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:52.9474501Z return self.act(input) 2025-12-04T09:42:52.9474628Z 2025-12-04T09:42:52.9474702Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9474904Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9475095Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9475297Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9475498Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9475691Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9475894Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9476097Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9476329Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9476671Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9476987Z res = mod(**inputs) 2025-12-04T09:42:52.9477350Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9477728Z outputs = self.model.decoder( 2025-12-04T09:42:52.9478100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9478475Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9478815Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9479164Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9479538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9479904Z return func(*args, **kwargs) 2025-12-04T09:42:52.9480261Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:42:52.9480704Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:52.9481106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9481477Z return func(*args, **kwargs) 2025-12-04T09:42:52.9481838Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:42:52.9482245Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:52.9482684Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:52.9483173Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:52.9483351Z 2025-12-04T09:42:52.9483426Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9483631Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9483877Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9484222Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9484536Z res = mod(**inputs) 2025-12-04T09:42:52.9484890Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9485270Z outputs = self.model.decoder( 2025-12-04T09:42:52.9485632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9486012Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9486389Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9486727Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9487089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9487448Z return func(*args, **kwargs) 2025-12-04T09:42:52.9487802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:42:52.9488209Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:52.9488580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:52.9488903Z return self.act(input) 2025-12-04T09:42:52.9489006Z 2025-12-04T09:42:52.9489084Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9489277Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9489470Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9489660Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9489846Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9490037Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9490230Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9490414Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9490639Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9490980Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9491282Z res = mod(**inputs) 2025-12-04T09:42:52.9491630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9492001Z outputs = self.model.decoder( 2025-12-04T09:42:52.9492361Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9492723Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9493055Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9493402Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9493806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9494157Z return func(*args, **kwargs) 2025-12-04T09:42:52.9494512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:42:52.9494907Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:52.9495281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9495644Z return func(*args, **kwargs) 2025-12-04T09:42:52.9495996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:42:52.9496377Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:52.9496782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:52.9497258Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:52.9497428Z 2025-12-04T09:42:52.9497510Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9497711Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9497925Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9498266Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9498571Z res = mod(**inputs) 2025-12-04T09:42:52.9498911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9499283Z outputs = self.model.decoder( 2025-12-04T09:42:52.9499646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9500015Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9500342Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9500686Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9501044Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9501386Z return func(*args, **kwargs) 2025-12-04T09:42:52.9501738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:42:52.9502146Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:52.9502516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:52.9502834Z return self.act(input) 2025-12-04T09:42:52.9502942Z 2025-12-04T09:42:52.9503013Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9503210Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9503395Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9503587Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9503781Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9503965Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9504153Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9504345Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9504565Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9504898Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9505209Z res = mod(**inputs) 2025-12-04T09:42:52.9505555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9506148Z outputs = self.model.decoder( 2025-12-04T09:42:52.9506508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9506954Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9507307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9507646Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9508013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9508373Z return func(*args, **kwargs) 2025-12-04T09:42:52.9508725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:42:52.9509153Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:52.9509535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9509888Z return func(*args, **kwargs) 2025-12-04T09:42:52.9510260Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:42:52.9510653Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:52.9511077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:52.9511535Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:52.9511706Z 2025-12-04T09:42:52.9511780Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9511976Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9512195Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9512522Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9512824Z res = mod(**inputs) 2025-12-04T09:42:52.9513221Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9513609Z outputs = self.model.decoder( 2025-12-04T09:42:52.9513974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9514353Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9514680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9515016Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9515374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9515734Z return func(*args, **kwargs) 2025-12-04T09:42:52.9516086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:42:52.9516486Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:52.9516866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:52.9517191Z return self.act(input) 2025-12-04T09:42:52.9517293Z 2025-12-04T09:42:52.9517365Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9517561Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9517753Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9517944Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9518129Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9518320Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9518513Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9518696Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9518920Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9519256Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9519555Z res = mod(**inputs) 2025-12-04T09:42:52.9519928Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9520313Z outputs = self.model.decoder( 2025-12-04T09:42:52.9520674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9521035Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9521365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9521709Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9522082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9522440Z return func(*args, **kwargs) 2025-12-04T09:42:52.9522797Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:42:52.9523211Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:52.9523596Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9523960Z return func(*args, **kwargs) 2025-12-04T09:42:52.9524321Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:42:52.9524725Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:52.9525152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:52.9525613Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:52.9525786Z 2025-12-04T09:42:52.9525871Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9526066Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9526295Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9526642Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9526954Z res = mod(**inputs) 2025-12-04T09:42:52.9527295Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9527672Z outputs = self.model.decoder( 2025-12-04T09:42:52.9528038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9528409Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9528752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9529100Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9529470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9529828Z return func(*args, **kwargs) 2025-12-04T09:42:52.9530192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:42:52.9530609Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:52.9530985Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:52.9531307Z return self.act(input) 2025-12-04T09:42:52.9531423Z 2025-12-04T09:42:52.9531498Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9531703Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9531896Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9532097Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9532294Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9532485Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9532684Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9532884Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9533140Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9533475Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9533783Z res = mod(**inputs) 2025-12-04T09:42:52.9534130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9534496Z outputs = self.model.decoder( 2025-12-04T09:42:52.9534860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9535250Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9535579Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9535917Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9536313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9536670Z return func(*args, **kwargs) 2025-12-04T09:42:52.9537020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:42:52.9537412Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:52.9537804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9538160Z return func(*args, **kwargs) 2025-12-04T09:42:52.9538511Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:42:52.9538903Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:52.9539340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:52.9539807Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:52.9539982Z 2025-12-04T09:42:52.9540056Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9540255Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9540475Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9540803Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9541107Z res = mod(**inputs) 2025-12-04T09:42:52.9541453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9541830Z outputs = self.model.decoder( 2025-12-04T09:42:52.9542184Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9542555Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9542886Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9543227Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9543588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9543940Z return func(*args, **kwargs) 2025-12-04T09:42:52.9544293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:42:52.9544700Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:52.9545072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:52.9545394Z return self.act(input) 2025-12-04T09:42:52.9545498Z 2025-12-04T09:42:52.9545570Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9545766Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9545959Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9546183Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9546370Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9546563Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9546757Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9546943Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9547164Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9547505Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9547809Z res = mod(**inputs) 2025-12-04T09:42:52.9548163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9548537Z outputs = self.model.decoder( 2025-12-04T09:42:52.9548891Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9549272Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9549609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9549962Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9550323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9550686Z return func(*args, **kwargs) 2025-12-04T09:42:52.9551049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:42:52.9551450Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:52.9551831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9552200Z return func(*args, **kwargs) 2025-12-04T09:42:52.9552564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:42:52.9552963Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:52.9553457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:52.9553919Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:52.9554091Z 2025-12-04T09:42:52.9554176Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9554364Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9554594Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9554929Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9555233Z res = mod(**inputs) 2025-12-04T09:42:52.9555574Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9555946Z outputs = self.model.decoder( 2025-12-04T09:42:52.9556307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9556669Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9557002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9557347Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9557707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9558056Z return func(*args, **kwargs) 2025-12-04T09:42:52.9558407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:42:52.9558819Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:52.9559189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:52.9559548Z return self.act(input) 2025-12-04T09:42:52.9559661Z 2025-12-04T09:42:52.9559733Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9559927Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9560111Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9560302Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9560492Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9560672Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9560862Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9561067Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9561280Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9561619Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9561926Z res = mod(**inputs) 2025-12-04T09:42:52.9562297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9562664Z outputs = self.model.decoder( 2025-12-04T09:42:52.9563022Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9563401Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9563721Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9564062Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9564423Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9564778Z return func(*args, **kwargs) 2025-12-04T09:42:52.9565125Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:42:52.9565518Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:42:52.9565899Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9566247Z return func(*args, **kwargs) 2025-12-04T09:42:52.9566593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:42:52.9566984Z attn_output, attn_weights = attention_interface( 2025-12-04T09:42:52.9567407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:42:52.9567864Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:42:52.9568043Z 2025-12-04T09:42:52.9568117Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9568313Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9568531Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9568867Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9569175Z res = mod(**inputs) 2025-12-04T09:42:52.9569516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:42:52.9569878Z outputs = self.model.decoder( 2025-12-04T09:42:52.9570239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:42:52.9570605Z layer_outputs = decoder_layer( 2025-12-04T09:42:52.9570929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:52.9571262Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:52.9571621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:42:52.9571973Z return func(*args, **kwargs) 2025-12-04T09:42:52.9572355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:42:52.9572763Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:42:52.9573133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:52.9573460Z return self.act(input) 2025-12-04T09:42:52.9573564Z 2025-12-04T09:42:52.9573636Z cudagraph partition due to non gpu ops 2025-12-04T09:42:52.9573856Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9574241Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9574544Z res = mod(**inputs) 2025-12-04T09:42:52.9574880Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1881, in forward 2025-12-04T09:42:52.9575275Z logits = self.lm_head(outputs[0]) 2025-12-04T09:42:52.9575394Z 2025-12-04T09:42:52.9575501Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:52.9575829Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:52.9576137Z res = mod(**inputs) 2025-12-04T09:42:52.9576478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1887, in forward 2025-12-04T09:42:52.9576921Z loss = loss_fct(logits.view(-1, self.config.vocab_size), labels.view(-1)) 2025-12-04T09:42:52.9577110Z 2025-12-04T09:43:02.8097740Z Compilation time (from dynamo_timed): 17.02738099 2025-12-04T09:43:02.8334951Z pass 2025-12-04T09:43:02.8335542Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:43:02.8336463Z TIMING: _recursive_pre_grad_passes:0.02905 _recursive_joint_graph_passes:0.61816 _recursive_post_grad_passes:0.05251 async_compile.wait:0.80777 code_gen:9.47733 inductor_compile:11.10048 backend_compile:14.81155 gc:0.00128 entire_frame_compile:17.02738 total_wall_time:17.02738 2025-12-04T09:43:02.8337372Z STATS: call_* op count: 337 | FakeTensorMode.__torch_dispatch__:18975 | FakeTensor.__torch_dispatch__:3530 | ProxyTorchDispatchMode.__torch_dispatch__:2869 2025-12-04T09:43:02.8337848Z Dynamo produced 1 graphs covering 337 ops with 0 graph breaks (0 unique) 2025-12-04T09:43:05.2313382Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:43:05.2314404Z import pynvml # type: ignore[import] 2025-12-04T09:43:08.3593976Z 2025-12-04T09:43:10.3662117Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:43:10.3664049Z loading model: 0it [00:02, ?it/s] 2025-12-04T09:43:10.3677883Z cpu eval MT5ForConditionalGeneration 2025-12-04T09:43:10.9512611Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:43:11.1985474Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:43:11.4432802Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:43:25.0342265Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0346516Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0348703Z res = mod(**inputs) 2025-12-04T09:43:25.0349221Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0349628Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0350020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0350425Z layer_outputs = layer_module( 2025-12-04T09:43:25.0351160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0351538Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0351914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0352286Z return func(*args, **kwargs) 2025-12-04T09:43:25.0352650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0353217Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0353606Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0353970Z return func(*args, **kwargs) 2025-12-04T09:43:25.0354480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0354883Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0355258Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0355629Z return func(*args, **kwargs) 2025-12-04T09:43:25.0356001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 424, in forward 2025-12-04T09:43:25.0356398Z position_bias = position_bias + causal_mask 2025-12-04T09:43:25.0356560Z 2025-12-04T09:43:25.0356668Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0357032Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0357357Z res = mod(**inputs) 2025-12-04T09:43:25.0357705Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0358092Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0358454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0358809Z layer_outputs = layer_module( 2025-12-04T09:43:25.0359151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0359519Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0359893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0360255Z return func(*args, **kwargs) 2025-12-04T09:43:25.0360609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0360973Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0361343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0361713Z return func(*args, **kwargs) 2025-12-04T09:43:25.0362066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0362436Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0362795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0363151Z return func(*args, **kwargs) 2025-12-04T09:43:25.0363501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.0363870Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.0364000Z 2025-12-04T09:43:25.0364103Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0364483Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0364802Z res = mod(**inputs) 2025-12-04T09:43:25.0365183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0365542Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0365891Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0366328Z layer_outputs = layer_module( 2025-12-04T09:43:25.0366660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0367032Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0367395Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0367774Z return func(*args, **kwargs) 2025-12-04T09:43:25.0368154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0368520Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0368881Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0369225Z return func(*args, **kwargs) 2025-12-04T09:43:25.0369567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0370122Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0370482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0370823Z return func(*args, **kwargs) 2025-12-04T09:43:25.0371158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.0371512Z key_states = self.k(current_states) 2025-12-04T09:43:25.0371636Z 2025-12-04T09:43:25.0371740Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0372081Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0372394Z res = mod(**inputs) 2025-12-04T09:43:25.0372722Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0373068Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0373420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0373773Z layer_outputs = layer_module( 2025-12-04T09:43:25.0374097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0374437Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0374797Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0375145Z return func(*args, **kwargs) 2025-12-04T09:43:25.0375473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0375829Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0376195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0376546Z return func(*args, **kwargs) 2025-12-04T09:43:25.0376896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0377277Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0377650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0378007Z return func(*args, **kwargs) 2025-12-04T09:43:25.0378380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.0378746Z value_states = self.v(current_states) 2025-12-04T09:43:25.0378872Z 2025-12-04T09:43:25.0378960Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0379158Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0379387Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0379742Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0380047Z res = mod(**inputs) 2025-12-04T09:43:25.0380408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0380780Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0381147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0381523Z layer_outputs = layer_module( 2025-12-04T09:43:25.0381867Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0382226Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0382593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0382969Z return func(*args, **kwargs) 2025-12-04T09:43:25.0383323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0383696Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0384063Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0384430Z return func(*args, **kwargs) 2025-12-04T09:43:25.0384786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0385168Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0385549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0385912Z return func(*args, **kwargs) 2025-12-04T09:43:25.0386261Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.0386625Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.0386757Z 2025-12-04T09:43:25.0386862Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0387216Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0387529Z res = mod(**inputs) 2025-12-04T09:43:25.0387865Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0388234Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0388599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0388965Z layer_outputs = layer_module( 2025-12-04T09:43:25.0389316Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0389674Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0390049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0390407Z return func(*args, **kwargs) 2025-12-04T09:43:25.0390762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0391152Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0391537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0391982Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0392395Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:43:25.0392792Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:43:25.0392942Z 2025-12-04T09:43:25.0393046Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0393472Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0393841Z res = mod(**inputs) 2025-12-04T09:43:25.0394191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0394565Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0394936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0395327Z layer_outputs = layer_module( 2025-12-04T09:43:25.0395666Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0396031Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0396406Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0396772Z return func(*args, **kwargs) 2025-12-04T09:43:25.0397121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0397514Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0397895Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0398300Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0398712Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:43:25.0399088Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:43:25.0399220Z 2025-12-04T09:43:25.0399329Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0399677Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0399993Z res = mod(**inputs) 2025-12-04T09:43:25.0400336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0400715Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0401069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0401441Z layer_outputs = layer_module( 2025-12-04T09:43:25.0401779Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0402156Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0402530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0402898Z return func(*args, **kwargs) 2025-12-04T09:43:25.0403253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0403633Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0404019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0404429Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0404825Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:43:25.0405203Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:43:25.0405349Z 2025-12-04T09:43:25.0405479Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0406163Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0406482Z res = mod(**inputs) 2025-12-04T09:43:25.0406830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0407203Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0407569Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0407996Z layer_outputs = layer_module( 2025-12-04T09:43:25.0408341Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0408701Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0409102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0409474Z return func(*args, **kwargs) 2025-12-04T09:43:25.0409834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0410226Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0410603Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0411024Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0411438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:43:25.0411814Z hidden_states = self.wo(hidden_states) 2025-12-04T09:43:25.0411950Z 2025-12-04T09:43:25.0412051Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0412408Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0412732Z res = mod(**inputs) 2025-12-04T09:43:25.0413070Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0413442Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0413812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0414199Z layer_outputs = layer_module( 2025-12-04T09:43:25.0414533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0414890Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0415266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0415624Z return func(*args, **kwargs) 2025-12-04T09:43:25.0415988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0416416Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0416788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0417143Z return func(*args, **kwargs) 2025-12-04T09:43:25.0417487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0417857Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0418230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0418587Z return func(*args, **kwargs) 2025-12-04T09:43:25.0418940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.0419324Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.0419452Z 2025-12-04T09:43:25.0419600Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0419945Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0420255Z res = mod(**inputs) 2025-12-04T09:43:25.0420594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0420952Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0421314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0421705Z layer_outputs = layer_module( 2025-12-04T09:43:25.0422032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0422385Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0422767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0423125Z return func(*args, **kwargs) 2025-12-04T09:43:25.0423466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0423836Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0424214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0424575Z return func(*args, **kwargs) 2025-12-04T09:43:25.0424910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0425279Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0425642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0425991Z return func(*args, **kwargs) 2025-12-04T09:43:25.0426335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.0426702Z key_states = self.k(current_states) 2025-12-04T09:43:25.0426827Z 2025-12-04T09:43:25.0426933Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0427272Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0427584Z res = mod(**inputs) 2025-12-04T09:43:25.0427924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0428288Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0428651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0429011Z layer_outputs = layer_module( 2025-12-04T09:43:25.0429348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0429697Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0430063Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0430423Z return func(*args, **kwargs) 2025-12-04T09:43:25.0430775Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0431143Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0431526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0431882Z return func(*args, **kwargs) 2025-12-04T09:43:25.0432219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0432590Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0432984Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0433440Z return func(*args, **kwargs) 2025-12-04T09:43:25.0433802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.0434185Z value_states = self.v(current_states) 2025-12-04T09:43:25.0434321Z 2025-12-04T09:43:25.0434410Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0434618Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0434863Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0435206Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0435519Z res = mod(**inputs) 2025-12-04T09:43:25.0435855Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0436240Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0436596Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0436950Z layer_outputs = layer_module( 2025-12-04T09:43:25.0437283Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0437639Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0438000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0438351Z return func(*args, **kwargs) 2025-12-04T09:43:25.0438695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0439066Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0439428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0439785Z return func(*args, **kwargs) 2025-12-04T09:43:25.0440131Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0440501Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0440856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0441211Z return func(*args, **kwargs) 2025-12-04T09:43:25.0441556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.0441923Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.0442045Z 2025-12-04T09:43:25.0442146Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0442495Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0442808Z res = mod(**inputs) 2025-12-04T09:43:25.0443134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0443495Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0443849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0444204Z layer_outputs = layer_module( 2025-12-04T09:43:25.0444521Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0444873Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0445237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0445585Z return func(*args, **kwargs) 2025-12-04T09:43:25.0445935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0446360Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0446726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0447112Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0447506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:43:25.0447880Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:43:25.0448042Z 2025-12-04T09:43:25.0448147Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0448480Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0448787Z res = mod(**inputs) 2025-12-04T09:43:25.0449120Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0449494Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0449853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0450211Z layer_outputs = layer_module( 2025-12-04T09:43:25.0450541Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0450880Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0451247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0451604Z return func(*args, **kwargs) 2025-12-04T09:43:25.0451944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0452327Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0452709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0453114Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0453498Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:43:25.0453865Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:43:25.0454005Z 2025-12-04T09:43:25.0454105Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0454448Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0454751Z res = mod(**inputs) 2025-12-04T09:43:25.0455086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0455449Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0455800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0456160Z layer_outputs = layer_module( 2025-12-04T09:43:25.0456493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0456840Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0457201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0457558Z return func(*args, **kwargs) 2025-12-04T09:43:25.0457906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0458274Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0458648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0459050Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0459479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:43:25.0459852Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:43:25.0459994Z 2025-12-04T09:43:25.0460092Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0460430Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0460733Z res = mod(**inputs) 2025-12-04T09:43:25.0461057Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0461433Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0461786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0462152Z layer_outputs = layer_module( 2025-12-04T09:43:25.0462491Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0462836Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0463198Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0463547Z return func(*args, **kwargs) 2025-12-04T09:43:25.0463894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0464267Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0464634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0465032Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0465424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:43:25.0465794Z hidden_states = self.wo(hidden_states) 2025-12-04T09:43:25.0465920Z 2025-12-04T09:43:25.0466018Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0466357Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0466662Z res = mod(**inputs) 2025-12-04T09:43:25.0466995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0467348Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0467704Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0468065Z layer_outputs = layer_module( 2025-12-04T09:43:25.0468390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0468738Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0469108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0469460Z return func(*args, **kwargs) 2025-12-04T09:43:25.0469800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0470164Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0470528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0470886Z return func(*args, **kwargs) 2025-12-04T09:43:25.0471224Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0471594Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0471963Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0472312Z return func(*args, **kwargs) 2025-12-04T09:43:25.0472689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.0473096Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.0473299Z 2025-12-04T09:43:25.0473411Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0473766Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0474081Z res = mod(**inputs) 2025-12-04T09:43:25.0474463Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0474812Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0475173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0475556Z layer_outputs = layer_module( 2025-12-04T09:43:25.0475899Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0476257Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0476631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0476993Z return func(*args, **kwargs) 2025-12-04T09:43:25.0477337Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0477710Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0478083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0478440Z return func(*args, **kwargs) 2025-12-04T09:43:25.0478782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0479158Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0479528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0479888Z return func(*args, **kwargs) 2025-12-04T09:43:25.0480227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.0480589Z key_states = self.k(current_states) 2025-12-04T09:43:25.0480714Z 2025-12-04T09:43:25.0480819Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0481159Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0481470Z res = mod(**inputs) 2025-12-04T09:43:25.0481808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0482176Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0482526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0482887Z layer_outputs = layer_module( 2025-12-04T09:43:25.0483221Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0483566Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0483931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0484291Z return func(*args, **kwargs) 2025-12-04T09:43:25.0484644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0485011Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0485380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0485737Z return func(*args, **kwargs) 2025-12-04T09:43:25.0486112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0486497Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0486927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0487286Z return func(*args, **kwargs) 2025-12-04T09:43:25.0487622Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.0488003Z value_states = self.v(current_states) 2025-12-04T09:43:25.0488128Z 2025-12-04T09:43:25.0488211Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0488415Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0488632Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0488987Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0489301Z res = mod(**inputs) 2025-12-04T09:43:25.0489627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0489992Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0490349Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0490716Z layer_outputs = layer_module( 2025-12-04T09:43:25.0491038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0491390Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0491753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0492101Z return func(*args, **kwargs) 2025-12-04T09:43:25.0492451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0492813Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0493180Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0493529Z return func(*args, **kwargs) 2025-12-04T09:43:25.0493872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0494241Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0494601Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0494952Z return func(*args, **kwargs) 2025-12-04T09:43:25.0495291Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.0495654Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.0495779Z 2025-12-04T09:43:25.0495877Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0496219Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0496523Z res = mod(**inputs) 2025-12-04T09:43:25.0496849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0497210Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0497570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0497937Z layer_outputs = layer_module( 2025-12-04T09:43:25.0498261Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0498617Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0499024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0499387Z return func(*args, **kwargs) 2025-12-04T09:43:25.0499723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0500098Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0500478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0500871Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0501298Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:43:25.0501679Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:43:25.0501821Z 2025-12-04T09:43:25.0501928Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0502280Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0502591Z res = mod(**inputs) 2025-12-04T09:43:25.0502923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0503286Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0503633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0503995Z layer_outputs = layer_module( 2025-12-04T09:43:25.0504329Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0504670Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0505035Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0505394Z return func(*args, **kwargs) 2025-12-04T09:43:25.0505936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0506310Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0506684Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0507084Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0507469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:43:25.0507838Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:43:25.0507976Z 2025-12-04T09:43:25.0508075Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0508421Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0508723Z res = mod(**inputs) 2025-12-04T09:43:25.0509066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0509431Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0509778Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0510138Z layer_outputs = layer_module( 2025-12-04T09:43:25.0510473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0510819Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0511175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0511532Z return func(*args, **kwargs) 2025-12-04T09:43:25.0511878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0512258Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0512681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0513086Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0513545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:43:25.0513924Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:43:25.0514075Z 2025-12-04T09:43:25.0514179Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0514575Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0514892Z res = mod(**inputs) 2025-12-04T09:43:25.0515233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0515620Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0515980Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0516336Z layer_outputs = layer_module( 2025-12-04T09:43:25.0516660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0517007Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0517372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0517730Z return func(*args, **kwargs) 2025-12-04T09:43:25.0518079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0518457Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0518831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0519228Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0519632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:43:25.0520001Z hidden_states = self.wo(hidden_states) 2025-12-04T09:43:25.0520135Z 2025-12-04T09:43:25.0520233Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0520575Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0520883Z res = mod(**inputs) 2025-12-04T09:43:25.0521217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0521574Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0521930Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0522293Z layer_outputs = layer_module( 2025-12-04T09:43:25.0522633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0522973Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0523339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0523702Z return func(*args, **kwargs) 2025-12-04T09:43:25.0524040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0524411Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0524785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0525139Z return func(*args, **kwargs) 2025-12-04T09:43:25.0525479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0525915Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0526281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0526633Z return func(*args, **kwargs) 2025-12-04T09:43:25.0526977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.0527341Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.0527466Z 2025-12-04T09:43:25.0527587Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0527923Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0528233Z res = mod(**inputs) 2025-12-04T09:43:25.0528564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0528945Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0529303Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0529665Z layer_outputs = layer_module( 2025-12-04T09:43:25.0529998Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0530336Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0530702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0531062Z return func(*args, **kwargs) 2025-12-04T09:43:25.0531412Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0531774Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0532139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0532499Z return func(*args, **kwargs) 2025-12-04T09:43:25.0532837Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0533209Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0533577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0533933Z return func(*args, **kwargs) 2025-12-04T09:43:25.0534273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.0534644Z key_states = self.k(current_states) 2025-12-04T09:43:25.0534773Z 2025-12-04T09:43:25.0534885Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0535220Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0535531Z res = mod(**inputs) 2025-12-04T09:43:25.0535868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0536235Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0536579Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0536944Z layer_outputs = layer_module( 2025-12-04T09:43:25.0537279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0537629Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0537984Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0538342Z return func(*args, **kwargs) 2025-12-04T09:43:25.0538688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0539075Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0539444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0539797Z return func(*args, **kwargs) 2025-12-04T09:43:25.0540143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0540506Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0540873Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0541249Z return func(*args, **kwargs) 2025-12-04T09:43:25.0541585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.0541951Z value_states = self.v(current_states) 2025-12-04T09:43:25.0542113Z 2025-12-04T09:43:25.0542190Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0542394Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0542614Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0542950Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0543258Z res = mod(**inputs) 2025-12-04T09:43:25.0543583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0543945Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0544299Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0544657Z layer_outputs = layer_module( 2025-12-04T09:43:25.0544977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0545323Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0545686Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0546037Z return func(*args, **kwargs) 2025-12-04T09:43:25.0546378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0546739Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0547099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0547448Z return func(*args, **kwargs) 2025-12-04T09:43:25.0547797Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0548164Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0548528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0548877Z return func(*args, **kwargs) 2025-12-04T09:43:25.0549218Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.0549580Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.0549703Z 2025-12-04T09:43:25.0549800Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0550135Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0550440Z res = mod(**inputs) 2025-12-04T09:43:25.0550777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0551127Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0551494Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0551866Z layer_outputs = layer_module( 2025-12-04T09:43:25.0552227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0552578Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0552947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0553401Z return func(*args, **kwargs) 2025-12-04T09:43:25.0553772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0554223Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0554607Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0554962Z return func(*args, **kwargs) 2025-12-04T09:43:25.0555294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 489, in forward 2025-12-04T09:43:25.0555735Z hidden_states = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:43:25.0555914Z 2025-12-04T09:43:25.0556020Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0556357Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0556679Z res = mod(**inputs) 2025-12-04T09:43:25.0557016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0557377Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0557724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0558079Z layer_outputs = layer_module( 2025-12-04T09:43:25.0558411Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0558753Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0559121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0559475Z return func(*args, **kwargs) 2025-12-04T09:43:25.0559819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0560185Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0560558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0560957Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0561353Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:43:25.0561728Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:43:25.0561878Z 2025-12-04T09:43:25.0561978Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0562318Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0562616Z res = mod(**inputs) 2025-12-04T09:43:25.0562946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0563306Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0563657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0564007Z layer_outputs = layer_module( 2025-12-04T09:43:25.0564339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0564688Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0565045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0565405Z return func(*args, **kwargs) 2025-12-04T09:43:25.0565776Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0566152Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0566530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0566936Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0567329Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:43:25.0567712Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:43:25.0567840Z 2025-12-04T09:43:25.0567939Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0568282Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0568609Z res = mod(**inputs) 2025-12-04T09:43:25.0568935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0569295Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0569648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0570005Z layer_outputs = layer_module( 2025-12-04T09:43:25.0570333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0570679Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0571041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0571389Z return func(*args, **kwargs) 2025-12-04T09:43:25.0571737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0572115Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0572484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0572871Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0573261Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:43:25.0573634Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:43:25.0573768Z 2025-12-04T09:43:25.0573874Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0574207Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0574518Z res = mod(**inputs) 2025-12-04T09:43:25.0574854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0575212Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0575565Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0575924Z layer_outputs = layer_module( 2025-12-04T09:43:25.0576256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0576599Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0576962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0577324Z return func(*args, **kwargs) 2025-12-04T09:43:25.0577660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0578039Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0578439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0578839Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0579221Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:43:25.0579580Z hidden_states = self.wo(hidden_states) 2025-12-04T09:43:25.0579706Z 2025-12-04T09:43:25.0579812Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0580153Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0580472Z res = mod(**inputs) 2025-12-04T09:43:25.0580808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0581171Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0581521Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0581905Z layer_outputs = layer_module( 2025-12-04T09:43:25.0582237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0582584Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0582944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0583305Z return func(*args, **kwargs) 2025-12-04T09:43:25.0583654Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0584013Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0584380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0584739Z return func(*args, **kwargs) 2025-12-04T09:43:25.0585088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0585452Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0585816Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0586169Z return func(*args, **kwargs) 2025-12-04T09:43:25.0586509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.0586863Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.0586997Z 2025-12-04T09:43:25.0587095Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0587430Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0587732Z res = mod(**inputs) 2025-12-04T09:43:25.0588064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0588425Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0588780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0589132Z layer_outputs = layer_module( 2025-12-04T09:43:25.0589462Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0589804Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0590160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0590515Z return func(*args, **kwargs) 2025-12-04T09:43:25.0590862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0591224Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0591610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0591969Z return func(*args, **kwargs) 2025-12-04T09:43:25.0592307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0592669Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0593033Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0593466Z return func(*args, **kwargs) 2025-12-04T09:43:25.0593853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.0594215Z key_states = self.k(current_states) 2025-12-04T09:43:25.0594363Z 2025-12-04T09:43:25.0594463Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0594827Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0595144Z res = mod(**inputs) 2025-12-04T09:43:25.0595482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0595857Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0596221Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0596581Z layer_outputs = layer_module( 2025-12-04T09:43:25.0596923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0597281Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0597656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0598014Z return func(*args, **kwargs) 2025-12-04T09:43:25.0598374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0598751Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0599118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0599490Z return func(*args, **kwargs) 2025-12-04T09:43:25.0599847Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0600223Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0600592Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0600959Z return func(*args, **kwargs) 2025-12-04T09:43:25.0601317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.0601691Z value_states = self.v(current_states) 2025-12-04T09:43:25.0601821Z 2025-12-04T09:43:25.0601904Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0602113Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0602345Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0602685Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0603001Z res = mod(**inputs) 2025-12-04T09:43:25.0603349Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0603717Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0604076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0604444Z layer_outputs = layer_module( 2025-12-04T09:43:25.0604785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0605643Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0606208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0606585Z return func(*args, **kwargs) 2025-12-04T09:43:25.0606950Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0607378Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0607753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0608142Z return func(*args, **kwargs) 2025-12-04T09:43:25.0608474Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0608835Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0609235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0609591Z return func(*args, **kwargs) 2025-12-04T09:43:25.0609927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.0610288Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.0610411Z 2025-12-04T09:43:25.0610521Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0610858Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0611173Z res = mod(**inputs) 2025-12-04T09:43:25.0611507Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0611864Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0612208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0612569Z layer_outputs = layer_module( 2025-12-04T09:43:25.0612898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0613246Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0613601Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0613955Z return func(*args, **kwargs) 2025-12-04T09:43:25.0614296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0614669Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0615044Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0615445Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0615841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:43:25.0616221Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:43:25.0616370Z 2025-12-04T09:43:25.0616467Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0616808Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0617115Z res = mod(**inputs) 2025-12-04T09:43:25.0617448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0617816Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0618174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0618532Z layer_outputs = layer_module( 2025-12-04T09:43:25.0618865Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0619262Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0619633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0619985Z return func(*args, **kwargs) 2025-12-04T09:43:25.0620331Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0620709Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0621078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0621496Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0621891Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:43:25.0622275Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:43:25.0622403Z 2025-12-04T09:43:25.0622503Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0622845Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0623156Z res = mod(**inputs) 2025-12-04T09:43:25.0623483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0623843Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0624197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0624560Z layer_outputs = layer_module( 2025-12-04T09:43:25.0624887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0625236Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0625605Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0625960Z return func(*args, **kwargs) 2025-12-04T09:43:25.0626296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0626674Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0627046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0627436Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0627835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:43:25.0628206Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:43:25.0628339Z 2025-12-04T09:43:25.0628443Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0628777Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0629082Z res = mod(**inputs) 2025-12-04T09:43:25.0629418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0629777Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0630124Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0630483Z layer_outputs = layer_module( 2025-12-04T09:43:25.0630817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0631159Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0631525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0631889Z return func(*args, **kwargs) 2025-12-04T09:43:25.0632262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0632633Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0633005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0633501Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0633892Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:43:25.0634293Z hidden_states = self.wo(hidden_states) 2025-12-04T09:43:25.0634427Z 2025-12-04T09:43:25.0634524Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0634871Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0635178Z res = mod(**inputs) 2025-12-04T09:43:25.0635550Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0635932Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0636311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0636681Z layer_outputs = layer_module( 2025-12-04T09:43:25.0637037Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0637406Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0637782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0638159Z return func(*args, **kwargs) 2025-12-04T09:43:25.0638530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0638922Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0639309Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0639683Z return func(*args, **kwargs) 2025-12-04T09:43:25.0640055Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0640444Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0640948Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0641355Z return func(*args, **kwargs) 2025-12-04T09:43:25.0641725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.0642108Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.0642242Z 2025-12-04T09:43:25.0642347Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0642713Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0643034Z res = mod(**inputs) 2025-12-04T09:43:25.0643384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0643753Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0644123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0644497Z layer_outputs = layer_module( 2025-12-04T09:43:25.0644846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0645202Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0645579Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0645950Z return func(*args, **kwargs) 2025-12-04T09:43:25.0646342Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0646721Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0647104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0647472Z return func(*args, **kwargs) 2025-12-04T09:43:25.0647811Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0648198Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0648563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0648908Z return func(*args, **kwargs) 2025-12-04T09:43:25.0649250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.0649624Z key_states = self.k(current_states) 2025-12-04T09:43:25.0649750Z 2025-12-04T09:43:25.0649857Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0650190Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0650496Z res = mod(**inputs) 2025-12-04T09:43:25.0650835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0651196Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0651545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0651903Z layer_outputs = layer_module( 2025-12-04T09:43:25.0652237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0652581Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0652945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0653298Z return func(*args, **kwargs) 2025-12-04T09:43:25.0653641Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0653999Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0654364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0654719Z return func(*args, **kwargs) 2025-12-04T09:43:25.0655052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0655420Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0655787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0656142Z return func(*args, **kwargs) 2025-12-04T09:43:25.0656477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.0656839Z value_states = self.v(current_states) 2025-12-04T09:43:25.0656966Z 2025-12-04T09:43:25.0657051Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0657247Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0657475Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0657811Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0658120Z res = mod(**inputs) 2025-12-04T09:43:25.0658451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0658814Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0659169Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0659543Z layer_outputs = layer_module( 2025-12-04T09:43:25.0659876Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0660222Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0660580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0660929Z return func(*args, **kwargs) 2025-12-04T09:43:25.0661274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0661657Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0662028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0662392Z return func(*args, **kwargs) 2025-12-04T09:43:25.0662741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0663109Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0663467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0663821Z return func(*args, **kwargs) 2025-12-04T09:43:25.0664163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.0664526Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.0664650Z 2025-12-04T09:43:25.0664749Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0665093Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0665400Z res = mod(**inputs) 2025-12-04T09:43:25.0665731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0666096Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0666457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0666818Z layer_outputs = layer_module( 2025-12-04T09:43:25.0667142Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0667491Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0667856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0668206Z return func(*args, **kwargs) 2025-12-04T09:43:25.0668552Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0668912Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0669282Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0669635Z return func(*args, **kwargs) 2025-12-04T09:43:25.0669977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 489, in forward 2025-12-04T09:43:25.0670392Z hidden_states = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:43:25.0670568Z 2025-12-04T09:43:25.0670675Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0671012Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0671322Z res = mod(**inputs) 2025-12-04T09:43:25.0671662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0672014Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0672403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0672770Z layer_outputs = layer_module( 2025-12-04T09:43:25.0673161Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0673527Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0673906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0674278Z return func(*args, **kwargs) 2025-12-04T09:43:25.0674652Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0675029Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0675406Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0675829Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0676225Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:43:25.0676620Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:43:25.0676776Z 2025-12-04T09:43:25.0676875Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0677218Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0677532Z res = mod(**inputs) 2025-12-04T09:43:25.0677871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0678253Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0678613Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0678980Z layer_outputs = layer_module( 2025-12-04T09:43:25.0679323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0679673Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0680045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0680414Z return func(*args, **kwargs) 2025-12-04T09:43:25.0680768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0681150Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0681534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0681949Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0682348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:43:25.0682724Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:43:25.0682862Z 2025-12-04T09:43:25.0682964Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0683315Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0683627Z res = mod(**inputs) 2025-12-04T09:43:25.0683969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0684335Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0684695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0685056Z layer_outputs = layer_module( 2025-12-04T09:43:25.0685398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0685758Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0686155Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0686520Z return func(*args, **kwargs) 2025-12-04T09:43:25.0686871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0687265Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0687639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0688063Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0688454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:43:25.0688820Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:43:25.0688955Z 2025-12-04T09:43:25.0689069Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0689407Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0689714Z res = mod(**inputs) 2025-12-04T09:43:25.0690037Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0690399Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0690752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0691108Z layer_outputs = layer_module( 2025-12-04T09:43:25.0691435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0691789Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0692151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0692502Z return func(*args, **kwargs) 2025-12-04T09:43:25.0692851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0693226Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0693594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0693985Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0694380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:43:25.0694748Z hidden_states = self.wo(hidden_states) 2025-12-04T09:43:25.0694875Z 2025-12-04T09:43:25.0694981Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0695314Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0695623Z res = mod(**inputs) 2025-12-04T09:43:25.0695956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0696314Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0696674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0697032Z layer_outputs = layer_module( 2025-12-04T09:43:25.0697364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0697705Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0698066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0698424Z return func(*args, **kwargs) 2025-12-04T09:43:25.0698766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0699134Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0699550Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0699912Z return func(*args, **kwargs) 2025-12-04T09:43:25.0700251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0700622Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0700989Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0701361Z return func(*args, **kwargs) 2025-12-04T09:43:25.0701710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.0702075Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.0702218Z 2025-12-04T09:43:25.0702321Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0702651Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0702956Z res = mod(**inputs) 2025-12-04T09:43:25.0703288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0703645Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0703989Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0704346Z layer_outputs = layer_module( 2025-12-04T09:43:25.0704677Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0705016Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0705378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0705888Z return func(*args, **kwargs) 2025-12-04T09:43:25.0706246Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0706609Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0706982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0707341Z return func(*args, **kwargs) 2025-12-04T09:43:25.0707679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0708048Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0708414Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0708771Z return func(*args, **kwargs) 2025-12-04T09:43:25.0709112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.0709477Z key_states = self.k(current_states) 2025-12-04T09:43:25.0709603Z 2025-12-04T09:43:25.0709711Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0710052Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0710353Z res = mod(**inputs) 2025-12-04T09:43:25.0710689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0711054Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0711402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0711759Z layer_outputs = layer_module( 2025-12-04T09:43:25.0712090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0712433Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0712851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0713305Z return func(*args, **kwargs) 2025-12-04T09:43:25.0713679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0714056Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0714453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0714839Z return func(*args, **kwargs) 2025-12-04T09:43:25.0715185Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0715548Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0715957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0716328Z return func(*args, **kwargs) 2025-12-04T09:43:25.0716681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.0717055Z value_states = self.v(current_states) 2025-12-04T09:43:25.0717194Z 2025-12-04T09:43:25.0717274Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0717487Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0717714Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0718077Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0718398Z res = mod(**inputs) 2025-12-04T09:43:25.0718737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0719110Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0719479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0719852Z layer_outputs = layer_module( 2025-12-04T09:43:25.0720189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0720551Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0720926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0721295Z return func(*args, **kwargs) 2025-12-04T09:43:25.0721646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0722025Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0722408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0722768Z return func(*args, **kwargs) 2025-12-04T09:43:25.0723128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0723508Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0723866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0724218Z return func(*args, **kwargs) 2025-12-04T09:43:25.0724556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.0724912Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.0725033Z 2025-12-04T09:43:25.0725131Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0725467Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0725771Z res = mod(**inputs) 2025-12-04T09:43:25.0726133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0726490Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0726844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0727207Z layer_outputs = layer_module( 2025-12-04T09:43:25.0727535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0727886Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0728266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0728617Z return func(*args, **kwargs) 2025-12-04T09:43:25.0728949Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0729370Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0729745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0730146Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0730535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:43:25.0730915Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:43:25.0731056Z 2025-12-04T09:43:25.0731161Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0731495Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0731802Z res = mod(**inputs) 2025-12-04T09:43:25.0732138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0732507Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0732862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0733224Z layer_outputs = layer_module( 2025-12-04T09:43:25.0733564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0733912Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0734267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0734626Z return func(*args, **kwargs) 2025-12-04T09:43:25.0734975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0735341Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0735712Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0736114Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0736512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:43:25.0736869Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:43:25.0737004Z 2025-12-04T09:43:25.0737101Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0737436Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0737738Z res = mod(**inputs) 2025-12-04T09:43:25.0738071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0738437Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0738791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0739147Z layer_outputs = layer_module( 2025-12-04T09:43:25.0739518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0739870Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0740240Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0740592Z return func(*args, **kwargs) 2025-12-04T09:43:25.0740934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0741325Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0741686Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0742083Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0742508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:43:25.0742881Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:43:25.0743019Z 2025-12-04T09:43:25.0743123Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0743478Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0743796Z res = mod(**inputs) 2025-12-04T09:43:25.0744145Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0744521Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0744885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0745248Z layer_outputs = layer_module( 2025-12-04T09:43:25.0745580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0745935Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0746308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0746669Z return func(*args, **kwargs) 2025-12-04T09:43:25.0747012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0747392Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0747768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0748165Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0748569Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:43:25.0748939Z hidden_states = self.wo(hidden_states) 2025-12-04T09:43:25.0749071Z 2025-12-04T09:43:25.0749184Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0749527Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0749844Z res = mod(**inputs) 2025-12-04T09:43:25.0750185Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0750545Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0750903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0751273Z layer_outputs = layer_module( 2025-12-04T09:43:25.0751608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0751951Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0752322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0752716Z return func(*args, **kwargs) 2025-12-04T09:43:25.0753069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0753510Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0753898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0754266Z return func(*args, **kwargs) 2025-12-04T09:43:25.0754614Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0755006Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0755429Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0755800Z return func(*args, **kwargs) 2025-12-04T09:43:25.0756168Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.0756540Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.0756670Z 2025-12-04T09:43:25.0756779Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0757117Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0757431Z res = mod(**inputs) 2025-12-04T09:43:25.0757769Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0758138Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0758494Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0758861Z layer_outputs = layer_module( 2025-12-04T09:43:25.0759203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0759562Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0759926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0760292Z return func(*args, **kwargs) 2025-12-04T09:43:25.0760641Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0761017Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0761396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0761761Z return func(*args, **kwargs) 2025-12-04T09:43:25.0762115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0762482Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0762860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0763222Z return func(*args, **kwargs) 2025-12-04T09:43:25.0763566Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.0763932Z key_states = self.k(current_states) 2025-12-04T09:43:25.0764068Z 2025-12-04T09:43:25.0764576Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0764932Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0765238Z res = mod(**inputs) 2025-12-04T09:43:25.0765582Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0765947Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0766308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0766710Z layer_outputs = layer_module( 2025-12-04T09:43:25.0767058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0767414Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0767778Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0768143Z return func(*args, **kwargs) 2025-12-04T09:43:25.0768485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0768884Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0769247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0769600Z return func(*args, **kwargs) 2025-12-04T09:43:25.0769960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0770310Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0770667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0771012Z return func(*args, **kwargs) 2025-12-04T09:43:25.0771349Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.0771696Z value_states = self.v(current_states) 2025-12-04T09:43:25.0771831Z 2025-12-04T09:43:25.0771907Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0772109Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0772322Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0772658Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0772963Z res = mod(**inputs) 2025-12-04T09:43:25.0773297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0773644Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0773997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0774351Z layer_outputs = layer_module( 2025-12-04T09:43:25.0774673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0775013Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0775374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0775726Z return func(*args, **kwargs) 2025-12-04T09:43:25.0776056Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0776420Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0776782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0777132Z return func(*args, **kwargs) 2025-12-04T09:43:25.0777464Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 478, in forward 2025-12-04T09:43:25.0777843Z normed_hidden_states = self.layer_norm(hidden_states) 2025-12-04T09:43:25.0778219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 147, in forward 2025-12-04T09:43:25.0778570Z return self.weight * hidden_states 2025-12-04T09:43:25.0778698Z 2025-12-04T09:43:25.0778795Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0779128Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0779433Z res = mod(**inputs) 2025-12-04T09:43:25.0779786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0780152Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0780510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0780864Z layer_outputs = layer_module( 2025-12-04T09:43:25.0781196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0781568Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0781929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0782277Z return func(*args, **kwargs) 2025-12-04T09:43:25.0782625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0783006Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0783369Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0783719Z return func(*args, **kwargs) 2025-12-04T09:43:25.0784061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0784429Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0784788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0785142Z return func(*args, **kwargs) 2025-12-04T09:43:25.0785482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.0785843Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.0785972Z 2025-12-04T09:43:25.0786069Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0786416Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0786729Z res = mod(**inputs) 2025-12-04T09:43:25.0787056Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0787417Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0787768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0788126Z layer_outputs = layer_module( 2025-12-04T09:43:25.0788452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0788799Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0789160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0789517Z return func(*args, **kwargs) 2025-12-04T09:43:25.0789853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0790220Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0790587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0790938Z return func(*args, **kwargs) 2025-12-04T09:43:25.0791280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0791647Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0792014Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0792363Z return func(*args, **kwargs) 2025-12-04T09:43:25.0792735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.0793102Z key_states = self.k(current_states) 2025-12-04T09:43:25.0793311Z 2025-12-04T09:43:25.0793426Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0793797Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0794148Z res = mod(**inputs) 2025-12-04T09:43:25.0794513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0794912Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0795275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0795636Z layer_outputs = layer_module( 2025-12-04T09:43:25.0795961Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0796325Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0796688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0797061Z return func(*args, **kwargs) 2025-12-04T09:43:25.0797400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0797765Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0798132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0798491Z return func(*args, **kwargs) 2025-12-04T09:43:25.0798829Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0799200Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0799569Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0799916Z return func(*args, **kwargs) 2025-12-04T09:43:25.0800263Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.0800623Z value_states = self.v(current_states) 2025-12-04T09:43:25.0800749Z 2025-12-04T09:43:25.0800831Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0801024Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0801249Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0801592Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0801895Z res = mod(**inputs) 2025-12-04T09:43:25.0802230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0802588Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0802943Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0803293Z layer_outputs = layer_module( 2025-12-04T09:43:25.0803627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0803976Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0804334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0804693Z return func(*args, **kwargs) 2025-12-04T09:43:25.0805040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0805406Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0805923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0806362Z return func(*args, **kwargs) 2025-12-04T09:43:25.0806702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0807062Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0807418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0807764Z return func(*args, **kwargs) 2025-12-04T09:43:25.0808105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.0808492Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.0808629Z 2025-12-04T09:43:25.0808729Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0809080Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0809413Z res = mod(**inputs) 2025-12-04T09:43:25.0809744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0810104Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0810455Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0810813Z layer_outputs = layer_module( 2025-12-04T09:43:25.0811151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0811500Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0811871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0812221Z return func(*args, **kwargs) 2025-12-04T09:43:25.0812563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.0812937Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.0813296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0813655Z return func(*args, **kwargs) 2025-12-04T09:43:25.0814000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.0814371Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.0814729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0815088Z return func(*args, **kwargs) 2025-12-04T09:43:25.0815431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.0815792Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.0815920Z 2025-12-04T09:43:25.0816020Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0816367Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0816674Z res = mod(**inputs) 2025-12-04T09:43:25.0816995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0817355Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0817711Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0818069Z layer_outputs = layer_module( 2025-12-04T09:43:25.0818396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0818739Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0819102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0819452Z return func(*args, **kwargs) 2025-12-04T09:43:25.0819870Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0820242Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0820609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0820966Z return func(*args, **kwargs) 2025-12-04T09:43:25.0821314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0821697Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0822066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0822422Z return func(*args, **kwargs) 2025-12-04T09:43:25.0822784Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.0823141Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.0823264Z 2025-12-04T09:43:25.0823361Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0823702Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0824002Z res = mod(**inputs) 2025-12-04T09:43:25.0824329Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0824676Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0825028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0825379Z layer_outputs = layer_module( 2025-12-04T09:43:25.0825697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0826040Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0826396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0826743Z return func(*args, **kwargs) 2025-12-04T09:43:25.0827087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0827444Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0827807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0828145Z return func(*args, **kwargs) 2025-12-04T09:43:25.0828483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 489, in forward 2025-12-04T09:43:25.0828886Z hidden_states = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:43:25.0829058Z 2025-12-04T09:43:25.0829165Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0829493Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0829794Z res = mod(**inputs) 2025-12-04T09:43:25.0830123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0830477Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0830818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0831174Z layer_outputs = layer_module( 2025-12-04T09:43:25.0831502Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0831833Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0832187Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0832571Z return func(*args, **kwargs) 2025-12-04T09:43:25.0832914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0833340Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0833732Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0834151Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0834565Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:43:25.0834964Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:43:25.0835116Z 2025-12-04T09:43:25.0835213Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0835549Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0835868Z res = mod(**inputs) 2025-12-04T09:43:25.0836207Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0836571Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0836926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0837283Z layer_outputs = layer_module( 2025-12-04T09:43:25.0837613Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0837965Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0838325Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0838684Z return func(*args, **kwargs) 2025-12-04T09:43:25.0839031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0839408Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0839775Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0840172Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0840575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:43:25.0840943Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:43:25.0841071Z 2025-12-04T09:43:25.0841167Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0841507Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0841819Z res = mod(**inputs) 2025-12-04T09:43:25.0842143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0842508Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0842859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0843217Z layer_outputs = layer_module( 2025-12-04T09:43:25.0843544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0843892Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0844256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0844612Z return func(*args, **kwargs) 2025-12-04T09:43:25.0844959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0845338Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0845758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0846154Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0846546Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:43:25.0846920Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:43:25.0847055Z 2025-12-04T09:43:25.0847158Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0847490Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0847822Z res = mod(**inputs) 2025-12-04T09:43:25.0848156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:43:25.0848511Z encoder_outputs = self.encoder( 2025-12-04T09:43:25.0848885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0849245Z layer_outputs = layer_module( 2025-12-04T09:43:25.0849586Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0849931Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0850292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0850646Z return func(*args, **kwargs) 2025-12-04T09:43:25.0850983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0851361Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0851736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0852134Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0852523Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:43:25.0852885Z hidden_states = self.wo(hidden_states) 2025-12-04T09:43:25.0853010Z 2025-12-04T09:43:25.0853116Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0853455Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0853754Z res = mod(**inputs) 2025-12-04T09:43:25.0854086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0854443Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0854790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0855148Z layer_outputs = layer_module( 2025-12-04T09:43:25.0855482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0855831Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0856194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0856550Z return func(*args, **kwargs) 2025-12-04T09:43:25.0856896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.0857258Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.0857625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0857978Z return func(*args, **kwargs) 2025-12-04T09:43:25.0858323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.0858687Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.0859088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0859451Z return func(*args, **kwargs) 2025-12-04T09:43:25.0859790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.0860153Z key_states = self.k(current_states) 2025-12-04T09:43:25.0860289Z 2025-12-04T09:43:25.0860386Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0860725Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0861044Z res = mod(**inputs) 2025-12-04T09:43:25.0861380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0861744Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0862113Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0862461Z layer_outputs = layer_module( 2025-12-04T09:43:25.0862790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0863131Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0863481Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0863833Z return func(*args, **kwargs) 2025-12-04T09:43:25.0864175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.0864533Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.0864887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0865238Z return func(*args, **kwargs) 2025-12-04T09:43:25.0865580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.0865938Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.0866305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0866652Z return func(*args, **kwargs) 2025-12-04T09:43:25.0866990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.0867346Z value_states = self.v(current_states) 2025-12-04T09:43:25.0867479Z 2025-12-04T09:43:25.0867555Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0867756Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0867969Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0868305Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0868611Z res = mod(**inputs) 2025-12-04T09:43:25.0868942Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0869287Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0869639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0869992Z layer_outputs = layer_module( 2025-12-04T09:43:25.0870311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0870657Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0871015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0871361Z return func(*args, **kwargs) 2025-12-04T09:43:25.0871727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.0872104Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.0872470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0872827Z return func(*args, **kwargs) 2025-12-04T09:43:25.0873234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.0873615Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.0874032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0874394Z return func(*args, **kwargs) 2025-12-04T09:43:25.0874756Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.0875148Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.0875274Z 2025-12-04T09:43:25.0875388Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0875722Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0876027Z res = mod(**inputs) 2025-12-04T09:43:25.0876361Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0876743Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0877108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0877479Z layer_outputs = layer_module( 2025-12-04T09:43:25.0877824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0878169Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0878550Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0878915Z return func(*args, **kwargs) 2025-12-04T09:43:25.0879265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0879646Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0880028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0880443Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0880845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:43:25.0881236Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:43:25.0881390Z 2025-12-04T09:43:25.0881489Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0881846Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0882155Z res = mod(**inputs) 2025-12-04T09:43:25.0882497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0882867Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0883228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0883597Z layer_outputs = layer_module( 2025-12-04T09:43:25.0883962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0884352Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0884743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0885136Z return func(*args, **kwargs) 2025-12-04T09:43:25.0885553Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0885967Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0886368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0886487Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0886748Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:43:25.0886844Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:43:25.0886848Z 2025-12-04T09:43:25.0886958Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0887174Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0887246Z res = mod(**inputs) 2025-12-04T09:43:25.0887512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0887584Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0887837Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0887907Z layer_outputs = layer_module( 2025-12-04T09:43:25.0888149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0888236Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0888492Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0888564Z return func(*args, **kwargs) 2025-12-04T09:43:25.0888791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0888877Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0889115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0889223Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0889458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:43:25.0889542Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:43:25.0889545Z 2025-12-04T09:43:25.0889642Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0889839Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0889900Z res = mod(**inputs) 2025-12-04T09:43:25.0890135Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0890211Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0890442Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0890516Z layer_outputs = layer_module( 2025-12-04T09:43:25.0890726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0890800Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0891036Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0891100Z return func(*args, **kwargs) 2025-12-04T09:43:25.0891331Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0891422Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0891651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0891767Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0892033Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:43:25.0892111Z hidden_states = self.wo(hidden_states) 2025-12-04T09:43:25.0892115Z 2025-12-04T09:43:25.0892220Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0892406Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0892474Z res = mod(**inputs) 2025-12-04T09:43:25.0892729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0892796Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0893031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0893112Z layer_outputs = layer_module( 2025-12-04T09:43:25.0893322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0893404Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0893628Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0893700Z return func(*args, **kwargs) 2025-12-04T09:43:25.0893918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0893997Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0894230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0894292Z return func(*args, **kwargs) 2025-12-04T09:43:25.0894521Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0894603Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0894826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0894895Z return func(*args, **kwargs) 2025-12-04T09:43:25.0895116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.0895190Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.0895194Z 2025-12-04T09:43:25.0895298Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0895483Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0895551Z res = mod(**inputs) 2025-12-04T09:43:25.0895775Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0895844Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0896079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0896144Z layer_outputs = layer_module( 2025-12-04T09:43:25.0896351Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0896432Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0896655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0896727Z return func(*args, **kwargs) 2025-12-04T09:43:25.0896947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0897023Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0897257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0897320Z return func(*args, **kwargs) 2025-12-04T09:43:25.0897580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0897657Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0897879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0897949Z return func(*args, **kwargs) 2025-12-04T09:43:25.0898171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.0898257Z key_states = self.k(current_states) 2025-12-04T09:43:25.0898260Z 2025-12-04T09:43:25.0898364Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0898549Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0898647Z res = mod(**inputs) 2025-12-04T09:43:25.0898878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0898946Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0899180Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0899245Z layer_outputs = layer_module( 2025-12-04T09:43:25.0899451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0899535Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0899758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0899829Z return func(*args, **kwargs) 2025-12-04T09:43:25.0900052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0900127Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0900362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0900426Z return func(*args, **kwargs) 2025-12-04T09:43:25.0900654Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0900730Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0900956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0901028Z return func(*args, **kwargs) 2025-12-04T09:43:25.0901251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.0901324Z value_states = self.v(current_states) 2025-12-04T09:43:25.0901328Z 2025-12-04T09:43:25.0901412Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0901489Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0901597Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0901779Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0901839Z res = mod(**inputs) 2025-12-04T09:43:25.0902074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0902142Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0902366Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0902440Z layer_outputs = layer_module( 2025-12-04T09:43:25.0902645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0902726Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0902980Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0903048Z return func(*args, **kwargs) 2025-12-04T09:43:25.0903289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0903364Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0903603Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0903674Z return func(*args, **kwargs) 2025-12-04T09:43:25.0903921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0904006Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0904233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0904314Z return func(*args, **kwargs) 2025-12-04T09:43:25.0904552Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.0904625Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.0904629Z 2025-12-04T09:43:25.0904733Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0904917Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0904977Z res = mod(**inputs) 2025-12-04T09:43:25.0905215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0905284Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0905515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0905589Z layer_outputs = layer_module( 2025-12-04T09:43:25.0905947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0906035Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0906261Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0906326Z return func(*args, **kwargs) 2025-12-04T09:43:25.0906557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.0906633Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.0906856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0906929Z return func(*args, **kwargs) 2025-12-04T09:43:25.0907152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.0907241Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.0907466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0907531Z return func(*args, **kwargs) 2025-12-04T09:43:25.0907763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.0907836Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.0907840Z 2025-12-04T09:43:25.0907949Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0908134Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0908192Z res = mod(**inputs) 2025-12-04T09:43:25.0908427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0908496Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0908778Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0908853Z layer_outputs = layer_module( 2025-12-04T09:43:25.0909065Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0909146Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0909376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0909438Z return func(*args, **kwargs) 2025-12-04T09:43:25.0909728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.0909803Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.0910038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0910135Z return func(*args, **kwargs) 2025-12-04T09:43:25.0910367Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.0910453Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.0910685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0910748Z return func(*args, **kwargs) 2025-12-04T09:43:25.0910983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.0911056Z key_states = self.k(current_states) 2025-12-04T09:43:25.0911059Z 2025-12-04T09:43:25.0911161Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0911345Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0911406Z res = mod(**inputs) 2025-12-04T09:43:25.0911648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0911715Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0911944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0912017Z layer_outputs = layer_module( 2025-12-04T09:43:25.0912227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0912305Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0912533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0912596Z return func(*args, **kwargs) 2025-12-04T09:43:25.0912831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.0912909Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.0913194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0913266Z return func(*args, **kwargs) 2025-12-04T09:43:25.0913493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.0913577Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.0913804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0913868Z return func(*args, **kwargs) 2025-12-04T09:43:25.0914105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.0914177Z value_states = self.v(current_states) 2025-12-04T09:43:25.0914181Z 2025-12-04T09:43:25.0914266Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0914341Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0914469Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0914661Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0914722Z res = mod(**inputs) 2025-12-04T09:43:25.0914965Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0915043Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0915278Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0915375Z layer_outputs = layer_module( 2025-12-04T09:43:25.0915602Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0915675Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0915930Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0915993Z return func(*args, **kwargs) 2025-12-04T09:43:25.0916222Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.0916304Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.0916532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0916603Z return func(*args, **kwargs) 2025-12-04T09:43:25.0916833Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.0916911Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.0917148Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0917213Z return func(*args, **kwargs) 2025-12-04T09:43:25.0917453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.0917526Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.0917529Z 2025-12-04T09:43:25.0917625Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0917818Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0917880Z res = mod(**inputs) 2025-12-04T09:43:25.0918109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0918186Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0918417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0918491Z layer_outputs = layer_module( 2025-12-04T09:43:25.0918706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0918778Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0919015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0919079Z return func(*args, **kwargs) 2025-12-04T09:43:25.0919305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0919400Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0919634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0919754Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0919979Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:43:25.0920073Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:43:25.0920105Z 2025-12-04T09:43:25.0920212Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0920398Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0920465Z res = mod(**inputs) 2025-12-04T09:43:25.0920697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0920765Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0921016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0921081Z layer_outputs = layer_module( 2025-12-04T09:43:25.0921293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0921390Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0921618Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0921690Z return func(*args, **kwargs) 2025-12-04T09:43:25.0921917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0922002Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0922235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0922344Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0922575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:43:25.0922650Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:43:25.0922653Z 2025-12-04T09:43:25.0922750Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0922947Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0923007Z res = mod(**inputs) 2025-12-04T09:43:25.0923238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0923314Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0923539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0923611Z layer_outputs = layer_module( 2025-12-04T09:43:25.0923830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0923901Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0924136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0924201Z return func(*args, **kwargs) 2025-12-04T09:43:25.0924434Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0924523Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0924751Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0924860Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0925089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:43:25.0925173Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:43:25.0925176Z 2025-12-04T09:43:25.0925281Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0925468Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0925537Z res = mod(**inputs) 2025-12-04T09:43:25.0925801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0925871Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0926108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0926176Z layer_outputs = layer_module( 2025-12-04T09:43:25.0926392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0926508Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0926737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0926806Z return func(*args, **kwargs) 2025-12-04T09:43:25.0927030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0927128Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0927367Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0927472Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0927709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:43:25.0927784Z hidden_states = self.wo(hidden_states) 2025-12-04T09:43:25.0927787Z 2025-12-04T09:43:25.0927884Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0928078Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0928136Z res = mod(**inputs) 2025-12-04T09:43:25.0928367Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0928447Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0928679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0928754Z layer_outputs = layer_module( 2025-12-04T09:43:25.0928964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0929037Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0929276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0929343Z return func(*args, **kwargs) 2025-12-04T09:43:25.0929570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0929652Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0929882Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0929954Z return func(*args, **kwargs) 2025-12-04T09:43:25.0930181Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0930257Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0930497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0930559Z return func(*args, **kwargs) 2025-12-04T09:43:25.0930792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.0930866Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.0930871Z 2025-12-04T09:43:25.0930969Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0931165Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0931226Z res = mod(**inputs) 2025-12-04T09:43:25.0931493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0931571Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0931797Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0931870Z layer_outputs = layer_module( 2025-12-04T09:43:25.0932080Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0932179Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0932416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0932479Z return func(*args, **kwargs) 2025-12-04T09:43:25.0932707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0932806Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0933025Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0933095Z return func(*args, **kwargs) 2025-12-04T09:43:25.0933316Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0933392Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0933621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0933684Z return func(*args, **kwargs) 2025-12-04T09:43:25.0933911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.0933983Z key_states = self.k(current_states) 2025-12-04T09:43:25.0933988Z 2025-12-04T09:43:25.0934083Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0934271Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0934332Z res = mod(**inputs) 2025-12-04T09:43:25.0934553Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0934628Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0934851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0934924Z layer_outputs = layer_module( 2025-12-04T09:43:25.0935128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0935199Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0935427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0935495Z return func(*args, **kwargs) 2025-12-04T09:43:25.0935725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0935800Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0936021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0936091Z return func(*args, **kwargs) 2025-12-04T09:43:25.0936316Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0936395Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0936625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0936688Z return func(*args, **kwargs) 2025-12-04T09:43:25.0936946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.0937019Z value_states = self.v(current_states) 2025-12-04T09:43:25.0937022Z 2025-12-04T09:43:25.0937097Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0937177Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0937273Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0937458Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0937525Z res = mod(**inputs) 2025-12-04T09:43:25.0937771Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0937845Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0938075Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0938157Z layer_outputs = layer_module( 2025-12-04T09:43:25.0938377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0938449Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0938678Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0938749Z return func(*args, **kwargs) 2025-12-04T09:43:25.0938974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0939056Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0939280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0939342Z return func(*args, **kwargs) 2025-12-04T09:43:25.0939573Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0939652Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0939885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0939947Z return func(*args, **kwargs) 2025-12-04T09:43:25.0940171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.0940251Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.0940254Z 2025-12-04T09:43:25.0940350Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0940533Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0940599Z res = mod(**inputs) 2025-12-04T09:43:25.0940828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0940905Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0941136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0941201Z layer_outputs = layer_module( 2025-12-04T09:43:25.0941418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0941490Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0941717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0941788Z return func(*args, **kwargs) 2025-12-04T09:43:25.0942011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0942095Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0942316Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0942378Z return func(*args, **kwargs) 2025-12-04T09:43:25.0942640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 489, in forward 2025-12-04T09:43:25.0942762Z hidden_states = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:43:25.0942766Z 2025-12-04T09:43:25.0942869Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0943050Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0943108Z res = mod(**inputs) 2025-12-04T09:43:25.0943363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0943432Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0943659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0943748Z layer_outputs = layer_module( 2025-12-04T09:43:25.0943957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0944037Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0944262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0944326Z return func(*args, **kwargs) 2025-12-04T09:43:25.0944557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.0944634Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.0944857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0944927Z return func(*args, **kwargs) 2025-12-04T09:43:25.0945150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.0945240Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.0945464Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0945527Z return func(*args, **kwargs) 2025-12-04T09:43:25.0945760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.0945832Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.0945835Z 2025-12-04T09:43:25.0945937Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0946121Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0946182Z res = mod(**inputs) 2025-12-04T09:43:25.0946414Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0946482Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0946707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0946781Z layer_outputs = layer_module( 2025-12-04T09:43:25.0946987Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0947069Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0947292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0947358Z return func(*args, **kwargs) 2025-12-04T09:43:25.0947590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.0947665Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.0947898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0947996Z return func(*args, **kwargs) 2025-12-04T09:43:25.0948222Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.0948308Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.0948536Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0948600Z return func(*args, **kwargs) 2025-12-04T09:43:25.0948832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.0948916Z key_states = self.k(current_states) 2025-12-04T09:43:25.0948920Z 2025-12-04T09:43:25.0949025Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0949214Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0949289Z res = mod(**inputs) 2025-12-04T09:43:25.0949525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0949593Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0949821Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0949895Z layer_outputs = layer_module( 2025-12-04T09:43:25.0950104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0950183Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0950409Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0950473Z return func(*args, **kwargs) 2025-12-04T09:43:25.0950706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.0950786Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.0951020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0951083Z return func(*args, **kwargs) 2025-12-04T09:43:25.0951308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.0951391Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.0951616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0951679Z return func(*args, **kwargs) 2025-12-04T09:43:25.0951909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.0951982Z value_states = self.v(current_states) 2025-12-04T09:43:25.0951987Z 2025-12-04T09:43:25.0952072Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0952148Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0952243Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0952432Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0952491Z res = mod(**inputs) 2025-12-04T09:43:25.0952720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0952796Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0953028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0953101Z layer_outputs = layer_module( 2025-12-04T09:43:25.0953423Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0953499Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0953779Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0953845Z return func(*args, **kwargs) 2025-12-04T09:43:25.0954073Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.0954155Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.0954391Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0954481Z return func(*args, **kwargs) 2025-12-04T09:43:25.0954726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.0954805Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.0955077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0955160Z return func(*args, **kwargs) 2025-12-04T09:43:25.0955402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.0955476Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.0955480Z 2025-12-04T09:43:25.0955577Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0955773Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0955836Z res = mod(**inputs) 2025-12-04T09:43:25.0956072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0956150Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0956385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0956461Z layer_outputs = layer_module( 2025-12-04T09:43:25.0956677Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0956751Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0956998Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0957063Z return func(*args, **kwargs) 2025-12-04T09:43:25.0957295Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0957393Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0957626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0957750Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0957984Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:43:25.0958084Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:43:25.0958087Z 2025-12-04T09:43:25.0958197Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0958386Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0958455Z res = mod(**inputs) 2025-12-04T09:43:25.0958690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0958760Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0959003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0959070Z layer_outputs = layer_module( 2025-12-04T09:43:25.0959288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0959370Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0959639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0959715Z return func(*args, **kwargs) 2025-12-04T09:43:25.0959947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0960034Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0960276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0960401Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0960639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:43:25.0960715Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:43:25.0960736Z 2025-12-04T09:43:25.0960834Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0961036Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0961097Z res = mod(**inputs) 2025-12-04T09:43:25.0961332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0961411Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0961645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0961722Z layer_outputs = layer_module( 2025-12-04T09:43:25.0961940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0962016Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0962256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0962322Z return func(*args, **kwargs) 2025-12-04T09:43:25.0962556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0962648Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0962881Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0962998Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0963230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:43:25.0963313Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:43:25.0963316Z 2025-12-04T09:43:25.0963421Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0963610Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0963680Z res = mod(**inputs) 2025-12-04T09:43:25.0963914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0963984Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0964225Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0964292Z layer_outputs = layer_module( 2025-12-04T09:43:25.0964507Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0964590Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0964827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0964900Z return func(*args, **kwargs) 2025-12-04T09:43:25.0965133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0965252Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0965491Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0965599Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0965840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:43:25.0965916Z hidden_states = self.wo(hidden_states) 2025-12-04T09:43:25.0965933Z 2025-12-04T09:43:25.0966032Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0966230Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0966291Z res = mod(**inputs) 2025-12-04T09:43:25.0966525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0966620Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0966860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0966938Z layer_outputs = layer_module( 2025-12-04T09:43:25.0967157Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0967232Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0967471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0967538Z return func(*args, **kwargs) 2025-12-04T09:43:25.0967773Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0967858Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0968096Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0968169Z return func(*args, **kwargs) 2025-12-04T09:43:25.0968411Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0968489Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0968737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0968802Z return func(*args, **kwargs) 2025-12-04T09:43:25.0969051Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.0969124Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.0969127Z 2025-12-04T09:43:25.0969222Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0969417Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0969478Z res = mod(**inputs) 2025-12-04T09:43:25.0969709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0969785Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0970015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0970087Z layer_outputs = layer_module( 2025-12-04T09:43:25.0970300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0970373Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0970610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0970673Z return func(*args, **kwargs) 2025-12-04T09:43:25.0970902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0971012Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0971241Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0971313Z return func(*args, **kwargs) 2025-12-04T09:43:25.0971539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0971617Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0971866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0971930Z return func(*args, **kwargs) 2025-12-04T09:43:25.0972166Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.0972256Z key_states = self.k(current_states) 2025-12-04T09:43:25.0972259Z 2025-12-04T09:43:25.0972359Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0972554Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0972615Z res = mod(**inputs) 2025-12-04T09:43:25.0972847Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0972924Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0973157Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0973233Z layer_outputs = layer_module( 2025-12-04T09:43:25.0973446Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0973519Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0973758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0973825Z return func(*args, **kwargs) 2025-12-04T09:43:25.0974054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0974136Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0974368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0974437Z return func(*args, **kwargs) 2025-12-04T09:43:25.0974665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0974740Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0974975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0975039Z return func(*args, **kwargs) 2025-12-04T09:43:25.0975277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.0975350Z value_states = self.v(current_states) 2025-12-04T09:43:25.0975354Z 2025-12-04T09:43:25.0975427Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0975509Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0975604Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0975787Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0975857Z res = mod(**inputs) 2025-12-04T09:43:25.0976089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0976162Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0976393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0976459Z layer_outputs = layer_module( 2025-12-04T09:43:25.0976711Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0976784Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0977013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0977084Z return func(*args, **kwargs) 2025-12-04T09:43:25.0977309Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.0977406Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.0977629Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0977691Z return func(*args, **kwargs) 2025-12-04T09:43:25.0977918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.0978010Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.0978245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0978310Z return func(*args, **kwargs) 2025-12-04T09:43:25.0978537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.0978617Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.0978620Z 2025-12-04T09:43:25.0978717Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0978904Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0978973Z res = mod(**inputs) 2025-12-04T09:43:25.0979205Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0979281Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0979514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0979582Z layer_outputs = layer_module( 2025-12-04T09:43:25.0979798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0979870Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0980100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0980171Z return func(*args, **kwargs) 2025-12-04T09:43:25.0980397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.0980479Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.0980709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0980776Z return func(*args, **kwargs) 2025-12-04T09:43:25.0981012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.0981092Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.0981328Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0981390Z return func(*args, **kwargs) 2025-12-04T09:43:25.0981615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.0981695Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.0981698Z 2025-12-04T09:43:25.0981795Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0981981Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0982049Z res = mod(**inputs) 2025-12-04T09:43:25.0982312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0982388Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0982615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0982681Z layer_outputs = layer_module( 2025-12-04T09:43:25.0982900Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0982986Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0983216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0983286Z return func(*args, **kwargs) 2025-12-04T09:43:25.0983512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.0983612Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.0983841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0983904Z return func(*args, **kwargs) 2025-12-04T09:43:25.0984142Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.0984221Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.0984458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0984522Z return func(*args, **kwargs) 2025-12-04T09:43:25.0984749Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.0984829Z key_states = self.k(current_states) 2025-12-04T09:43:25.0984834Z 2025-12-04T09:43:25.0984931Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0985117Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0985186Z res = mod(**inputs) 2025-12-04T09:43:25.0985414Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0985490Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0985719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0985788Z layer_outputs = layer_module( 2025-12-04T09:43:25.0986006Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0986086Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0986316Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0986391Z return func(*args, **kwargs) 2025-12-04T09:43:25.0986620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.0986702Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.0986930Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0986993Z return func(*args, **kwargs) 2025-12-04T09:43:25.0987227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.0987306Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.0987539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0987601Z return func(*args, **kwargs) 2025-12-04T09:43:25.0988249Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.0988335Z value_states = self.v(current_states) 2025-12-04T09:43:25.0988338Z 2025-12-04T09:43:25.0988412Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0988483Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.0988587Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0988768Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0988835Z res = mod(**inputs) 2025-12-04T09:43:25.0989083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0989153Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0989396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0989479Z layer_outputs = layer_module( 2025-12-04T09:43:25.0989698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0989780Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0990020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0990093Z return func(*args, **kwargs) 2025-12-04T09:43:25.0990324Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.0990401Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.0990640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0990702Z return func(*args, **kwargs) 2025-12-04T09:43:25.0990936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.0991023Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.0991255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0991325Z return func(*args, **kwargs) 2025-12-04T09:43:25.0991559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.0991631Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.0991635Z 2025-12-04T09:43:25.0991738Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0991929Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0991997Z res = mod(**inputs) 2025-12-04T09:43:25.0992230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0992300Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0992542Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0992608Z layer_outputs = layer_module( 2025-12-04T09:43:25.0992823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0992903Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0993194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0993277Z return func(*args, **kwargs) 2025-12-04T09:43:25.0993511Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.0993588Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.0993833Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0993931Z return func(*args, **kwargs) 2025-12-04T09:43:25.0994172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 529, in forward 2025-12-04T09:43:25.0994306Z layer_output = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:43:25.0994310Z 2025-12-04T09:43:25.0994409Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0994603Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0994682Z res = mod(**inputs) 2025-12-04T09:43:25.0994918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0994996Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0995232Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0995333Z layer_outputs = layer_module( 2025-12-04T09:43:25.0995549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0995621Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0995860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0995922Z return func(*args, **kwargs) 2025-12-04T09:43:25.0996151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0996246Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0996484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0996603Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0996835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:43:25.0996929Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:43:25.0996932Z 2025-12-04T09:43:25.0997040Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0997231Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0997298Z res = mod(**inputs) 2025-12-04T09:43:25.0997530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.0997601Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.0997841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.0997907Z layer_outputs = layer_module( 2025-12-04T09:43:25.0998122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.0998206Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.0998436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.0998505Z return func(*args, **kwargs) 2025-12-04T09:43:25.0998734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.0998818Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.0999053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.0999162Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.0999397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:43:25.0999472Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:43:25.0999475Z 2025-12-04T09:43:25.0999616Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.0999817Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.0999879Z res = mod(**inputs) 2025-12-04T09:43:25.1000114Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1000193Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1000430Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1000521Z layer_outputs = layer_module( 2025-12-04T09:43:25.1000730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1000802Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1001039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1001118Z return func(*args, **kwargs) 2025-12-04T09:43:25.1001348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.1001440Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.1001668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.1001783Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.1002015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:43:25.1002097Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:43:25.1002100Z 2025-12-04T09:43:25.1002205Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1002396Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1002465Z res = mod(**inputs) 2025-12-04T09:43:25.1002699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1002766Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1003004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1003071Z layer_outputs = layer_module( 2025-12-04T09:43:25.1003280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1003360Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1003588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1003660Z return func(*args, **kwargs) 2025-12-04T09:43:25.1003895Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.1003976Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.1004211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.1004316Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.1004549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:43:25.1004624Z hidden_states = self.wo(hidden_states) 2025-12-04T09:43:25.1004629Z 2025-12-04T09:43:25.1004725Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1004921Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1004980Z res = mod(**inputs) 2025-12-04T09:43:25.1005210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1005318Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1005548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1005620Z layer_outputs = layer_module( 2025-12-04T09:43:25.1006010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1006088Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1006327Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1028836Z return func(*args, **kwargs) 2025-12-04T09:43:25.1029271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.1029362Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.1029768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1029854Z return func(*args, **kwargs) 2025-12-04T09:43:25.1030106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.1030203Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.1030446Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1030516Z return func(*args, **kwargs) 2025-12-04T09:43:25.1030762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.1030837Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.1030845Z 2025-12-04T09:43:25.1030956Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1031167Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1031237Z res = mod(**inputs) 2025-12-04T09:43:25.1031489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1031565Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1031801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1031881Z layer_outputs = layer_module( 2025-12-04T09:43:25.1032098Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1032178Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1032421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1032488Z return func(*args, **kwargs) 2025-12-04T09:43:25.1032732Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.1032810Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.1033041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1033188Z return func(*args, **kwargs) 2025-12-04T09:43:25.1033428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.1033516Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.1033745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1033810Z return func(*args, **kwargs) 2025-12-04T09:43:25.1034046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.1034121Z key_states = self.k(current_states) 2025-12-04T09:43:25.1034125Z 2025-12-04T09:43:25.1034292Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1034496Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1034558Z res = mod(**inputs) 2025-12-04T09:43:25.1034803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1034878Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1035111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1035212Z layer_outputs = layer_module( 2025-12-04T09:43:25.1035432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1035509Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1035771Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1035838Z return func(*args, **kwargs) 2025-12-04T09:43:25.1036076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.1036155Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.1036388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1036462Z return func(*args, **kwargs) 2025-12-04T09:43:25.1036691Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.1036775Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.1037002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1037067Z return func(*args, **kwargs) 2025-12-04T09:43:25.1037308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.1037381Z value_states = self.v(current_states) 2025-12-04T09:43:25.1037385Z 2025-12-04T09:43:25.1037463Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.1037544Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.1037643Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1037841Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1037904Z res = mod(**inputs) 2025-12-04T09:43:25.1038136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1038212Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1038440Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1038511Z layer_outputs = layer_module( 2025-12-04T09:43:25.1038734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1038809Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1039041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1039107Z return func(*args, **kwargs) 2025-12-04T09:43:25.1039334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.1039419Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.1039648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1039712Z return func(*args, **kwargs) 2025-12-04T09:43:25.1039977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.1040056Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.1040291Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1040353Z return func(*args, **kwargs) 2025-12-04T09:43:25.1040580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.1040661Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.1040704Z 2025-12-04T09:43:25.1040804Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1041000Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1041063Z res = mod(**inputs) 2025-12-04T09:43:25.1041297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1041392Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1041627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1041695Z layer_outputs = layer_module( 2025-12-04T09:43:25.1041917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1041992Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1042231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1042297Z return func(*args, **kwargs) 2025-12-04T09:43:25.1042529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.1042613Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.1042849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1042915Z return func(*args, **kwargs) 2025-12-04T09:43:25.1043152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.1043232Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.1043471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1043535Z return func(*args, **kwargs) 2025-12-04T09:43:25.1043766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.1043847Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.1043850Z 2025-12-04T09:43:25.1043951Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1044151Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1044216Z res = mod(**inputs) 2025-12-04T09:43:25.1044449Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1044526Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1044756Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1044823Z layer_outputs = layer_module( 2025-12-04T09:43:25.1045044Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1045119Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1045357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1045423Z return func(*args, **kwargs) 2025-12-04T09:43:25.1045680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.1045765Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.1045999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1046063Z return func(*args, **kwargs) 2025-12-04T09:43:25.1046296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.1046379Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.1046631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1046693Z return func(*args, **kwargs) 2025-12-04T09:43:25.1046917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.1047009Z key_states = self.k(current_states) 2025-12-04T09:43:25.1047013Z 2025-12-04T09:43:25.1047113Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1047305Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1047365Z res = mod(**inputs) 2025-12-04T09:43:25.1047597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1047673Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1047907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1047981Z layer_outputs = layer_module( 2025-12-04T09:43:25.1048191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1048262Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1048499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1048562Z return func(*args, **kwargs) 2025-12-04T09:43:25.1048791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.1048874Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.1049102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1049173Z return func(*args, **kwargs) 2025-12-04T09:43:25.1049402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.1049478Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.1049714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1049779Z return func(*args, **kwargs) 2025-12-04T09:43:25.1050010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.1050089Z value_states = self.v(current_states) 2025-12-04T09:43:25.1050093Z 2025-12-04T09:43:25.1050167Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.1050247Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.1050346Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1050532Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1050602Z res = mod(**inputs) 2025-12-04T09:43:25.1050839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1050909Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1051152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1051248Z layer_outputs = layer_module( 2025-12-04T09:43:25.1051469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1051543Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1051774Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1051846Z return func(*args, **kwargs) 2025-12-04T09:43:25.1052072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.1052174Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.1052405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1052470Z return func(*args, **kwargs) 2025-12-04T09:43:25.1052723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.1052802Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.1053029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1053101Z return func(*args, **kwargs) 2025-12-04T09:43:25.1053329Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.1053408Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.1053413Z 2025-12-04T09:43:25.1053510Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1053701Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1053771Z res = mod(**inputs) 2025-12-04T09:43:25.1054001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1054072Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1054314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1054380Z layer_outputs = layer_module( 2025-12-04T09:43:25.1054601Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1054674Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1054902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1054978Z return func(*args, **kwargs) 2025-12-04T09:43:25.1055204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.1055300Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.1055531Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.1055645Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.1055882Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:43:25.1055975Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:43:25.1055979Z 2025-12-04T09:43:25.1056075Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1056271Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1056334Z res = mod(**inputs) 2025-12-04T09:43:25.1056576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1056646Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1056876Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1056983Z layer_outputs = layer_module( 2025-12-04T09:43:25.1057195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1057278Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1057507Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1057571Z return func(*args, **kwargs) 2025-12-04T09:43:25.1057803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.1057906Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.1058132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.1058267Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.1058501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:43:25.1058582Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:43:25.1058585Z 2025-12-04T09:43:25.1058683Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1058868Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1058938Z res = mod(**inputs) 2025-12-04T09:43:25.1059170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1059239Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1059480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1059546Z layer_outputs = layer_module( 2025-12-04T09:43:25.1059773Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1059848Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1060078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1060149Z return func(*args, **kwargs) 2025-12-04T09:43:25.1060376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.1060467Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.1060696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.1060803Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.1061038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:43:25.1061121Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:43:25.1061125Z 2025-12-04T09:43:25.1061224Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1061416Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1061477Z res = mod(**inputs) 2025-12-04T09:43:25.1061717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1061784Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1062013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1062089Z layer_outputs = layer_module( 2025-12-04T09:43:25.1062299Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1062376Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1062646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1062711Z return func(*args, **kwargs) 2025-12-04T09:43:25.1062947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.1063029Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.1063257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.1063374Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.1063618Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:43:25.1063698Z hidden_states = self.wo(hidden_states) 2025-12-04T09:43:25.1063702Z 2025-12-04T09:43:25.1063799Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1064003Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1064069Z res = mod(**inputs) 2025-12-04T09:43:25.1064301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1064369Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1064608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1064673Z layer_outputs = layer_module( 2025-12-04T09:43:25.1064894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1064968Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1065198Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1065271Z return func(*args, **kwargs) 2025-12-04T09:43:25.1065500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.1065592Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.1065820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 218, in forward 2025-12-04T09:43:25.1065940Z hidden_states = hidden_states + self.dropout(forwarded_states) 2025-12-04T09:43:25.1065944Z 2025-12-04T09:43:25.1066048Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1066237Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1066297Z res = mod(**inputs) 2025-12-04T09:43:25.1066541Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1066610Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1066855Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1066921Z layer_outputs = layer_module( 2025-12-04T09:43:25.1067133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1067214Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1067445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1067516Z return func(*args, **kwargs) 2025-12-04T09:43:25.1067751Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.1067827Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.1068063Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1068127Z return func(*args, **kwargs) 2025-12-04T09:43:25.1068385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.1068472Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.1068702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1068770Z return func(*args, **kwargs) 2025-12-04T09:43:25.1068998Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.1069086Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.1069089Z 2025-12-04T09:43:25.1069191Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1069375Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1069435Z res = mod(**inputs) 2025-12-04T09:43:25.1069690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1069758Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1069996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1070061Z layer_outputs = layer_module( 2025-12-04T09:43:25.1070270Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1070349Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1070580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1070649Z return func(*args, **kwargs) 2025-12-04T09:43:25.1070874Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.1070950Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.1071187Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1071250Z return func(*args, **kwargs) 2025-12-04T09:43:25.1071477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.1071562Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.1071795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1071867Z return func(*args, **kwargs) 2025-12-04T09:43:25.1072097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.1072167Z key_states = self.k(current_states) 2025-12-04T09:43:25.1072171Z 2025-12-04T09:43:25.1072278Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1072468Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1072527Z res = mod(**inputs) 2025-12-04T09:43:25.1072766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1072834Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1073069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1073220Z layer_outputs = layer_module( 2025-12-04T09:43:25.1073443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1073527Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1073756Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1073828Z return func(*args, **kwargs) 2025-12-04T09:43:25.1074088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.1074164Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.1074401Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1074468Z return func(*args, **kwargs) 2025-12-04T09:43:25.1074703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.1074803Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.1075030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1075101Z return func(*args, **kwargs) 2025-12-04T09:43:25.1075326Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.1075420Z value_states = self.v(current_states) 2025-12-04T09:43:25.1075424Z 2025-12-04T09:43:25.1075508Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.1075582Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.1075680Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1075873Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1075934Z res = mod(**inputs) 2025-12-04T09:43:25.1076171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1076240Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1076469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1076541Z layer_outputs = layer_module( 2025-12-04T09:43:25.1076754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1076826Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1077062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1077125Z return func(*args, **kwargs) 2025-12-04T09:43:25.1077359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.1077433Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.1077663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1077734Z return func(*args, **kwargs) 2025-12-04T09:43:25.1077959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.1078043Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.1078275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1078337Z return func(*args, **kwargs) 2025-12-04T09:43:25.1078571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.1078641Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.1078644Z 2025-12-04T09:43:25.1078739Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1078929Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1078990Z res = mod(**inputs) 2025-12-04T09:43:25.1079226Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1079295Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1079558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1079634Z layer_outputs = layer_module( 2025-12-04T09:43:25.1079845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1079917Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1080154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1080225Z return func(*args, **kwargs) 2025-12-04T09:43:25.1080467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.1080541Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.1080775Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1080852Z return func(*args, **kwargs) 2025-12-04T09:43:25.1081090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.1081170Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.1081401Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1081474Z return func(*args, **kwargs) 2025-12-04T09:43:25.1081700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.1081783Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.1081787Z 2025-12-04T09:43:25.1081883Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1082068Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1082137Z res = mod(**inputs) 2025-12-04T09:43:25.1082374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1082442Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1082678Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1082744Z layer_outputs = layer_module( 2025-12-04T09:43:25.1082961Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1083032Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1083262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1083332Z return func(*args, **kwargs) 2025-12-04T09:43:25.1083562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.1083638Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.1083876Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1083938Z return func(*args, **kwargs) 2025-12-04T09:43:25.1084174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.1084252Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.1084481Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1084553Z return func(*args, **kwargs) 2025-12-04T09:43:25.1084781Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.1084856Z key_states = self.k(current_states) 2025-12-04T09:43:25.1084859Z 2025-12-04T09:43:25.1084954Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1085175Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1085243Z res = mod(**inputs) 2025-12-04T09:43:25.1085477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1085545Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1085785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1085848Z layer_outputs = layer_module( 2025-12-04T09:43:25.1086080Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1086151Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1086378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1086462Z return func(*args, **kwargs) 2025-12-04T09:43:25.1086692Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.1086773Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.1087006Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1087068Z return func(*args, **kwargs) 2025-12-04T09:43:25.1087299Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.1087377Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.1087608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1087677Z return func(*args, **kwargs) 2025-12-04T09:43:25.1087906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.1087987Z value_states = self.v(current_states) 2025-12-04T09:43:25.1087992Z 2025-12-04T09:43:25.1088066Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.1088140Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.1088245Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1088434Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1088493Z res = mod(**inputs) 2025-12-04T09:43:25.1088739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1088810Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1089050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1089118Z layer_outputs = layer_module( 2025-12-04T09:43:25.1089330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1089413Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1089645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1089708Z return func(*args, **kwargs) 2025-12-04T09:43:25.1089946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.1090021Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.1090259Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1090324Z return func(*args, **kwargs) 2025-12-04T09:43:25.1090552Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.1090639Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.1090897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1090968Z return func(*args, **kwargs) 2025-12-04T09:43:25.1091197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.1091270Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.1091273Z 2025-12-04T09:43:25.1091378Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1091565Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1091640Z res = mod(**inputs) 2025-12-04T09:43:25.1091884Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1091952Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1092206Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1092275Z layer_outputs = layer_module( 2025-12-04T09:43:25.1092486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1092565Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1092793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1092856Z return func(*args, **kwargs) 2025-12-04T09:43:25.1093093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.1093179Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.1093414Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.1093525Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.1093754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:43:25.1093855Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:43:25.1093858Z 2025-12-04T09:43:25.1093953Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1094147Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1094207Z res = mod(**inputs) 2025-12-04T09:43:25.1094438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1094516Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1094748Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1094813Z layer_outputs = layer_module( 2025-12-04T09:43:25.1095034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1095107Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1095343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1095408Z return func(*args, **kwargs) 2025-12-04T09:43:25.1095635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.1095729Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.1095957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.1096072Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.1096300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:43:25.1096374Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:43:25.1096404Z 2025-12-04T09:43:25.1096511Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1096697Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1096757Z res = mod(**inputs) 2025-12-04T09:43:25.1096996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1097064Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1097300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1097381Z layer_outputs = layer_module( 2025-12-04T09:43:25.1097593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1097686Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1097921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1097984Z return func(*args, **kwargs) 2025-12-04T09:43:25.1098221Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.1098304Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.1098538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.1098646Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.1098874Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:43:25.1098962Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:43:25.1098965Z 2025-12-04T09:43:25.1099063Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1099259Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1099320Z res = mod(**inputs) 2025-12-04T09:43:25.1099550Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1099625Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1099853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1099918Z layer_outputs = layer_module( 2025-12-04T09:43:25.1100137Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1100209Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1100444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1100510Z return func(*args, **kwargs) 2025-12-04T09:43:25.1100739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.1100828Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.1101057Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.1101171Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.1101400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:43:25.1101475Z hidden_states = self.wo(hidden_states) 2025-12-04T09:43:25.1101478Z 2025-12-04T09:43:25.1101580Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1101767Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1101828Z res = mod(**inputs) 2025-12-04T09:43:25.1102097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1102167Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1102403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1102469Z layer_outputs = layer_module( 2025-12-04T09:43:25.1102685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1102766Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1103012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1103075Z return func(*args, **kwargs) 2025-12-04T09:43:25.1103311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.1103409Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.1103648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1103712Z return func(*args, **kwargs) 2025-12-04T09:43:25.1103943Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.1104028Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.1104257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1104327Z return func(*args, **kwargs) 2025-12-04T09:43:25.1104556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.1104627Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.1104633Z 2025-12-04T09:43:25.1104737Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1104926Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1104986Z res = mod(**inputs) 2025-12-04T09:43:25.1105232Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1105300Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1105540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1105605Z layer_outputs = layer_module( 2025-12-04T09:43:25.1106006Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1106095Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1106330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1106397Z return func(*args, **kwargs) 2025-12-04T09:43:25.1106637Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.1106716Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.1106955Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1107019Z return func(*args, **kwargs) 2025-12-04T09:43:25.1107248Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.1107339Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.1107571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1107644Z return func(*args, **kwargs) 2025-12-04T09:43:25.1107873Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.1107997Z key_states = self.k(current_states) 2025-12-04T09:43:25.1108001Z 2025-12-04T09:43:25.1108109Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1108297Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1108357Z res = mod(**inputs) 2025-12-04T09:43:25.1108596Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1108664Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1108926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1108993Z layer_outputs = layer_module( 2025-12-04T09:43:25.1109205Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1109308Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1109543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1109607Z return func(*args, **kwargs) 2025-12-04T09:43:25.1109844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.1109920Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.1110158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1110223Z return func(*args, **kwargs) 2025-12-04T09:43:25.1110448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.1110533Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.1110766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1110839Z return func(*args, **kwargs) 2025-12-04T09:43:25.1111068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.1111140Z value_states = self.v(current_states) 2025-12-04T09:43:25.1111143Z 2025-12-04T09:43:25.1111225Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.1111299Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.1111397Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1111596Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1111655Z res = mod(**inputs) 2025-12-04T09:43:25.1111893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1111960Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1112197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1112271Z layer_outputs = layer_module( 2025-12-04T09:43:25.1112483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1112556Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1112792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1112854Z return func(*args, **kwargs) 2025-12-04T09:43:25.1113092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.1113212Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.1113451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1113523Z return func(*args, **kwargs) 2025-12-04T09:43:25.1113796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.1113878Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.1114110Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1114173Z return func(*args, **kwargs) 2025-12-04T09:43:25.1114405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.1114493Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.1114497Z 2025-12-04T09:43:25.1114592Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1114788Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1114847Z res = mod(**inputs) 2025-12-04T09:43:25.1115106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1115175Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1115409Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1115482Z layer_outputs = layer_module( 2025-12-04T09:43:25.1115704Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1115778Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1116018Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1116082Z return func(*args, **kwargs) 2025-12-04T09:43:25.1116315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.1116391Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.1116622Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1116693Z return func(*args, **kwargs) 2025-12-04T09:43:25.1116921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 489, in forward 2025-12-04T09:43:25.1117053Z hidden_states = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:43:25.1117056Z 2025-12-04T09:43:25.1117151Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1117338Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1117404Z res = mod(**inputs) 2025-12-04T09:43:25.1117637Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1117707Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1117947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1118014Z layer_outputs = layer_module( 2025-12-04T09:43:25.1118238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1118311Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1118540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1118611Z return func(*args, **kwargs) 2025-12-04T09:43:25.1118840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.1118914Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.1119149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1119211Z return func(*args, **kwargs) 2025-12-04T09:43:25.1119470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.1119548Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.1119779Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1119848Z return func(*args, **kwargs) 2025-12-04T09:43:25.1120073Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.1120159Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.1120169Z 2025-12-04T09:43:25.1120266Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1120453Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1120534Z res = mod(**inputs) 2025-12-04T09:43:25.1120768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1120836Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1121075Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1121140Z layer_outputs = layer_module( 2025-12-04T09:43:25.1121357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1121428Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1121666Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1121736Z return func(*args, **kwargs) 2025-12-04T09:43:25.1121964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.1122037Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.1122275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1122338Z return func(*args, **kwargs) 2025-12-04T09:43:25.1122573Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.1122650Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.1122877Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1122948Z return func(*args, **kwargs) 2025-12-04T09:43:25.1123171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.1123240Z key_states = self.k(current_states) 2025-12-04T09:43:25.1123249Z 2025-12-04T09:43:25.1123347Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1123533Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1123600Z res = mod(**inputs) 2025-12-04T09:43:25.1123832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1123898Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1124133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1124197Z layer_outputs = layer_module( 2025-12-04T09:43:25.1124414Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1124483Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1124710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1124780Z return func(*args, **kwargs) 2025-12-04T09:43:25.1125034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.1125111Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.1125346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1125408Z return func(*args, **kwargs) 2025-12-04T09:43:25.1125638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.1125738Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.1125966Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1126032Z return func(*args, **kwargs) 2025-12-04T09:43:25.1126264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.1126358Z value_states = self.v(current_states) 2025-12-04T09:43:25.1126362Z 2025-12-04T09:43:25.1126435Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.1126507Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.1126609Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1126796Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1126855Z res = mod(**inputs) 2025-12-04T09:43:25.1127094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1127161Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1127399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1127463Z layer_outputs = layer_module( 2025-12-04T09:43:25.1127684Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1127764Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1127996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1128058Z return func(*args, **kwargs) 2025-12-04T09:43:25.1128296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.1128371Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.1128609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1128671Z return func(*args, **kwargs) 2025-12-04T09:43:25.1128900Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.1128985Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.1129215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1129277Z return func(*args, **kwargs) 2025-12-04T09:43:25.1129511Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.1129583Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.1129586Z 2025-12-04T09:43:25.1129689Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1129875Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1129936Z res = mod(**inputs) 2025-12-04T09:43:25.1130174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1130242Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1130510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1130577Z layer_outputs = layer_module( 2025-12-04T09:43:25.1130789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1130868Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1131094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1131157Z return func(*args, **kwargs) 2025-12-04T09:43:25.1131421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.1131505Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.1131741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.1131865Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.1132087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:43:25.1132186Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:43:25.1132189Z 2025-12-04T09:43:25.1132284Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1132471Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1132529Z res = mod(**inputs) 2025-12-04T09:43:25.1132753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1132826Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1133049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1133113Z layer_outputs = layer_module( 2025-12-04T09:43:25.1133325Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1133396Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1133623Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1133685Z return func(*args, **kwargs) 2025-12-04T09:43:25.1133907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.1134000Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.1134230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.1134335Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.1134561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:43:25.1134638Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:43:25.1134641Z 2025-12-04T09:43:25.1134743Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1134930Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1134989Z res = mod(**inputs) 2025-12-04T09:43:25.1135220Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1135286Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1135520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1135584Z layer_outputs = layer_module( 2025-12-04T09:43:25.1135788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1135867Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1136115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1136181Z return func(*args, **kwargs) 2025-12-04T09:43:25.1136415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.1136499Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.1136736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.1136855Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.1137079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:43:25.1137167Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:43:25.1137185Z 2025-12-04T09:43:25.1137280Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1137472Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1137531Z res = mod(**inputs) 2025-12-04T09:43:25.1137763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1137837Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1138067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1138133Z layer_outputs = layer_module( 2025-12-04T09:43:25.1138347Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1138419Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1138652Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1138718Z return func(*args, **kwargs) 2025-12-04T09:43:25.1138946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.1139035Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.1139260Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.1139362Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.1139594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:43:25.1139670Z hidden_states = self.wo(hidden_states) 2025-12-04T09:43:25.1139673Z 2025-12-04T09:43:25.1139777Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1139966Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1140027Z res = mod(**inputs) 2025-12-04T09:43:25.1140268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1140336Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1140572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1140635Z layer_outputs = layer_module( 2025-12-04T09:43:25.1140840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1140920Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1141146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1141207Z return func(*args, **kwargs) 2025-12-04T09:43:25.1141441Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.1141544Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.1141780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1141845Z return func(*args, **kwargs) 2025-12-04T09:43:25.1142072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.1142155Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.1142385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1142463Z return func(*args, **kwargs) 2025-12-04T09:43:25.1142695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.1142767Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.1142785Z 2025-12-04T09:43:25.1142887Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1143072Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1143133Z res = mod(**inputs) 2025-12-04T09:43:25.1143368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1143434Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1143668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1143734Z layer_outputs = layer_module( 2025-12-04T09:43:25.1143940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1144020Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1144245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1144313Z return func(*args, **kwargs) 2025-12-04T09:43:25.1144548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.1144622Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.1144855Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1144918Z return func(*args, **kwargs) 2025-12-04T09:43:25.1145145Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.1145230Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.1145458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1145520Z return func(*args, **kwargs) 2025-12-04T09:43:25.1145753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.1145821Z key_states = self.k(current_states) 2025-12-04T09:43:25.1145824Z 2025-12-04T09:43:25.1145928Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1146110Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1146169Z res = mod(**inputs) 2025-12-04T09:43:25.1146405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1146473Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1146708Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1146772Z layer_outputs = layer_module( 2025-12-04T09:43:25.1146981Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1147096Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1147335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1147397Z return func(*args, **kwargs) 2025-12-04T09:43:25.1147632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.1147706Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.1147939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1148017Z return func(*args, **kwargs) 2025-12-04T09:43:25.1148247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.1148329Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.1148577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1148640Z return func(*args, **kwargs) 2025-12-04T09:43:25.1148873Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.1148943Z value_states = self.v(current_states) 2025-12-04T09:43:25.1148947Z 2025-12-04T09:43:25.1149027Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.1149099Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.1149194Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1149388Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1149448Z res = mod(**inputs) 2025-12-04T09:43:25.1149694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1149764Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1149997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1150068Z layer_outputs = layer_module( 2025-12-04T09:43:25.1150277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1150348Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1150586Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1150651Z return func(*args, **kwargs) 2025-12-04T09:43:25.1150886Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:43:25.1150960Z self_attention_outputs = self.layer[0]( 2025-12-04T09:43:25.1151190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1151261Z return func(*args, **kwargs) 2025-12-04T09:43:25.1151490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:43:25.1151565Z attention_output = self.SelfAttention( 2025-12-04T09:43:25.1151801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1151861Z return func(*args, **kwargs) 2025-12-04T09:43:25.1152097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.1152168Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.1152171Z 2025-12-04T09:43:25.1152267Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1152459Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1152519Z res = mod(**inputs) 2025-12-04T09:43:25.1152783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1152850Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1153073Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1153205Z layer_outputs = layer_module( 2025-12-04T09:43:25.1153425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1153517Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1153753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1153815Z return func(*args, **kwargs) 2025-12-04T09:43:25.1154051Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.1154142Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.1154373Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1154442Z return func(*args, **kwargs) 2025-12-04T09:43:25.1154670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.1154748Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.1154992Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1155057Z return func(*args, **kwargs) 2025-12-04T09:43:25.1155297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:43:25.1155370Z query_states = self.q(hidden_states) 2025-12-04T09:43:25.1155375Z 2025-12-04T09:43:25.1155471Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1155668Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1155728Z res = mod(**inputs) 2025-12-04T09:43:25.1155971Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1156039Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1156268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1156342Z layer_outputs = layer_module( 2025-12-04T09:43:25.1156554Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1156628Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1156862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1156927Z return func(*args, **kwargs) 2025-12-04T09:43:25.1157165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.1157240Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.1157468Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1157538Z return func(*args, **kwargs) 2025-12-04T09:43:25.1157768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.1157846Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.1158083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1158146Z return func(*args, **kwargs) 2025-12-04T09:43:25.1158410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:43:25.1158482Z key_states = self.k(current_states) 2025-12-04T09:43:25.1158485Z 2025-12-04T09:43:25.1158579Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1158768Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1158828Z res = mod(**inputs) 2025-12-04T09:43:25.1159073Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1159155Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1159387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1159459Z layer_outputs = layer_module( 2025-12-04T09:43:25.1159669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1159753Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1159987Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1160049Z return func(*args, **kwargs) 2025-12-04T09:43:25.1160280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.1160355Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.1160577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1160648Z return func(*args, **kwargs) 2025-12-04T09:43:25.1160871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.1160946Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.1161178Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1161240Z return func(*args, **kwargs) 2025-12-04T09:43:25.1161470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:43:25.1161540Z value_states = self.v(current_states) 2025-12-04T09:43:25.1161544Z 2025-12-04T09:43:25.1161616Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.1161695Z cudagraph partition due to non gpu ops 2025-12-04T09:43:25.1161791Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1161982Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1162041Z res = mod(**inputs) 2025-12-04T09:43:25.1162265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1162343Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1162570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1162635Z layer_outputs = layer_module( 2025-12-04T09:43:25.1162849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1162918Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1163146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1163209Z return func(*args, **kwargs) 2025-12-04T09:43:25.1163431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.1163514Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.1163741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1163803Z return func(*args, **kwargs) 2025-12-04T09:43:25.1164061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:43:25.1164138Z attention_output = self.EncDecAttention( 2025-12-04T09:43:25.1164373Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1164435Z return func(*args, **kwargs) 2025-12-04T09:43:25.1164662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:43:25.1164754Z attn_output = self.o(attn_output) 2025-12-04T09:43:25.1164758Z 2025-12-04T09:43:25.1164854Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1165052Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1165128Z res = mod(**inputs) 2025-12-04T09:43:25.1165365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1165438Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1165668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1165732Z layer_outputs = layer_module( 2025-12-04T09:43:25.1165947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1166019Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1166253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1166316Z return func(*args, **kwargs) 2025-12-04T09:43:25.1166541Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:43:25.1166621Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:43:25.1166849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1166910Z return func(*args, **kwargs) 2025-12-04T09:43:25.1167144Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 529, in forward 2025-12-04T09:43:25.1167267Z layer_output = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:43:25.1167270Z 2025-12-04T09:43:25.1167374Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1167563Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1167621Z res = mod(**inputs) 2025-12-04T09:43:25.1167860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1167929Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1168167Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1168233Z layer_outputs = layer_module( 2025-12-04T09:43:25.1168444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1168521Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1168752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1168814Z return func(*args, **kwargs) 2025-12-04T09:43:25.1169048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.1169132Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.1169364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.1169507Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.1169734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:43:25.1169832Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:43:25.1169835Z 2025-12-04T09:43:25.1169932Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1170121Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1170196Z res = mod(**inputs) 2025-12-04T09:43:25.1170426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1170499Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1170729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1170810Z layer_outputs = layer_module( 2025-12-04T09:43:25.1171031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1171102Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1171343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1171407Z return func(*args, **kwargs) 2025-12-04T09:43:25.1171637Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.1171731Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.1171965Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.1172074Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.1172316Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:43:25.1172392Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:43:25.1172395Z 2025-12-04T09:43:25.1172498Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1172685Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1172745Z res = mod(**inputs) 2025-12-04T09:43:25.1172985Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1173054Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1173292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1173358Z layer_outputs = layer_module( 2025-12-04T09:43:25.1173572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1173655Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1173887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1173950Z return func(*args, **kwargs) 2025-12-04T09:43:25.1174186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.1174269Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.1174504Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.1174612Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.1174840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:43:25.1174926Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:43:25.1174931Z 2025-12-04T09:43:25.1175054Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1175248Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1175308Z res = mod(**inputs) 2025-12-04T09:43:25.1175539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:43:25.1175612Z decoder_outputs = self.decoder( 2025-12-04T09:43:25.1175840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:43:25.1175921Z layer_outputs = layer_module( 2025-12-04T09:43:25.1176141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:25.1176213Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:25.1176449Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:25.1176530Z return func(*args, **kwargs) 2025-12-04T09:43:25.1176764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:43:25.1176855Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:43:25.1177086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:43:25.1177193Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:43:25.1177430Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:43:25.1177502Z hidden_states = self.wo(hidden_states) 2025-12-04T09:43:25.1177506Z 2025-12-04T09:43:25.1177610Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1177794Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1177859Z res = mod(**inputs) 2025-12-04T09:43:25.1178102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1816, in forward 2025-12-04T09:43:25.1178183Z lm_logits = self.lm_head(sequence_output) 2025-12-04T09:43:25.1178186Z 2025-12-04T09:43:25.1178290Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:25.1178476Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:25.1178534Z res = mod(**inputs) 2025-12-04T09:43:25.1178780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1823, in forward 2025-12-04T09:43:25.1178913Z loss = loss_fct(lm_logits.view(-1, lm_logits.size(-1)), labels.view(-1)) 2025-12-04T09:43:25.1178916Z 2025-12-04T09:43:34.8280881Z Compilation time (from dynamo_timed): 22.685492013 2025-12-04T09:43:34.8438991Z pass 2025-12-04T09:43:34.8443785Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:43:34.8448596Z TIMING: _recursive_pre_grad_passes:0.31932 _recursive_joint_graph_passes:0.72744 _recursive_post_grad_passes:0.07583 async_compile.wait:0.74697 code_gen:9.79357 inductor_compile:11.95366 backend_compile:18.9661 gc:0.00018 entire_frame_compile:22.68549 total_wall_time:22.68549 2025-12-04T09:43:34.8450104Z STATS: call_* op count: 1189 | FakeTensorMode.__torch_dispatch__:32815 | FakeTensor.__torch_dispatch__:5771 | ProxyTorchDispatchMode.__torch_dispatch__:6849 2025-12-04T09:43:34.8450637Z Dynamo produced 1 graphs covering 1189 ops with 0 graph breaks (0 unique) 2025-12-04T09:43:37.3431481Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:43:37.3432563Z import pynvml # type: ignore[import] 2025-12-04T09:43:40.4146902Z 2025-12-04T09:43:40.4268445Z loading model: 0it [00:00, ?it/s]If you want to use `MegatronBertForCausalLM` as a standalone, add `is_decoder=True.` 2025-12-04T09:43:40.4273899Z WARNING:transformers.models.megatron_bert.modeling_megatron_bert:If you want to use `MegatronBertForCausalLM` as a standalone, add `is_decoder=True.` 2025-12-04T09:43:43.0130781Z 2025-12-04T09:43:43.0134500Z loading model: 0it [00:02, ?it/s] 2025-12-04T09:43:43.0154562Z cpu eval MegatronBertForCausalLM 2025-12-04T09:43:44.2897482Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:43:44.7929601Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:43:45.3216341Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:44:01.1367745Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1372707Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1374749Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1375119Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1375461Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1375660Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1375860Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1376062Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1376327Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1380310Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1382719Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1383244Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1387382Z res = mod(**inputs) 2025-12-04T09:44:01.1388138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1389373Z outputs = self.bert( 2025-12-04T09:44:01.1390145Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1390621Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1391102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1391543Z layer_outputs = layer_module( 2025-12-04T09:44:01.1391914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1392284Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1392689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1393093Z return func(*args, **kwargs) 2025-12-04T09:44:01.1393692Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1394188Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1394638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1395042Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1395490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1395975Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1396428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1396886Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1397643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1397999Z return self.act(input) 2025-12-04T09:44:01.1398111Z 2025-12-04T09:44:01.1398202Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1398409Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1398620Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1398835Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1399042Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1399239Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1399491Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1399687Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1399872Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1400069Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1400266Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1400541Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1400910Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1401236Z res = mod(**inputs) 2025-12-04T09:44:01.1401633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1402088Z outputs = self.bert( 2025-12-04T09:44:01.1402470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1402876Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1403266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1403667Z layer_outputs = layer_module( 2025-12-04T09:44:01.1404004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1404360Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1404718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1405083Z return func(*args, **kwargs) 2025-12-04T09:44:01.1405475Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1406258Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1406788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1407168Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1407593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1408043Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1408474Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1408902Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1409265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1409579Z return self.act(input) 2025-12-04T09:44:01.1409693Z 2025-12-04T09:44:01.1409769Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1409971Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1410167Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1410350Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1410542Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1410734Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1410919Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1411113Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1411423Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1411612Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1411805Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1412024Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1412364Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1412679Z res = mod(**inputs) 2025-12-04T09:44:01.1413073Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1413501Z outputs = self.bert( 2025-12-04T09:44:01.1413879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1414294Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1414730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1415137Z layer_outputs = layer_module( 2025-12-04T09:44:01.1415480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1415895Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1416263Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1416624Z return func(*args, **kwargs) 2025-12-04T09:44:01.1417018Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1417464Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1417859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1418243Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1418669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1419129Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1419562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1419994Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1420365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1420696Z return self.act(input) 2025-12-04T09:44:01.1420802Z 2025-12-04T09:44:01.1420883Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1421072Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1421271Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1421463Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1421651Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1421845Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1422037Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1422222Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1422417Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1422610Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1422801Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1423015Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1423369Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1423685Z res = mod(**inputs) 2025-12-04T09:44:01.1424062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1424466Z outputs = self.bert( 2025-12-04T09:44:01.1424882Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1425293Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1425685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1426089Z layer_outputs = layer_module( 2025-12-04T09:44:01.1426423Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1426787Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1427166Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1427536Z return func(*args, **kwargs) 2025-12-04T09:44:01.1427935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1428349Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1428723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1429090Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1429504Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1429943Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1430365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1430790Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1431140Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1431465Z return self.act(input) 2025-12-04T09:44:01.1431579Z 2025-12-04T09:44:01.1431654Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1431850Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1432036Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1432226Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1432418Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1432598Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1432788Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1432977Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1433243Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1433447Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1433645Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1433878Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1434225Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1434549Z res = mod(**inputs) 2025-12-04T09:44:01.1434949Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1435363Z outputs = self.bert( 2025-12-04T09:44:01.1435747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1436165Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1436577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1436985Z layer_outputs = layer_module( 2025-12-04T09:44:01.1437333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1437693Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1438095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1438471Z return func(*args, **kwargs) 2025-12-04T09:44:01.1438877Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1439302Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1439688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1440093Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1440530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1441006Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1441440Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1441902Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1442277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1442613Z return self.act(input) 2025-12-04T09:44:01.1442718Z 2025-12-04T09:44:01.1442795Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1442999Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1443203Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1443398Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1443595Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1443793Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1443980Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1444179Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1444380Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1444571Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1444771Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1445000Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1445353Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1445666Z res = mod(**inputs) 2025-12-04T09:44:01.1446067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1446478Z outputs = self.bert( 2025-12-04T09:44:01.1446868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1447289Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1447700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1448118Z layer_outputs = layer_module( 2025-12-04T09:44:01.1448460Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1448817Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1449196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1449565Z return func(*args, **kwargs) 2025-12-04T09:44:01.1449969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1450387Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1450771Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1451146Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1451604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1452069Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1452518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1452952Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1453394Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1453747Z return self.act(input) 2025-12-04T09:44:01.1453850Z 2025-12-04T09:44:01.1453932Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1454126Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1454321Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1454518Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1454760Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1454962Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1455153Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1455335Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1455524Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1455712Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1455894Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1456115Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1456456Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1456763Z res = mod(**inputs) 2025-12-04T09:44:01.1457137Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1457541Z outputs = self.bert( 2025-12-04T09:44:01.1457927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1458325Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1458725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1459123Z layer_outputs = layer_module( 2025-12-04T09:44:01.1459460Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1459799Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1460165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1460523Z return func(*args, **kwargs) 2025-12-04T09:44:01.1460918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1461333Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1461723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1462104Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1462522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1462993Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1463426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1463863Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1464218Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1464544Z return self.act(input) 2025-12-04T09:44:01.1464657Z 2025-12-04T09:44:01.1464730Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1464993Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1465180Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1465374Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1465567Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1465752Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1465943Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1466134Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1466316Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1466528Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1466723Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1466937Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1467283Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1467609Z res = mod(**inputs) 2025-12-04T09:44:01.1467998Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1468392Z outputs = self.bert( 2025-12-04T09:44:01.1468776Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1469178Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1469577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1469975Z layer_outputs = layer_module( 2025-12-04T09:44:01.1470313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1470660Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1471022Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1471382Z return func(*args, **kwargs) 2025-12-04T09:44:01.1471782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1472195Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1472575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1472957Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1473447Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1473920Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1474348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1474811Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1475193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1475527Z return self.act(input) 2025-12-04T09:44:01.1475644Z 2025-12-04T09:44:01.1475720Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1475935Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1476136Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1476324Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1476521Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1476720Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1476909Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1477124Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1477324Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1477515Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1477716Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1477980Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1478325Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1478642Z res = mod(**inputs) 2025-12-04T09:44:01.1479038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1479452Z outputs = self.bert( 2025-12-04T09:44:01.1479840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1480308Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1480720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1481134Z layer_outputs = layer_module( 2025-12-04T09:44:01.1481488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1481856Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1482234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1482591Z return func(*args, **kwargs) 2025-12-04T09:44:01.1482992Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1483411Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1483806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1484185Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1484625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1485095Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1485533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1485970Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1486340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1486669Z return self.act(input) 2025-12-04T09:44:01.1486775Z 2025-12-04T09:44:01.1486855Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1487059Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1487257Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1487457Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1487646Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1487845Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1488047Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1488244Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1488443Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1488638Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1488827Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1489065Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1489408Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1489717Z res = mod(**inputs) 2025-12-04T09:44:01.1490090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1490492Z outputs = self.bert( 2025-12-04T09:44:01.1490870Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1491263Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1491694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1492098Z layer_outputs = layer_module( 2025-12-04T09:44:01.1492433Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1492770Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1493134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1493511Z return func(*args, **kwargs) 2025-12-04T09:44:01.1493897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1494316Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1494717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1495096Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1495514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1495971Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1496402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1496842Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1497202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1497530Z return self.act(input) 2025-12-04T09:44:01.1497635Z 2025-12-04T09:44:01.1497716Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1497906Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1498102Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1498298Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1498490Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1498675Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1498866Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1499058Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1499244Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1499433Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1499620Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1499834Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1500175Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1500483Z res = mod(**inputs) 2025-12-04T09:44:01.1500864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1501265Z outputs = self.bert( 2025-12-04T09:44:01.1501649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1502052Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1502444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1502845Z layer_outputs = layer_module( 2025-12-04T09:44:01.1503178Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1503525Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1503882Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1504237Z return func(*args, **kwargs) 2025-12-04T09:44:01.1504657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1505071Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1505456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1505973Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1506401Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1506901Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1507330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1507767Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1508180Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1508504Z return self.act(input) 2025-12-04T09:44:01.1508621Z 2025-12-04T09:44:01.1508700Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1508906Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1509098Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1509294Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1509498Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1509692Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1509887Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1510083Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1510279Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1510466Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1510661Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1510886Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1511230Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1511540Z res = mod(**inputs) 2025-12-04T09:44:01.1511926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1512325Z outputs = self.bert( 2025-12-04T09:44:01.1512697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1513096Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1513560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1513983Z layer_outputs = layer_module( 2025-12-04T09:44:01.1514327Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1514693Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1515073Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1515438Z return func(*args, **kwargs) 2025-12-04T09:44:01.1515831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1516249Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1516644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1517028Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1517470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1517941Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1518422Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1518875Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1519250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1519586Z return self.act(input) 2025-12-04T09:44:01.1519693Z 2025-12-04T09:44:01.1519769Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1519978Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1520197Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1520395Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1520600Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1520802Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1521008Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1521218Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1521413Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1521610Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1521801Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1522029Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1522380Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1522690Z res = mod(**inputs) 2025-12-04T09:44:01.1523085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1523497Z outputs = self.bert( 2025-12-04T09:44:01.1523888Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1524294Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1524712Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1525125Z layer_outputs = layer_module( 2025-12-04T09:44:01.1525469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1525826Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1526202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1526574Z return func(*args, **kwargs) 2025-12-04T09:44:01.1526974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1527400Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1527795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1528184Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1528625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1529087Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1529523Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1529958Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1530313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1530638Z return self.act(input) 2025-12-04T09:44:01.1530740Z 2025-12-04T09:44:01.1530820Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1531008Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1531199Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1531393Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1531632Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1531827Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1532021Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1532215Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1532397Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1532586Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1532779Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1532994Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1533360Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1533673Z res = mod(**inputs) 2025-12-04T09:44:01.1534054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1534475Z outputs = self.bert( 2025-12-04T09:44:01.1534861Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1535267Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1535662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1536068Z layer_outputs = layer_module( 2025-12-04T09:44:01.1536408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1536763Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1537130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1537488Z return func(*args, **kwargs) 2025-12-04T09:44:01.1537883Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1538308Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1538704Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1539087Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1539516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1539972Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1540407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1540847Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1541214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1541537Z return self.act(input) 2025-12-04T09:44:01.1541649Z 2025-12-04T09:44:01.1541726Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1541928Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1542121Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1542318Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1542515Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1542702Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1542896Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1543091Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1543289Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1543476Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1543671Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1543893Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1544236Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1544550Z res = mod(**inputs) 2025-12-04T09:44:01.1544970Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1545359Z outputs = self.bert( 2025-12-04T09:44:01.1545737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1546134Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1546524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1546931Z layer_outputs = layer_module( 2025-12-04T09:44:01.1547268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1547628Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1548016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1548369Z return func(*args, **kwargs) 2025-12-04T09:44:01.1548757Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1549171Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1549547Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1549929Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1550354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1550811Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1551233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1551672Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1552035Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1552358Z return self.act(input) 2025-12-04T09:44:01.1552462Z 2025-12-04T09:44:01.1552536Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1552732Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1552928Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1553116Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1553418Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1553621Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1553815Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1554020Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1554226Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1554430Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1554628Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1554861Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1555214Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1555526Z res = mod(**inputs) 2025-12-04T09:44:01.1555920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1556329Z outputs = self.bert( 2025-12-04T09:44:01.1556708Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1557123Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1557528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1557941Z layer_outputs = layer_module( 2025-12-04T09:44:01.1558309Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1558653Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1559010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1559363Z return func(*args, **kwargs) 2025-12-04T09:44:01.1559740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1560180Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1560563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1560934Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1561393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1561860Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1562290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1562722Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1563082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1563409Z return self.act(input) 2025-12-04T09:44:01.1563517Z 2025-12-04T09:44:01.1563597Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1563788Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1563983Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1564179Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1564367Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1564561Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1564758Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1564948Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1565143Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1565341Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1565527Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1565754Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1566113Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1566430Z res = mod(**inputs) 2025-12-04T09:44:01.1566812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1567213Z outputs = self.bert( 2025-12-04T09:44:01.1567595Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1568004Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1568399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1568801Z layer_outputs = layer_module( 2025-12-04T09:44:01.1569133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1569471Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1569836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1570206Z return func(*args, **kwargs) 2025-12-04T09:44:01.1570600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1571015Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1571429Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1571811Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1572237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1572706Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1573143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1573602Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1573956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1574280Z return self.act(input) 2025-12-04T09:44:01.1574408Z 2025-12-04T09:44:01.1574481Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1574682Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1574872Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1575066Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1575261Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1575445Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1575637Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1575827Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1576010Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1576203Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1576405Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1576617Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1576960Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1577268Z res = mod(**inputs) 2025-12-04T09:44:01.1577655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1578051Z outputs = self.bert( 2025-12-04T09:44:01.1578436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1578840Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1579235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1579642Z layer_outputs = layer_module( 2025-12-04T09:44:01.1579977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1580328Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1580689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1581053Z return func(*args, **kwargs) 2025-12-04T09:44:01.1581448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1581864Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1582250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1582628Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1583061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1583520Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1583943Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1584384Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1584783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1585102Z return self.act(input) 2025-12-04T09:44:01.1585216Z 2025-12-04T09:44:01.1585286Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1585482Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1585674Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1585861Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1586053Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1586265Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1586450Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1586643Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1586834Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1587018Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1587227Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1587449Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1587798Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1588098Z res = mod(**inputs) 2025-12-04T09:44:01.1588478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1588886Z outputs = self.bert( 2025-12-04T09:44:01.1589258Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1589662Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1590061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1590461Z layer_outputs = layer_module( 2025-12-04T09:44:01.1590791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1591144Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1591508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1591856Z return func(*args, **kwargs) 2025-12-04T09:44:01.1592249Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1592661Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1593056Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1593520Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1593964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1594448Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1594892Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1595313Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1595701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1596040Z return self.act(input) 2025-12-04T09:44:01.1596151Z 2025-12-04T09:44:01.1596230Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1596445Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1596649Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1596851Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1597046Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1597302Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1597503Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1597742Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1597956Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1598153Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1598359Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1598600Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1598955Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1599286Z res = mod(**inputs) 2025-12-04T09:44:01.1599693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1600141Z outputs = self.bert( 2025-12-04T09:44:01.1600531Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1600978Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1601398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1601816Z layer_outputs = layer_module( 2025-12-04T09:44:01.1602170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1602541Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1602924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1603298Z return func(*args, **kwargs) 2025-12-04T09:44:01.1603713Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1604151Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1604547Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1604930Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1605355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1605975Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1606393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1606826Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1607187Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1607504Z return self.act(input) 2025-12-04T09:44:01.1607610Z 2025-12-04T09:44:01.1607683Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1607881Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1608075Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1608260Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1608454Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1608646Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1608830Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1609021Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1609209Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1609396Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1609578Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1609802Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1610144Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1610443Z res = mod(**inputs) 2025-12-04T09:44:01.1610817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1611211Z outputs = self.bert( 2025-12-04T09:44:01.1611660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1612062Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1612461Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1612864Z layer_outputs = layer_module( 2025-12-04T09:44:01.1613191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1613575Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1613945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1614309Z return func(*args, **kwargs) 2025-12-04T09:44:01.1614714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1615115Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1615486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1615857Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1616267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1616718Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1617136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1617554Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1617915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1618241Z return self.act(input) 2025-12-04T09:44:01.1618345Z 2025-12-04T09:44:01.1618426Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1618617Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1618808Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1619001Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1619181Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1619379Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1619569Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1619756Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1619946Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1620136Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1620327Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1620537Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1620884Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1621187Z res = mod(**inputs) 2025-12-04T09:44:01.1621551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1621948Z outputs = self.bert( 2025-12-04T09:44:01.1622320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1622710Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1623097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1623488Z layer_outputs = layer_module( 2025-12-04T09:44:01.1623816Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1624153Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1624538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1624900Z return func(*args, **kwargs) 2025-12-04T09:44:01.1625293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1625699Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1626087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1626480Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1626908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1627361Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1627831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1628262Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1628617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1628941Z return self.act(input) 2025-12-04T09:44:01.1629054Z 2025-12-04T09:44:01.1629129Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1629328Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1629519Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1629713Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1629906Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1630090Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1630281Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1630474Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1630659Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1630853Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1631046Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1631267Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1631604Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1631911Z res = mod(**inputs) 2025-12-04T09:44:01.1632291Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1632681Z outputs = self.bert( 2025-12-04T09:44:01.1633058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1633525Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1633933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1634338Z layer_outputs = layer_module( 2025-12-04T09:44:01.1634686Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1635030Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1635384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1635741Z return func(*args, **kwargs) 2025-12-04T09:44:01.1636127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1636534Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1636905Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1637280Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1637728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1638191Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1638616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1639054Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1639417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1639754Z return self.act(input) 2025-12-04T09:44:01.1639857Z 2025-12-04T09:44:01.1639930Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1640129Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1640323Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1640527Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1640719Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1640913Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1641100Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1641294Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1641485Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1641671Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1641871Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1642096Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1642444Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1642755Z res = mod(**inputs) 2025-12-04T09:44:01.1643141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:44:01.1643546Z outputs = self.bert( 2025-12-04T09:44:01.1643926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:44:01.1644334Z encoder_outputs = self.encoder( 2025-12-04T09:44:01.1644740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:44:01.1645142Z layer_outputs = layer_module( 2025-12-04T09:44:01.1645474Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:01.1645824Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:01.1646195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:01.1646556Z return func(*args, **kwargs) 2025-12-04T09:44:01.1646941Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:44:01.1647363Z layer_output = apply_chunking_to_forward( 2025-12-04T09:44:01.1647751Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:44:01.1648125Z return forward_fn(*input_tensors) 2025-12-04T09:44:01.1648557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:44:01.1649018Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:44:01.1649452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:44:01.1649884Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:01.1650250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:44:01.1650582Z return self.act(input) 2025-12-04T09:44:01.1650686Z 2025-12-04T09:44:01.1650800Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1650992Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1651187Z cudagraph partition due to non gpu ops 2025-12-04T09:44:01.1651408Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:01.1651744Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:01.1652051Z res = mod(**inputs) 2025-12-04T09:44:01.1652436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1078, in forward 2025-12-04T09:44:01.1652862Z lm_loss = self.loss_function( 2025-12-04T09:44:01.1653218Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 67, in ForCausalLMLoss 2025-12-04T09:44:01.1653676Z loss = fixed_cross_entropy(logits, shift_labels, num_items_in_batch, ignore_index, **kwargs) 2025-12-04T09:44:01.1654158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 36, in fixed_cross_entropy 2025-12-04T09:44:01.1654635Z loss = nn.functional.cross_entropy(source, target, ignore_index=ignore_index, reduction=reduction) 2025-12-04T09:44:01.1654882Z 2025-12-04T09:44:12.6677973Z Compilation time (from dynamo_timed): 26.434743024 2025-12-04T09:44:12.6728103Z pass 2025-12-04T09:44:12.6732284Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:44:12.6736614Z TIMING: _recursive_pre_grad_passes:0.05245 _recursive_joint_graph_passes:0.7075 _recursive_post_grad_passes:0.0873 async_compile.wait:0.79583 code_gen:10.79976 inductor_compile:13.7609 backend_compile:21.36014 gc:0.00053 entire_frame_compile:26.43474 total_wall_time:26.43474 2025-12-04T09:44:12.6738026Z STATS: call_* op count: 723 | FakeTensorMode.__torch_dispatch__:39096 | FakeTensor.__torch_dispatch__:6663 | ProxyTorchDispatchMode.__torch_dispatch__:6492 2025-12-04T09:44:12.6738531Z Dynamo produced 1 graphs covering 723 ops with 0 graph breaks (0 unique) 2025-12-04T09:44:15.3746553Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:44:15.3747354Z import pynvml # type: ignore[import] 2025-12-04T09:44:18.5181317Z 2025-12-04T09:44:18.9719137Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:44:18.9719462Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:44:18.9777776Z cpu eval MobileBertForMaskedLM 2025-12-04T09:44:19.2308873Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:44:19.3757898Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:44:19.6965517Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:44:49.6631967Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6632692Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6633037Z res = mod(**inputs) 2025-12-04T09:44:49.6633588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6634018Z outputs = self.mobilebert( 2025-12-04T09:44:49.6634543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 791, in forward 2025-12-04T09:44:49.6634958Z embedding_output = self.embeddings( 2025-12-04T09:44:49.6635362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 199, in forward 2025-12-04T09:44:49.6635767Z inputs_embeds = torch.cat( 2025-12-04T09:44:49.6635880Z 2025-12-04T09:44:49.6636354Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6636610Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6636971Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6637289Z res = mod(**inputs) 2025-12-04T09:44:49.6637683Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6638094Z outputs = self.mobilebert( 2025-12-04T09:44:49.6638548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 791, in forward 2025-12-04T09:44:49.6638947Z embedding_output = self.embeddings( 2025-12-04T09:44:49.6639352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 215, in forward 2025-12-04T09:44:49.6639829Z embeddings = self.LayerNorm(embeddings) 2025-12-04T09:44:49.6640233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6640663Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6640822Z 2025-12-04T09:44:49.6640923Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6641156Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6641501Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6641864Z res = mod(**inputs) 2025-12-04T09:44:49.6642259Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6642678Z outputs = self.mobilebert( 2025-12-04T09:44:49.6643049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6643448Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6643836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6644220Z layer_outputs = layer_module( 2025-12-04T09:44:49.6644603Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.6645076Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.6645556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.6645977Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.6646401Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.6646813Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.6647226Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6647628Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6647774Z 2025-12-04T09:44:49.6647850Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6648051Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6648240Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6648440Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6648635Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6648846Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6649033Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6649229Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6649423Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6649612Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6649882Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6650238Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6650558Z res = mod(**inputs) 2025-12-04T09:44:49.6650950Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6651359Z outputs = self.mobilebert( 2025-12-04T09:44:49.6651755Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6652177Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6652572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6652977Z layer_outputs = layer_module( 2025-12-04T09:44:49.6653391Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.6653791Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.6654179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.6654605Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.6655023Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.6655614Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.6656062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6656465Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6656608Z 2025-12-04T09:44:49.6656682Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6656906Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6657241Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6657536Z res = mod(**inputs) 2025-12-04T09:44:49.6658579Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6659229Z outputs = self.mobilebert( 2025-12-04T09:44:49.6659656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6660088Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6660504Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6660917Z layer_outputs = layer_module( 2025-12-04T09:44:49.6661321Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6661749Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6662181Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.6662629Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.6663076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.6663523Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.6663701Z 2025-12-04T09:44:49.6663786Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6664027Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6664380Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6664707Z res = mod(**inputs) 2025-12-04T09:44:49.6665251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6665669Z outputs = self.mobilebert( 2025-12-04T09:44:49.6666044Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6666441Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6666830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6667292Z layer_outputs = layer_module( 2025-12-04T09:44:49.6667679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6668132Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6668553Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.6668991Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.6669432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.6669884Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.6670334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6670755Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6670913Z 2025-12-04T09:44:49.6670991Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6671224Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6671574Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6671894Z res = mod(**inputs) 2025-12-04T09:44:49.6672281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6672689Z outputs = self.mobilebert( 2025-12-04T09:44:49.6673071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6673635Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6674027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6674449Z layer_outputs = layer_module( 2025-12-04T09:44:49.6674818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6675225Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6675634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.6676048Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.6676478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.6676909Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.6677065Z 2025-12-04T09:44:49.6677148Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6677368Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6677707Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6678014Z res = mod(**inputs) 2025-12-04T09:44:49.6678381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6678815Z outputs = self.mobilebert( 2025-12-04T09:44:49.6679206Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6679611Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6680001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6680400Z layer_outputs = layer_module( 2025-12-04T09:44:49.6680856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6681313Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6681712Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.6682184Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.6682641Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.6683089Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.6683533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6683954Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6684099Z 2025-12-04T09:44:49.6684187Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6684429Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6684777Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6685084Z res = mod(**inputs) 2025-12-04T09:44:49.6685466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6685872Z outputs = self.mobilebert( 2025-12-04T09:44:49.6686259Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6686658Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6687057Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6687456Z layer_outputs = layer_module( 2025-12-04T09:44:49.6687843Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6688269Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6688690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.6689132Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.6689561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.6690001Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.6690157Z 2025-12-04T09:44:49.6690255Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6690479Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6690811Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6691125Z res = mod(**inputs) 2025-12-04T09:44:49.6691497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6691890Z outputs = self.mobilebert( 2025-12-04T09:44:49.6692274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6692716Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6693109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6693505Z layer_outputs = layer_module( 2025-12-04T09:44:49.6693896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6694324Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6694765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.6695204Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.6695656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.6696124Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.6696562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6696982Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6697130Z 2025-12-04T09:44:49.6697207Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6697433Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6697767Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6698079Z res = mod(**inputs) 2025-12-04T09:44:49.6698457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6698854Z outputs = self.mobilebert( 2025-12-04T09:44:49.6699236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6699632Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6700029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6700424Z layer_outputs = layer_module( 2025-12-04T09:44:49.6700814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.6701256Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.6701696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.6702124Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.6702288Z 2025-12-04T09:44:49.6702362Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6702591Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6702937Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6703243Z res = mod(**inputs) 2025-12-04T09:44:49.6703620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6704020Z outputs = self.mobilebert( 2025-12-04T09:44:49.6704406Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6704818Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6705214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6705633Z layer_outputs = layer_module( 2025-12-04T09:44:49.6706350Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.6706950Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.6707453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.6707911Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.6708362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6708830Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6708979Z 2025-12-04T09:44:49.6709065Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6709288Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6709644Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6709999Z res = mod(**inputs) 2025-12-04T09:44:49.6710398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6710815Z outputs = self.mobilebert( 2025-12-04T09:44:49.6711225Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6711651Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6712052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6712471Z layer_outputs = layer_module( 2025-12-04T09:44:49.6712880Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.6713457Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.6713953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.6714408Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.6714862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.6715291Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.6715715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6716143Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6716293Z 2025-12-04T09:44:49.6716375Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6716611Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6716953Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6717278Z res = mod(**inputs) 2025-12-04T09:44:49.6717667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6718065Z outputs = self.mobilebert( 2025-12-04T09:44:49.6718456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6718858Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6719255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6719651Z layer_outputs = layer_module( 2025-12-04T09:44:49.6720048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.6720539Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.6721087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.6721530Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.6721973Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.6722394Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.6722807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6723254Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6723404Z 2025-12-04T09:44:49.6723483Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6723688Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6723884Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6724104Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6724305Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6724500Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6724706Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6724904Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6725095Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6725294Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6725523Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6725870Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6726177Z res = mod(**inputs) 2025-12-04T09:44:49.6726560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6726965Z outputs = self.mobilebert( 2025-12-04T09:44:49.6727351Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6727758Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6728167Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6728558Z layer_outputs = layer_module( 2025-12-04T09:44:49.6728933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.6729342Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.6729750Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.6730194Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.6730625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.6731067Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.6731506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6731916Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6732053Z 2025-12-04T09:44:49.6732126Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6732348Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6732687Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6732985Z res = mod(**inputs) 2025-12-04T09:44:49.6733355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6733748Z outputs = self.mobilebert( 2025-12-04T09:44:49.6734138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6734580Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6734980Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6735380Z layer_outputs = layer_module( 2025-12-04T09:44:49.6735766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6736187Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6736626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.6737070Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.6737495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.6737958Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.6738122Z 2025-12-04T09:44:49.6738197Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6738423Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6738758Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6739066Z res = mod(**inputs) 2025-12-04T09:44:49.6739499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6739899Z outputs = self.mobilebert( 2025-12-04T09:44:49.6740286Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6740684Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6741077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6741476Z layer_outputs = layer_module( 2025-12-04T09:44:49.6741870Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6742291Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6742714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.6743158Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.6743608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.6744062Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.6744517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6744932Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6745082Z 2025-12-04T09:44:49.6745158Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6745798Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6746152Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6746464Z res = mod(**inputs) 2025-12-04T09:44:49.6746867Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6747271Z outputs = self.mobilebert( 2025-12-04T09:44:49.6747661Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6748078Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6748516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6748920Z layer_outputs = layer_module( 2025-12-04T09:44:49.6749315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6749847Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6750294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.6750746Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.6751225Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.6751673Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.6751834Z 2025-12-04T09:44:49.6751943Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6752170Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6752531Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6752855Z res = mod(**inputs) 2025-12-04T09:44:49.6753294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6753714Z outputs = self.mobilebert( 2025-12-04T09:44:49.6754130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6754580Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6755011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6755450Z layer_outputs = layer_module( 2025-12-04T09:44:49.6755855Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6756288Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6756711Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.6757175Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.6757638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.6758098Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.6758556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6758988Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6759133Z 2025-12-04T09:44:49.6759218Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6759445Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6759802Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6760125Z res = mod(**inputs) 2025-12-04T09:44:49.6760509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6760912Z outputs = self.mobilebert( 2025-12-04T09:44:49.6761311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6761729Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6762127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6762540Z layer_outputs = layer_module( 2025-12-04T09:44:49.6763715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6764171Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6764596Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.6765047Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.6765496Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.6765965Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.6766128Z 2025-12-04T09:44:49.6766210Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6766442Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6766794Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6767129Z res = mod(**inputs) 2025-12-04T09:44:49.6767522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6767936Z outputs = self.mobilebert( 2025-12-04T09:44:49.6768329Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6768733Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6769124Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6769521Z layer_outputs = layer_module( 2025-12-04T09:44:49.6769914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6770333Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6770753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.6771205Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.6771648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.6772095Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.6772545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6772962Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6773104Z 2025-12-04T09:44:49.6773178Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6773402Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6773746Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6774057Z res = mod(**inputs) 2025-12-04T09:44:49.6774439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6774861Z outputs = self.mobilebert( 2025-12-04T09:44:49.6775257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6775665Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6776070Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6776483Z layer_outputs = layer_module( 2025-12-04T09:44:49.6776893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.6777335Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.6777824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.6778264Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.6778420Z 2025-12-04T09:44:49.6778502Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6778723Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6779069Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6779389Z res = mod(**inputs) 2025-12-04T09:44:49.6779782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6780193Z outputs = self.mobilebert( 2025-12-04T09:44:49.6780601Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6781047Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6781448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6781864Z layer_outputs = layer_module( 2025-12-04T09:44:49.6782267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.6782799Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.6783287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.6783741Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.6784204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6784642Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6784782Z 2025-12-04T09:44:49.6784861Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6785088Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6785432Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6785735Z res = mod(**inputs) 2025-12-04T09:44:49.6786113Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6786514Z outputs = self.mobilebert( 2025-12-04T09:44:49.6786910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6787319Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6787721Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6788147Z layer_outputs = layer_module( 2025-12-04T09:44:49.6788555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.6789048Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.6789544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.6790007Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.6790465Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.6790928Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.6791401Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6791898Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6792049Z 2025-12-04T09:44:49.6792129Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6792369Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6792732Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6793079Z res = mod(**inputs) 2025-12-04T09:44:49.6793565Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6794053Z outputs = self.mobilebert( 2025-12-04T09:44:49.6794501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6794944Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6795369Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6795807Z layer_outputs = layer_module( 2025-12-04T09:44:49.6796213Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.6796708Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.6797217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.6797668Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.6798117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.6798538Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.6798970Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6799419Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6799566Z 2025-12-04T09:44:49.6799650Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6799864Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6800079Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6800293Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6800487Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6800689Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6800890Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6801083Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6801286Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6801483Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6801700Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6802051Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6802374Z res = mod(**inputs) 2025-12-04T09:44:49.6802763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6803173Z outputs = self.mobilebert( 2025-12-04T09:44:49.6803571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6803982Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6804381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6804788Z layer_outputs = layer_module( 2025-12-04T09:44:49.6805197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.6805633Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.6806236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.6806694Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.6807142Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.6807595Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.6808036Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6808483Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6808622Z 2025-12-04T09:44:49.6808705Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6808921Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6809262Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6809601Z res = mod(**inputs) 2025-12-04T09:44:49.6809979Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6810374Z outputs = self.mobilebert( 2025-12-04T09:44:49.6810771Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6811171Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6811567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6811970Z layer_outputs = layer_module( 2025-12-04T09:44:49.6812363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6812784Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6813203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.6813640Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.6814076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.6814511Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.6814670Z 2025-12-04T09:44:49.6814746Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6814971Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6815319Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6815634Z res = mod(**inputs) 2025-12-04T09:44:49.6815999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6816403Z outputs = self.mobilebert( 2025-12-04T09:44:49.6816791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6817201Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6817597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6817997Z layer_outputs = layer_module( 2025-12-04T09:44:49.6818387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6818808Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6819242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.6819692Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.6820174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.6820616Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.6821069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6821492Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6821631Z 2025-12-04T09:44:49.6821714Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6821953Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6822299Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6822611Z res = mod(**inputs) 2025-12-04T09:44:49.6822981Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6823407Z outputs = self.mobilebert( 2025-12-04T09:44:49.6823794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6824195Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6824585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6824985Z layer_outputs = layer_module( 2025-12-04T09:44:49.6825378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6825797Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6826223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.6826664Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.6827110Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.6827546Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.6827713Z 2025-12-04T09:44:49.6827788Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6828015Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6828359Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6828670Z res = mod(**inputs) 2025-12-04T09:44:49.6829046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6829447Z outputs = self.mobilebert( 2025-12-04T09:44:49.6829831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6830235Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6830631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6831044Z layer_outputs = layer_module( 2025-12-04T09:44:49.6831433Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6831854Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6832276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.6832730Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.6833252Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.6833752Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.6834268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6834685Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6834836Z 2025-12-04T09:44:49.6834911Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6835137Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6835483Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6835809Z res = mod(**inputs) 2025-12-04T09:44:49.6836187Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6836593Z outputs = self.mobilebert( 2025-12-04T09:44:49.6836961Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6837387Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6837772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6838156Z layer_outputs = layer_module( 2025-12-04T09:44:49.6838527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6838937Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6839344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.6839767Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.6840189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.6840613Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.6840768Z 2025-12-04T09:44:49.6840852Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6841076Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6841407Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6841715Z res = mod(**inputs) 2025-12-04T09:44:49.6842079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6842466Z outputs = self.mobilebert( 2025-12-04T09:44:49.6842842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6843234Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6843616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6844002Z layer_outputs = layer_module( 2025-12-04T09:44:49.6844387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6844796Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6845198Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.6845639Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.6846077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.6846509Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.6846940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6847390Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6847535Z 2025-12-04T09:44:49.6847609Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6847829Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6848154Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6848455Z res = mod(**inputs) 2025-12-04T09:44:49.6848821Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6849224Z outputs = self.mobilebert( 2025-12-04T09:44:49.6849599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6849989Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6850369Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6850779Z layer_outputs = layer_module( 2025-12-04T09:44:49.6851162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.6851598Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.6852036Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.6852463Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.6852622Z 2025-12-04T09:44:49.6852694Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6852914Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6853246Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6853548Z res = mod(**inputs) 2025-12-04T09:44:49.6853924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6854310Z outputs = self.mobilebert( 2025-12-04T09:44:49.6854677Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6855069Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6855451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6855840Z layer_outputs = layer_module( 2025-12-04T09:44:49.6856213Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.6856685Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.6857156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.6857586Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.6858024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6858431Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6858570Z 2025-12-04T09:44:49.6858650Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6858859Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6859202Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6859505Z res = mod(**inputs) 2025-12-04T09:44:49.6859869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6860257Z outputs = self.mobilebert( 2025-12-04T09:44:49.6860696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6861085Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6861463Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6861854Z layer_outputs = layer_module( 2025-12-04T09:44:49.6862237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.6862724Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.6863184Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.6863622Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.6864074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.6864504Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.6864925Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6865326Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6865460Z 2025-12-04T09:44:49.6865542Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6865751Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6866085Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6866386Z res = mod(**inputs) 2025-12-04T09:44:49.6866746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6867120Z outputs = self.mobilebert( 2025-12-04T09:44:49.6867492Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6867874Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6868255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6868633Z layer_outputs = layer_module( 2025-12-04T09:44:49.6869013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.6869480Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.6869941Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.6870365Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.6870782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.6871175Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.6871562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6871963Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6872104Z 2025-12-04T09:44:49.6872178Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6872373Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6872559Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6872748Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6872937Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6873130Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6873426Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6873633Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6873867Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6874074Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6874306Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6874669Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6874971Z res = mod(**inputs) 2025-12-04T09:44:49.6875341Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6875755Z outputs = self.mobilebert( 2025-12-04T09:44:49.6876130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6876521Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6876905Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6877314Z layer_outputs = layer_module( 2025-12-04T09:44:49.6877686Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.6878091Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.6878494Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.6878925Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.6879360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.6879797Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.6880236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6880645Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6880790Z 2025-12-04T09:44:49.6880861Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6881079Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6881414Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6881711Z res = mod(**inputs) 2025-12-04T09:44:49.6882076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6882465Z outputs = self.mobilebert( 2025-12-04T09:44:49.6882834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6883224Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6883611Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6884004Z layer_outputs = layer_module( 2025-12-04T09:44:49.6884383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6884797Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6885208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.6885634Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.6886052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.6886481Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.6886632Z 2025-12-04T09:44:49.6886715Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6886928Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6887297Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6887605Z res = mod(**inputs) 2025-12-04T09:44:49.6887966Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6888346Z outputs = self.mobilebert( 2025-12-04T09:44:49.6888722Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6889148Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6889533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6889913Z layer_outputs = layer_module( 2025-12-04T09:44:49.6890294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6890733Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6891139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.6891584Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.6892029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.6892464Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.6892899Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6893310Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6893453Z 2025-12-04T09:44:49.6893529Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6893749Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6894080Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6894392Z res = mod(**inputs) 2025-12-04T09:44:49.6894760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6895150Z outputs = self.mobilebert( 2025-12-04T09:44:49.6895532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6895929Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6896313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6896696Z layer_outputs = layer_module( 2025-12-04T09:44:49.6897089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6897507Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6897921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.6898346Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.6898773Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.6899202Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.6899358Z 2025-12-04T09:44:49.6899430Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6899650Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6899990Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6900300Z res = mod(**inputs) 2025-12-04T09:44:49.6900694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6901092Z outputs = self.mobilebert( 2025-12-04T09:44:49.6901471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6901855Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6902241Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6902651Z layer_outputs = layer_module( 2025-12-04T09:44:49.6903040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6903440Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6903868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.6904307Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.6904744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.6905168Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.6905603Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6906201Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6906343Z 2025-12-04T09:44:49.6906424Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6906639Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6906976Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6907289Z res = mod(**inputs) 2025-12-04T09:44:49.6907653Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6908049Z outputs = self.mobilebert( 2025-12-04T09:44:49.6908425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6908820Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6909195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6909590Z layer_outputs = layer_module( 2025-12-04T09:44:49.6909975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6910381Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6910794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.6911226Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.6911652Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.6912075Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.6912238Z 2025-12-04T09:44:49.6912311Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6912535Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6912874Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6913227Z res = mod(**inputs) 2025-12-04T09:44:49.6913625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6914044Z outputs = self.mobilebert( 2025-12-04T09:44:49.6914489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6914887Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6915269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6915658Z layer_outputs = layer_module( 2025-12-04T09:44:49.6916034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6916483Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6916896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.6917331Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.6917795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.6918234Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.6918672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6919085Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6919223Z 2025-12-04T09:44:49.6919297Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6919518Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6919859Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6920160Z res = mod(**inputs) 2025-12-04T09:44:49.6920529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6920922Z outputs = self.mobilebert( 2025-12-04T09:44:49.6921302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6921689Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6922072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6922464Z layer_outputs = layer_module( 2025-12-04T09:44:49.6922840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.6923279Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.6923714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.6924143Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.6924299Z 2025-12-04T09:44:49.6924374Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6924593Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6924929Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6925238Z res = mod(**inputs) 2025-12-04T09:44:49.6925598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6925990Z outputs = self.mobilebert( 2025-12-04T09:44:49.6926369Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6926759Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6927149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6927547Z layer_outputs = layer_module( 2025-12-04T09:44:49.6927961Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.6928419Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.6928881Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.6929309Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.6929732Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6930153Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6930300Z 2025-12-04T09:44:49.6930373Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6930592Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6930940Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6931246Z res = mod(**inputs) 2025-12-04T09:44:49.6931609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6931998Z outputs = self.mobilebert( 2025-12-04T09:44:49.6932369Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6932758Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6933146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6933530Z layer_outputs = layer_module( 2025-12-04T09:44:49.6933908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.6934382Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.6934859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.6935302Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.6935739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.6936179Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.6936612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6937013Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6937158Z 2025-12-04T09:44:49.6937233Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6937459Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6937796Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6938095Z res = mod(**inputs) 2025-12-04T09:44:49.6938464Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6938853Z outputs = self.mobilebert( 2025-12-04T09:44:49.6939219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6939614Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6939995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6940386Z layer_outputs = layer_module( 2025-12-04T09:44:49.6940759Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.6941272Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.6941736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.6942152Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.6942558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.6942953Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.6943367Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6943763Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6943904Z 2025-12-04T09:44:49.6943977Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6944190Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6944385Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6944568Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6944758Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6944949Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6945131Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6945323Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6945513Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6945695Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6945916Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6946255Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6946559Z res = mod(**inputs) 2025-12-04T09:44:49.6946915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6947308Z outputs = self.mobilebert( 2025-12-04T09:44:49.6947688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6948069Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6948448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6948847Z layer_outputs = layer_module( 2025-12-04T09:44:49.6949228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.6949621Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.6950019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.6950459Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.6950895Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.6951323Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.6951755Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6952166Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6952303Z 2025-12-04T09:44:49.6952382Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6952597Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6952933Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6953315Z res = mod(**inputs) 2025-12-04T09:44:49.6953678Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6954077Z outputs = self.mobilebert( 2025-12-04T09:44:49.6954511Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6954930Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6955312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6955713Z layer_outputs = layer_module( 2025-12-04T09:44:49.6956105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6956544Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6956970Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.6957422Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.6957859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.6958293Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.6958463Z 2025-12-04T09:44:49.6958538Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6958763Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6959106Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6959414Z res = mod(**inputs) 2025-12-04T09:44:49.6959798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6960201Z outputs = self.mobilebert( 2025-12-04T09:44:49.6960584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6960991Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6961387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6961792Z layer_outputs = layer_module( 2025-12-04T09:44:49.6962179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6962610Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6963038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.6963494Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.6963944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.6964396Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.6964842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6965264Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6965413Z 2025-12-04T09:44:49.6965487Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6965712Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6966059Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6966367Z res = mod(**inputs) 2025-12-04T09:44:49.6966746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6967152Z outputs = self.mobilebert( 2025-12-04T09:44:49.6967537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6967928Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6968344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6968735Z layer_outputs = layer_module( 2025-12-04T09:44:49.6969112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6969521Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6969926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.6970367Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.6970780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.6971228Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.6971382Z 2025-12-04T09:44:49.6971465Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6971686Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6972010Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6972318Z res = mod(**inputs) 2025-12-04T09:44:49.6972683Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6973064Z outputs = self.mobilebert( 2025-12-04T09:44:49.6973443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6973836Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6974221Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6974602Z layer_outputs = layer_module( 2025-12-04T09:44:49.6974981Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6975451Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6975919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.6976434Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.6977089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.6977609Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.6978143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.6978608Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.6978810Z 2025-12-04T09:44:49.6978905Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6979192Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6979613Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.6979945Z res = mod(**inputs) 2025-12-04T09:44:49.6980381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.6980855Z outputs = self.mobilebert( 2025-12-04T09:44:49.6981289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.6981715Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.6982182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.6982692Z layer_outputs = layer_module( 2025-12-04T09:44:49.6983120Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.6983613Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.6984085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.6984567Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.6985096Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.6985589Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.6985790Z 2025-12-04T09:44:49.6985887Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.6986216Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.6986599Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7009426Z res = mod(**inputs) 2025-12-04T09:44:49.7009896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7010327Z outputs = self.mobilebert( 2025-12-04T09:44:49.7010714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7011113Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7011520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7011916Z layer_outputs = layer_module( 2025-12-04T09:44:49.7012295Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7012719Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7013130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7013565Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7014003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7014438Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7014880Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7015282Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7015434Z 2025-12-04T09:44:49.7015515Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7015753Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7016103Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7016409Z res = mod(**inputs) 2025-12-04T09:44:49.7016780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7017175Z outputs = self.mobilebert( 2025-12-04T09:44:49.7017546Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7017938Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7018317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7018701Z layer_outputs = layer_module( 2025-12-04T09:44:49.7019074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.7019646Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.7020091Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7020529Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7020689Z 2025-12-04T09:44:49.7020766Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7020997Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7021344Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7021677Z res = mod(**inputs) 2025-12-04T09:44:49.7022053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7022450Z outputs = self.mobilebert( 2025-12-04T09:44:49.7022863Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7023245Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7023631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7024024Z layer_outputs = layer_module( 2025-12-04T09:44:49.7024397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7024867Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7025340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.7025776Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.7026206Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7026615Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7026762Z 2025-12-04T09:44:49.7026838Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7027064Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7027394Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7027701Z res = mod(**inputs) 2025-12-04T09:44:49.7028064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7028447Z outputs = self.mobilebert( 2025-12-04T09:44:49.7028826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7029212Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7029602Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7029980Z layer_outputs = layer_module( 2025-12-04T09:44:49.7030362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7030823Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7031292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.7031716Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.7032149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.7032577Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7033038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7033535Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7033690Z 2025-12-04T09:44:49.7033768Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7034005Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7034349Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7034664Z res = mod(**inputs) 2025-12-04T09:44:49.7035119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7035504Z outputs = self.mobilebert( 2025-12-04T09:44:49.7035870Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7036287Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7036672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7037062Z layer_outputs = layer_module( 2025-12-04T09:44:49.7037434Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.7037910Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.7038389Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.7038815Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.7039238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.7039646Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.7040048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7040450Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7040595Z 2025-12-04T09:44:49.7040670Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7040866Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7041059Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7041247Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7041437Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7041629Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7041812Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7042004Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7042197Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7042382Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7042608Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7042802Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7042871Z res = mod(**inputs) 2025-12-04T09:44:49.7043141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7043216Z outputs = self.mobilebert( 2025-12-04T09:44:49.7043484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7043554Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7043827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7043894Z layer_outputs = layer_module( 2025-12-04T09:44:49.7044167Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.7044281Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.7044550Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.7044676Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.7044941Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.7045055Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7045383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7045469Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7045473Z 2025-12-04T09:44:49.7045552Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7045682Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7045871Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7045941Z res = mod(**inputs) 2025-12-04T09:44:49.7046203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7046278Z outputs = self.mobilebert( 2025-12-04T09:44:49.7046540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7046611Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7046882Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7046948Z layer_outputs = layer_module( 2025-12-04T09:44:49.7047210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7047312Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7047574Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7047685Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7047947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7048053Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7048058Z 2025-12-04T09:44:49.7048139Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7048235Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7048427Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7048489Z res = mod(**inputs) 2025-12-04T09:44:49.7048756Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7048831Z outputs = self.mobilebert( 2025-12-04T09:44:49.7049092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7049160Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7049432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7049498Z layer_outputs = layer_module( 2025-12-04T09:44:49.7049768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7049858Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7050119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7050273Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7050537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7050659Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7050921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7051022Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7051025Z 2025-12-04T09:44:49.7051105Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7051202Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7051389Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7051473Z res = mod(**inputs) 2025-12-04T09:44:49.7051743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7051818Z outputs = self.mobilebert( 2025-12-04T09:44:49.7052083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7052150Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7052425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7052491Z layer_outputs = layer_module( 2025-12-04T09:44:49.7052764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7052852Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7053124Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7053237Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7053504Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7053606Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7053615Z 2025-12-04T09:44:49.7053689Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7053782Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7053978Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7054039Z res = mod(**inputs) 2025-12-04T09:44:49.7054309Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7054384Z outputs = self.mobilebert( 2025-12-04T09:44:49.7054657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7054731Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7054998Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7055063Z layer_outputs = layer_module( 2025-12-04T09:44:49.7055337Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7055423Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7055689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7055810Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7056106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7056226Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7056479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7056563Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7056566Z 2025-12-04T09:44:49.7056643Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7056736Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7056945Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7057004Z res = mod(**inputs) 2025-12-04T09:44:49.7057270Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7057361Z outputs = self.mobilebert( 2025-12-04T09:44:49.7057629Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7057696Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7057967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7058031Z layer_outputs = layer_module( 2025-12-04T09:44:49.7058299Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7058389Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7058652Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7058764Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7059031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7059141Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7059144Z 2025-12-04T09:44:49.7059215Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7059307Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7059500Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7059560Z res = mod(**inputs) 2025-12-04T09:44:49.7059826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7059901Z outputs = self.mobilebert( 2025-12-04T09:44:49.7060163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7060238Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7060505Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7060572Z layer_outputs = layer_module( 2025-12-04T09:44:49.7060846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7060932Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7061204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7061321Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7061584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7061704Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7061996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7062089Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7062092Z 2025-12-04T09:44:49.7062165Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7062259Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7062449Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7062507Z res = mod(**inputs) 2025-12-04T09:44:49.7062790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7062864Z outputs = self.mobilebert( 2025-12-04T09:44:49.7063126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7063218Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7063482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7063547Z layer_outputs = layer_module( 2025-12-04T09:44:49.7063814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.7063927Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.7064192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7064296Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7064299Z 2025-12-04T09:44:49.7064371Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7064471Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7064653Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7064715Z res = mod(**inputs) 2025-12-04T09:44:49.7064980Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7065045Z outputs = self.mobilebert( 2025-12-04T09:44:49.7065313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7065379Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7065635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7065709Z layer_outputs = layer_module( 2025-12-04T09:44:49.7065966Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7066123Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7066382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.7066496Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.7066761Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7066847Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7066850Z 2025-12-04T09:44:49.7066924Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7067024Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7067206Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7067272Z res = mod(**inputs) 2025-12-04T09:44:49.7067529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7067626Z outputs = self.mobilebert( 2025-12-04T09:44:49.7067897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7067964Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7068235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7068299Z layer_outputs = layer_module( 2025-12-04T09:44:49.7068595Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7068743Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7069013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.7069148Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.7069421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.7069533Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7069795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7069886Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7069891Z 2025-12-04T09:44:49.7069965Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7070068Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7070252Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7070312Z res = mod(**inputs) 2025-12-04T09:44:49.7070587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7070652Z outputs = self.mobilebert( 2025-12-04T09:44:49.7070915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7070989Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7071254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7071329Z layer_outputs = layer_module( 2025-12-04T09:44:49.7071594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.7071746Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.7072017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.7072123Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.7072392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.7072474Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.7072736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7072827Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7072831Z 2025-12-04T09:44:49.7072904Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7072976Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7073059Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7073132Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7073338Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7073415Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7073523Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7073607Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7073681Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7073755Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7073864Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7074059Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7074121Z res = mod(**inputs) 2025-12-04T09:44:49.7074431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7074500Z outputs = self.mobilebert( 2025-12-04T09:44:49.7074793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7074876Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7075147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7075223Z layer_outputs = layer_module( 2025-12-04T09:44:49.7075493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.7075583Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.7075866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.7075988Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.7076279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.7076400Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7076691Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7076786Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7076790Z 2025-12-04T09:44:49.7076865Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7076972Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7077170Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7077232Z res = mod(**inputs) 2025-12-04T09:44:49.7077530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7077600Z outputs = self.mobilebert( 2025-12-04T09:44:49.7077893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7077965Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7078254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7078331Z layer_outputs = layer_module( 2025-12-04T09:44:49.7078617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7078709Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7079002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7079113Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7079407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7079516Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7079520Z 2025-12-04T09:44:49.7079626Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7079735Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7079926Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7079996Z res = mod(**inputs) 2025-12-04T09:44:49.7080269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7080337Z outputs = self.mobilebert( 2025-12-04T09:44:49.7080641Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7080711Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7080988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7081079Z layer_outputs = layer_module( 2025-12-04T09:44:49.7081357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7081457Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7081733Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7081854Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7082142Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7082261Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7082544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7082632Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7082635Z 2025-12-04T09:44:49.7082714Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7082819Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7083012Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7083074Z res = mod(**inputs) 2025-12-04T09:44:49.7083362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7083429Z outputs = self.mobilebert( 2025-12-04T09:44:49.7083717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7083786Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7084074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7084152Z layer_outputs = layer_module( 2025-12-04T09:44:49.7084476Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7084569Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7084829Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7084930Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7085199Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7085301Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7085304Z 2025-12-04T09:44:49.7085383Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7085477Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7085691Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7085760Z res = mod(**inputs) 2025-12-04T09:44:49.7086029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7086092Z outputs = self.mobilebert( 2025-12-04T09:44:49.7086369Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7086437Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7086725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7086789Z layer_outputs = layer_module( 2025-12-04T09:44:49.7087054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7087166Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7087433Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7087557Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7087817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7087928Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7088200Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7088283Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7088286Z 2025-12-04T09:44:49.7088356Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7088456Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7088639Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7088704Z res = mod(**inputs) 2025-12-04T09:44:49.7088964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7089029Z outputs = self.mobilebert( 2025-12-04T09:44:49.7089297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7089363Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7089635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7089700Z layer_outputs = layer_module( 2025-12-04T09:44:49.7089962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7090059Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7090319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7090420Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7090690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7090793Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7090797Z 2025-12-04T09:44:49.7090875Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7090968Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7091153Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7091220Z res = mod(**inputs) 2025-12-04T09:44:49.7091530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7091602Z outputs = self.mobilebert( 2025-12-04T09:44:49.7091865Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7091933Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7092203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7092268Z layer_outputs = layer_module( 2025-12-04T09:44:49.7092553Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7092651Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7092918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7093062Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7093325Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7093438Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7093710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7093793Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7093798Z 2025-12-04T09:44:49.7093875Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7093971Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7094155Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7094221Z res = mod(**inputs) 2025-12-04T09:44:49.7094485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7094550Z outputs = self.mobilebert( 2025-12-04T09:44:49.7094816Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7094883Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7095151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7095217Z layer_outputs = layer_module( 2025-12-04T09:44:49.7095479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.7095599Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.7095859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7095973Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7095976Z 2025-12-04T09:44:49.7096049Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7096142Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7096331Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7096391Z res = mod(**inputs) 2025-12-04T09:44:49.7096654Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7096728Z outputs = self.mobilebert( 2025-12-04T09:44:49.7096990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7097065Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7097355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7097422Z layer_outputs = layer_module( 2025-12-04T09:44:49.7097686Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7097831Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7098097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.7098225Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.7098488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7098579Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7098597Z 2025-12-04T09:44:49.7098671Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7098767Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7098958Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7099018Z res = mod(**inputs) 2025-12-04T09:44:49.7099292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7099357Z outputs = self.mobilebert( 2025-12-04T09:44:49.7099631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7099707Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7099975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7100047Z layer_outputs = layer_module( 2025-12-04T09:44:49.7100315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7100463Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7100735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.7100849Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.7101114Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.7101235Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7101499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7101591Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7101595Z 2025-12-04T09:44:49.7101669Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7101766Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7101957Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7102015Z res = mod(**inputs) 2025-12-04T09:44:49.7102283Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7102348Z outputs = self.mobilebert( 2025-12-04T09:44:49.7102610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7102685Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7102945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7103010Z layer_outputs = layer_module( 2025-12-04T09:44:49.7103314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.7103465Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.7103735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.7103836Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.7104097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.7104202Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.7104460Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7104566Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7104569Z 2025-12-04T09:44:49.7104642Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7104714Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7104792Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7104860Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7104932Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7105008Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7105076Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7105150Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7105220Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7105289Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7105390Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7105570Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7105631Z res = mod(**inputs) 2025-12-04T09:44:49.7106033Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7106104Z outputs = self.mobilebert( 2025-12-04T09:44:49.7106378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7106447Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7106713Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7106790Z layer_outputs = layer_module( 2025-12-04T09:44:49.7107058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.7107139Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.7107413Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.7107532Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.7107804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.7107917Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7108182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7108273Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7108278Z 2025-12-04T09:44:49.7108348Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7108454Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7108635Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7108698Z res = mod(**inputs) 2025-12-04T09:44:49.7109030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7109097Z outputs = self.mobilebert( 2025-12-04T09:44:49.7109359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7109444Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7109708Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7109809Z layer_outputs = layer_module( 2025-12-04T09:44:49.7110077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7110166Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7110441Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7110578Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7110850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7110952Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7110955Z 2025-12-04T09:44:49.7111026Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7111126Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7111310Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7111370Z res = mod(**inputs) 2025-12-04T09:44:49.7111639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7111703Z outputs = self.mobilebert( 2025-12-04T09:44:49.7111976Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7112043Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7112302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7112376Z layer_outputs = layer_module( 2025-12-04T09:44:49.7112638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7112732Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7112992Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7113106Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7113451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7113572Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7113846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7113940Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7113944Z 2025-12-04T09:44:49.7114017Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7114120Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7114310Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7114370Z res = mod(**inputs) 2025-12-04T09:44:49.7114660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7114726Z outputs = self.mobilebert( 2025-12-04T09:44:49.7115036Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7115103Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7115367Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7115440Z layer_outputs = layer_module( 2025-12-04T09:44:49.7115710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7115818Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7116096Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7116200Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7116500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7116604Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7116607Z 2025-12-04T09:44:49.7116679Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7116781Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7116967Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7117032Z res = mod(**inputs) 2025-12-04T09:44:49.7117299Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7117365Z outputs = self.mobilebert( 2025-12-04T09:44:49.7117639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7117708Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7117978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7118050Z layer_outputs = layer_module( 2025-12-04T09:44:49.7118319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7118415Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7118684Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7118802Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7119076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7119190Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7119470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7119554Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7119557Z 2025-12-04T09:44:49.7119630Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7119733Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7119925Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7119983Z res = mod(**inputs) 2025-12-04T09:44:49.7120259Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7120326Z outputs = self.mobilebert( 2025-12-04T09:44:49.7120599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7120665Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7120965Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7121041Z layer_outputs = layer_module( 2025-12-04T09:44:49.7121311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7121406Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7121673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7121796Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7122072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7122173Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7122192Z 2025-12-04T09:44:49.7122275Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7122374Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7122561Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7122630Z res = mod(**inputs) 2025-12-04T09:44:49.7122901Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7122966Z outputs = self.mobilebert( 2025-12-04T09:44:49.7123242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7123310Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7123583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7123650Z layer_outputs = layer_module( 2025-12-04T09:44:49.7123923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7124019Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7124285Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7124400Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7124675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7124790Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7125068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7125154Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7125157Z 2025-12-04T09:44:49.7125232Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7125338Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7125526Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7125593Z res = mod(**inputs) 2025-12-04T09:44:49.7125862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7125927Z outputs = self.mobilebert( 2025-12-04T09:44:49.7126201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7126270Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7126538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7126614Z layer_outputs = layer_module( 2025-12-04T09:44:49.7126911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.7127034Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.7127303Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7127405Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7127409Z 2025-12-04T09:44:49.7127498Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7127610Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7127799Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7127859Z res = mod(**inputs) 2025-12-04T09:44:49.7128122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7128212Z outputs = self.mobilebert( 2025-12-04T09:44:49.7128474Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7128539Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7128808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7128872Z layer_outputs = layer_module( 2025-12-04T09:44:49.7129147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7129296Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7129559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.7129683Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.7129946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7130035Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7130038Z 2025-12-04T09:44:49.7130107Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7130198Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7130383Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7130442Z res = mod(**inputs) 2025-12-04T09:44:49.7130704Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7130773Z outputs = self.mobilebert( 2025-12-04T09:44:49.7131031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7131106Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7131368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7131433Z layer_outputs = layer_module( 2025-12-04T09:44:49.7131698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7131839Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7132111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.7132223Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.7132486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.7132629Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7132893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7132981Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7132984Z 2025-12-04T09:44:49.7133054Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7133148Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7133335Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7133409Z res = mod(**inputs) 2025-12-04T09:44:49.7133686Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7133758Z outputs = self.mobilebert( 2025-12-04T09:44:49.7134044Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7134116Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7134376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7134441Z layer_outputs = layer_module( 2025-12-04T09:44:49.7134709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.7134857Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.7135127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.7135227Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.7135491Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.7135579Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.7135839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7135922Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7135932Z 2025-12-04T09:44:49.7136005Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7136075Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7136151Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7136224Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7136292Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7136370Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7136437Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7136506Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7136586Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7136658Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7136759Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7136939Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7136999Z res = mod(**inputs) 2025-12-04T09:44:49.7137268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7137332Z outputs = self.mobilebert( 2025-12-04T09:44:49.7137593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7137667Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7137930Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7138003Z layer_outputs = layer_module( 2025-12-04T09:44:49.7138322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.7138404Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.7138674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.7138788Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.7139049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.7139187Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7139448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7139561Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7139565Z 2025-12-04T09:44:49.7139637Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7139730Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7139919Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7139977Z res = mod(**inputs) 2025-12-04T09:44:49.7140246Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7140309Z outputs = self.mobilebert( 2025-12-04T09:44:49.7140569Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7140643Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7140910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7140977Z layer_outputs = layer_module( 2025-12-04T09:44:49.7141249Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7141336Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7141600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7141705Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7141967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7142077Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7142080Z 2025-12-04T09:44:49.7142150Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7142249Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7142433Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7142495Z res = mod(**inputs) 2025-12-04T09:44:49.7142766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7142830Z outputs = self.mobilebert( 2025-12-04T09:44:49.7143091Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7143162Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7143427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7143497Z layer_outputs = layer_module( 2025-12-04T09:44:49.7143760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7143847Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7144143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7144256Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7144524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7144636Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7144896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7145001Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7145004Z 2025-12-04T09:44:49.7145074Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7145173Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7145372Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7145430Z res = mod(**inputs) 2025-12-04T09:44:49.7145697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7145761Z outputs = self.mobilebert( 2025-12-04T09:44:49.7146022Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7146096Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7146360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7146430Z layer_outputs = layer_module( 2025-12-04T09:44:49.7146694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7146781Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7147052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7147157Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7147428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7147530Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7147534Z 2025-12-04T09:44:49.7147606Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7147708Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7147893Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7147951Z res = mod(**inputs) 2025-12-04T09:44:49.7148222Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7148289Z outputs = self.mobilebert( 2025-12-04T09:44:49.7148557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7148623Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7148885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7148956Z layer_outputs = layer_module( 2025-12-04T09:44:49.7149220Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7149313Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7149575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7149718Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7149989Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7150100Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7150361Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7150452Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7150471Z 2025-12-04T09:44:49.7150543Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7150644Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7150829Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7150888Z res = mod(**inputs) 2025-12-04T09:44:49.7151178Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7151243Z outputs = self.mobilebert( 2025-12-04T09:44:49.7151512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7151579Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7151843Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7151914Z layer_outputs = layer_module( 2025-12-04T09:44:49.7152179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7152264Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7152530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7152637Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7152909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7153008Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7153012Z 2025-12-04T09:44:49.7153085Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7153258Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7153450Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7153523Z res = mod(**inputs) 2025-12-04T09:44:49.7153788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7153856Z outputs = self.mobilebert( 2025-12-04T09:44:49.7154134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7154204Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7154471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7154549Z layer_outputs = layer_module( 2025-12-04T09:44:49.7154819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7154914Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7155177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7155296Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7155573Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7155720Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7156010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7156098Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7156101Z 2025-12-04T09:44:49.7156176Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7156280Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7156470Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7156555Z res = mod(**inputs) 2025-12-04T09:44:49.7156835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7156903Z outputs = self.mobilebert( 2025-12-04T09:44:49.7157201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7157268Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7157539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7157614Z layer_outputs = layer_module( 2025-12-04T09:44:49.7157885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.7158006Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.7158280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7158381Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7158384Z 2025-12-04T09:44:49.7158469Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7158566Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7158758Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7158828Z res = mod(**inputs) 2025-12-04T09:44:49.7159190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7159291Z outputs = self.mobilebert( 2025-12-04T09:44:49.7159675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7159769Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7160156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7160224Z layer_outputs = layer_module( 2025-12-04T09:44:49.7160495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7160644Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7160908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.7161026Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.7161290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7161384Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7161387Z 2025-12-04T09:44:49.7161460Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7161555Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7161750Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7161813Z res = mod(**inputs) 2025-12-04T09:44:49.7162682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7162764Z outputs = self.mobilebert( 2025-12-04T09:44:49.7163036Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7163111Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7163391Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7163478Z layer_outputs = layer_module( 2025-12-04T09:44:49.7163754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7163904Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7164202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.7164316Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.7164591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.7164711Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7164982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7165069Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7165080Z 2025-12-04T09:44:49.7165155Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7165252Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7165445Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7165506Z res = mod(**inputs) 2025-12-04T09:44:49.7165774Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7165847Z outputs = self.mobilebert( 2025-12-04T09:44:49.7166117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7166190Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7166458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7166526Z layer_outputs = layer_module( 2025-12-04T09:44:49.7166807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.7166970Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.7167248Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.7167357Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.7167620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.7167708Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.7167968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7168051Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7168054Z 2025-12-04T09:44:49.7168133Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7168202Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7168280Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7168349Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7168445Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7168523Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7168593Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7168662Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7168736Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7168805Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7168899Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7169091Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7169177Z res = mod(**inputs) 2025-12-04T09:44:49.7169453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7169518Z outputs = self.mobilebert( 2025-12-04T09:44:49.7169803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7169876Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7170138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7170204Z layer_outputs = layer_module( 2025-12-04T09:44:49.7170473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.7170553Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.7170823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.7170937Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.7171198Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.7171323Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7171586Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7171676Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7171679Z 2025-12-04T09:44:49.7171748Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7171843Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7172035Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7172096Z res = mod(**inputs) 2025-12-04T09:44:49.7172360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7172431Z outputs = self.mobilebert( 2025-12-04T09:44:49.7172696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7172769Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7173028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7173093Z layer_outputs = layer_module( 2025-12-04T09:44:49.7173357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7173441Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7173705Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7173810Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7174072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7174214Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7174218Z 2025-12-04T09:44:49.7174289Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7174392Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7174577Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7174637Z res = mod(**inputs) 2025-12-04T09:44:49.7174909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7174996Z outputs = self.mobilebert( 2025-12-04T09:44:49.7175262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7175340Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7175619Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7175690Z layer_outputs = layer_module( 2025-12-04T09:44:49.7175949Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7176036Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7176306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7176423Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7176689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7176800Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7177064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7177147Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7177150Z 2025-12-04T09:44:49.7177218Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7177310Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7177498Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7177557Z res = mod(**inputs) 2025-12-04T09:44:49.7177824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7177890Z outputs = self.mobilebert( 2025-12-04T09:44:49.7178149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7178222Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7178486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7178558Z layer_outputs = layer_module( 2025-12-04T09:44:49.7178820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7178907Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7179175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7179281Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7179543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7179653Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7179657Z 2025-12-04T09:44:49.7179728Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7179863Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7180049Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7180108Z res = mod(**inputs) 2025-12-04T09:44:49.7180380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7180447Z outputs = self.mobilebert( 2025-12-04T09:44:49.7180715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7180797Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7181055Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7181128Z layer_outputs = layer_module( 2025-12-04T09:44:49.7181411Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7181497Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7181769Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7181882Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7182152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7182267Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7182529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7182623Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7182628Z 2025-12-04T09:44:49.7182699Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7182801Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7182985Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7183044Z res = mod(**inputs) 2025-12-04T09:44:49.7183315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7183380Z outputs = self.mobilebert( 2025-12-04T09:44:49.7183641Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7183717Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7183978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7184055Z layer_outputs = layer_module( 2025-12-04T09:44:49.7184317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7184405Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7184675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7184778Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7185048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7185152Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7185155Z 2025-12-04T09:44:49.7185228Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7185329Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7185512Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7185602Z res = mod(**inputs) 2025-12-04T09:44:49.7185875Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7185940Z outputs = self.mobilebert( 2025-12-04T09:44:49.7186207Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7186273Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7186533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7186626Z layer_outputs = layer_module( 2025-12-04T09:44:49.7186881Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7186995Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7187260Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7187375Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7187645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7187756Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7188019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7188111Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7188115Z 2025-12-04T09:44:49.7188187Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7188290Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7188475Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7188536Z res = mod(**inputs) 2025-12-04T09:44:49.7188808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7188872Z outputs = self.mobilebert( 2025-12-04T09:44:49.7189139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7189205Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7189472Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7189546Z layer_outputs = layer_module( 2025-12-04T09:44:49.7189810Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.7189925Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.7190199Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7190301Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7190304Z 2025-12-04T09:44:49.7190380Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7190472Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7190656Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7190724Z res = mod(**inputs) 2025-12-04T09:44:49.7190985Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7191058Z outputs = self.mobilebert( 2025-12-04T09:44:49.7191321Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7191388Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7191680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7191746Z layer_outputs = layer_module( 2025-12-04T09:44:49.7192008Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7192163Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7192444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.7192564Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.7192826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7192926Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7192932Z 2025-12-04T09:44:49.7193010Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7193105Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7193388Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7193458Z res = mod(**inputs) 2025-12-04T09:44:49.7193743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7193826Z outputs = self.mobilebert( 2025-12-04T09:44:49.7194107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7194177Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7194468Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7194543Z layer_outputs = layer_module( 2025-12-04T09:44:49.7194868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7195014Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7195275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.7195396Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.7195659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.7195779Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7196061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7196157Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7196161Z 2025-12-04T09:44:49.7196248Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7196351Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7196560Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7196625Z res = mod(**inputs) 2025-12-04T09:44:49.7196910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7196991Z outputs = self.mobilebert( 2025-12-04T09:44:49.7197275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7197345Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7197698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7197769Z layer_outputs = layer_module( 2025-12-04T09:44:49.7198065Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.7198227Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.7198514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.7198650Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.7198935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.7199028Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.7199333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7199422Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7199425Z 2025-12-04T09:44:49.7199511Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7199587Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7199662Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7199746Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7199819Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7199901Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7199978Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7200056Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7200138Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7200214Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7200315Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7200524Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7200591Z res = mod(**inputs) 2025-12-04T09:44:49.7200877Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7200953Z outputs = self.mobilebert( 2025-12-04T09:44:49.7201239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7201317Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7201604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7201676Z layer_outputs = layer_module( 2025-12-04T09:44:49.7201967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.7202055Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.7202349Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.7202474Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.7202761Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.7202893Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7203177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7203274Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7203277Z 2025-12-04T09:44:49.7203353Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7203455Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7203660Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7203764Z res = mod(**inputs) 2025-12-04T09:44:49.7204055Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7204133Z outputs = self.mobilebert( 2025-12-04T09:44:49.7204427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7204498Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7204785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7204850Z layer_outputs = layer_module( 2025-12-04T09:44:49.7205117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7205221Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7205487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7205599Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7206031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7206146Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7206149Z 2025-12-04T09:44:49.7206220Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7206316Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7206504Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7206563Z res = mod(**inputs) 2025-12-04T09:44:49.7206828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7206896Z outputs = self.mobilebert( 2025-12-04T09:44:49.7207155Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7207229Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7207486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7207551Z layer_outputs = layer_module( 2025-12-04T09:44:49.7207818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7207906Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7208172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7208287Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7208548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7208669Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7208932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7209022Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7209025Z 2025-12-04T09:44:49.7209099Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7209192Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7209382Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7209440Z res = mod(**inputs) 2025-12-04T09:44:49.7209715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7209837Z outputs = self.mobilebert( 2025-12-04T09:44:49.7210100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7210173Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7210434Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7210499Z layer_outputs = layer_module( 2025-12-04T09:44:49.7210794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7210880Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7211151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7211284Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7211550Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7211659Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7211663Z 2025-12-04T09:44:49.7211736Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7211836Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7212022Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7212082Z res = mod(**inputs) 2025-12-04T09:44:49.7212357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7212424Z outputs = self.mobilebert( 2025-12-04T09:44:49.7212684Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7212762Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7213022Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7213096Z layer_outputs = layer_module( 2025-12-04T09:44:49.7213359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7213447Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7213720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7213834Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7214102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7214218Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7214483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7214575Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7214578Z 2025-12-04T09:44:49.7214648Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7214741Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7214934Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7214995Z res = mod(**inputs) 2025-12-04T09:44:49.7215262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7215327Z outputs = self.mobilebert( 2025-12-04T09:44:49.7215619Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7215696Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7215955Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7216029Z layer_outputs = layer_module( 2025-12-04T09:44:49.7216292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7216377Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7216663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7216764Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7217027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7217151Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7217155Z 2025-12-04T09:44:49.7217227Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7217329Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7217515Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7217576Z res = mod(**inputs) 2025-12-04T09:44:49.7217842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7217909Z outputs = self.mobilebert( 2025-12-04T09:44:49.7218170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7218236Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7218491Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7218567Z layer_outputs = layer_module( 2025-12-04T09:44:49.7218824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7218909Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7219171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7219282Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7219545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7219654Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7219911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7220004Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7220008Z 2025-12-04T09:44:49.7220079Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7220178Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7220360Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7220418Z res = mod(**inputs) 2025-12-04T09:44:49.7220683Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7220747Z outputs = self.mobilebert( 2025-12-04T09:44:49.7221003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7221076Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7221362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7221435Z layer_outputs = layer_module( 2025-12-04T09:44:49.7221701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.7221813Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.7222080Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7222197Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7222200Z 2025-12-04T09:44:49.7222277Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7222369Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7222551Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7222641Z res = mod(**inputs) 2025-12-04T09:44:49.7222906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7222972Z outputs = self.mobilebert( 2025-12-04T09:44:49.7223241Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7223308Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7223574Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7223641Z layer_outputs = layer_module( 2025-12-04T09:44:49.7223907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7224061Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7224326Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.7224444Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.7224705Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7224790Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7224793Z 2025-12-04T09:44:49.7224874Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7224967Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7225151Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7225218Z res = mod(**inputs) 2025-12-04T09:44:49.7225478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7225553Z outputs = self.mobilebert( 2025-12-04T09:44:49.7225817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7225883Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7226154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7226219Z layer_outputs = layer_module( 2025-12-04T09:44:49.7226487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7226635Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7226898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.7227021Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.7227312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.7227431Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7227688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7227772Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7227775Z 2025-12-04T09:44:49.7227854Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7227966Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7228152Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7228219Z res = mod(**inputs) 2025-12-04T09:44:49.7228485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7228577Z outputs = self.mobilebert( 2025-12-04T09:44:49.7228839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7228905Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7229183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7229248Z layer_outputs = layer_module( 2025-12-04T09:44:49.7229519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.7229672Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.7229934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.7230043Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.7230307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.7230386Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.7230655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7230738Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7230741Z 2025-12-04T09:44:49.7230818Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7230889Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7230960Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7231036Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7231107Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7231173Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7231245Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7231318Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7231397Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7231467Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7231562Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7231755Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7231815Z res = mod(**inputs) 2025-12-04T09:44:49.7232079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7232154Z outputs = self.mobilebert( 2025-12-04T09:44:49.7232423Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7232494Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7232810Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7232876Z layer_outputs = layer_module( 2025-12-04T09:44:49.7233209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.7233301Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.7233571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.7233693Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.7233983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.7234107Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7234381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7234491Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7234495Z 2025-12-04T09:44:49.7234579Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7234670Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7234857Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7234919Z res = mod(**inputs) 2025-12-04T09:44:49.7235177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7235254Z outputs = self.mobilebert( 2025-12-04T09:44:49.7235567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7235635Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7235915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7235982Z layer_outputs = layer_module( 2025-12-04T09:44:49.7236256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7236343Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7236606Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7236721Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7236990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7237098Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7237103Z 2025-12-04T09:44:49.7237176Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7237274Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7237470Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7237531Z res = mod(**inputs) 2025-12-04T09:44:49.7237801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7237872Z outputs = self.mobilebert( 2025-12-04T09:44:49.7238143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7238219Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7238486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7238554Z layer_outputs = layer_module( 2025-12-04T09:44:49.7238860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7238950Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7239224Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7239341Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7239611Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7239749Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7240021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7240115Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7240142Z 2025-12-04T09:44:49.7240217Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7240316Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7240512Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7240573Z res = mod(**inputs) 2025-12-04T09:44:49.7240845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7240921Z outputs = self.mobilebert( 2025-12-04T09:44:49.7241189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7241266Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7241534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7241603Z layer_outputs = layer_module( 2025-12-04T09:44:49.7241882Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7241971Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7242248Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7242354Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7242623Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7242736Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7242739Z 2025-12-04T09:44:49.7242811Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7242904Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7243098Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7243160Z res = mod(**inputs) 2025-12-04T09:44:49.7243438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7243506Z outputs = self.mobilebert( 2025-12-04T09:44:49.7243777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7243855Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7244124Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7244200Z layer_outputs = layer_module( 2025-12-04T09:44:49.7244470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7244556Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7244863Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7244981Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7245250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7245371Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7245640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7245753Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7245757Z 2025-12-04T09:44:49.7245830Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7245926Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7246124Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7246202Z res = mod(**inputs) 2025-12-04T09:44:49.7246495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7246562Z outputs = self.mobilebert( 2025-12-04T09:44:49.7246836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7246909Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7247185Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7247251Z layer_outputs = layer_module( 2025-12-04T09:44:49.7247544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7247632Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7247909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7248012Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7248287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7248397Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7248400Z 2025-12-04T09:44:49.7248470Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7248574Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7248760Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7248819Z res = mod(**inputs) 2025-12-04T09:44:49.7249094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7249161Z outputs = self.mobilebert( 2025-12-04T09:44:49.7249431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7249505Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7249769Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7249836Z layer_outputs = layer_module( 2025-12-04T09:44:49.7250104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7250192Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7250468Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7250582Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7250886Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7250998Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7251256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7251346Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7251349Z 2025-12-04T09:44:49.7251419Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7251529Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7251719Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7251777Z res = mod(**inputs) 2025-12-04T09:44:49.7252046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7252130Z outputs = self.mobilebert( 2025-12-04T09:44:49.7252393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7252465Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7252729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7252798Z layer_outputs = layer_module( 2025-12-04T09:44:49.7253059Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.7253172Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.7253440Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7253543Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7253545Z 2025-12-04T09:44:49.7253619Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7253719Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7253898Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7253961Z res = mod(**inputs) 2025-12-04T09:44:49.7254222Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7254287Z outputs = self.mobilebert( 2025-12-04T09:44:49.7254561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7254627Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7254896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7254964Z layer_outputs = layer_module( 2025-12-04T09:44:49.7255228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7255383Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7255645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.7255756Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.7256029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7256114Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7256117Z 2025-12-04T09:44:49.7256198Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7256292Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7256503Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7256571Z res = mod(**inputs) 2025-12-04T09:44:49.7256837Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7256910Z outputs = self.mobilebert( 2025-12-04T09:44:49.7257170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7257255Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7257517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7257581Z layer_outputs = layer_module( 2025-12-04T09:44:49.7257843Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7258017Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7258280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.7258401Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.7258669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.7258781Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7259054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7259138Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7259141Z 2025-12-04T09:44:49.7259222Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7259316Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7259500Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7259568Z res = mod(**inputs) 2025-12-04T09:44:49.7259832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7259905Z outputs = self.mobilebert( 2025-12-04T09:44:49.7260170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7260237Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7260506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7260568Z layer_outputs = layer_module( 2025-12-04T09:44:49.7260832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.7260986Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.7261245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.7261349Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.7261612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.7261695Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.7261969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7262051Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7262055Z 2025-12-04T09:44:49.7262134Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7262234Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7262306Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7262386Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7262455Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7262524Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7262601Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7262669Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7262739Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7262815Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7262928Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7263120Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7263179Z res = mod(**inputs) 2025-12-04T09:44:49.7263445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7263539Z outputs = self.mobilebert( 2025-12-04T09:44:49.7263796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7263862Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7264124Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7264189Z layer_outputs = layer_module( 2025-12-04T09:44:49.7264452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.7264531Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.7264785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.7264902Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.7265160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.7265277Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7265529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7265609Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7265612Z 2025-12-04T09:44:49.7265685Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7265776Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7265956Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7266025Z res = mod(**inputs) 2025-12-04T09:44:49.7266281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7266356Z outputs = self.mobilebert( 2025-12-04T09:44:49.7266611Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7266674Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7266938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7267004Z layer_outputs = layer_module( 2025-12-04T09:44:49.7267266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7267353Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7267609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7267721Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7268012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7268117Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7268129Z 2025-12-04T09:44:49.7268201Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7268295Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7268489Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7268573Z res = mod(**inputs) 2025-12-04T09:44:49.7268849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7268920Z outputs = self.mobilebert( 2025-12-04T09:44:49.7269183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7269274Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7269536Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7269599Z layer_outputs = layer_module( 2025-12-04T09:44:49.7269866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7269951Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7270214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7270337Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7270599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7270721Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7270988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7271070Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7271073Z 2025-12-04T09:44:49.7271151Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7271245Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7271438Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7271498Z res = mod(**inputs) 2025-12-04T09:44:49.7271762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7271833Z outputs = self.mobilebert( 2025-12-04T09:44:49.7272101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7272170Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7272443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7272508Z layer_outputs = layer_module( 2025-12-04T09:44:49.7272782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7272868Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7273206Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7273332Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7273603Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7273751Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7273755Z 2025-12-04T09:44:49.7273833Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7273931Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7274132Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7274194Z res = mod(**inputs) 2025-12-04T09:44:49.7274468Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7274559Z outputs = self.mobilebert( 2025-12-04T09:44:49.7274825Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7274900Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7275162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7275251Z layer_outputs = layer_module( 2025-12-04T09:44:49.7275525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7275610Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7275889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7276005Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7276275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7276398Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7276669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7276765Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7276767Z 2025-12-04T09:44:49.7276840Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7276937Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7277129Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7277191Z res = mod(**inputs) 2025-12-04T09:44:49.7277461Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7277539Z outputs = self.mobilebert( 2025-12-04T09:44:49.7277808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7277885Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7278153Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7278224Z layer_outputs = layer_module( 2025-12-04T09:44:49.7278500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7278587Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7278862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7278965Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7279246Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7279356Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7279359Z 2025-12-04T09:44:49.7279432Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7279529Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7279758Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7279820Z res = mod(**inputs) 2025-12-04T09:44:49.7280095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7280162Z outputs = self.mobilebert( 2025-12-04T09:44:49.7280434Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7280526Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7280799Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7280873Z layer_outputs = layer_module( 2025-12-04T09:44:49.7281143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7281254Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7281530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7281645Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7281917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7282033Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7282303Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7282393Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7282396Z 2025-12-04T09:44:49.7282471Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7282566Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7282769Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7282829Z res = mod(**inputs) 2025-12-04T09:44:49.7283111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7283179Z outputs = self.mobilebert( 2025-12-04T09:44:49.7283452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7283529Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7283804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7283871Z layer_outputs = layer_module( 2025-12-04T09:44:49.7284155Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.7284270Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.7284552Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7284656Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7284659Z 2025-12-04T09:44:49.7284732Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7284835Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7285023Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7285092Z res = mod(**inputs) 2025-12-04T09:44:49.7285365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7285433Z outputs = self.mobilebert( 2025-12-04T09:44:49.7285743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7285812Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7286080Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7286155Z layer_outputs = layer_module( 2025-12-04T09:44:49.7286419Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7286589Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7286857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.7286967Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.7287321Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7287405Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7287408Z 2025-12-04T09:44:49.7287489Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7287586Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7287769Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7287836Z res = mod(**inputs) 2025-12-04T09:44:49.7288117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7288182Z outputs = self.mobilebert( 2025-12-04T09:44:49.7288451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7288515Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7288788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7288854Z layer_outputs = layer_module( 2025-12-04T09:44:49.7289117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7289269Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7289530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.7289648Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.7289918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.7290030Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7290303Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7290388Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7290391Z 2025-12-04T09:44:49.7290472Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7290566Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7290750Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7290820Z res = mod(**inputs) 2025-12-04T09:44:49.7291088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7291155Z outputs = self.mobilebert( 2025-12-04T09:44:49.7291427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7291493Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7291808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7291874Z layer_outputs = layer_module( 2025-12-04T09:44:49.7292137Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.7292295Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.7292573Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.7292682Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.7292947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.7293042Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.7293314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7293399Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7293401Z 2025-12-04T09:44:49.7293475Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7293554Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7293626Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7293700Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7293770Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7293836Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7293910Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7293979Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7294047Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7294122Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7294215Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7294399Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7294466Z res = mod(**inputs) 2025-12-04T09:44:49.7294731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7294804Z outputs = self.mobilebert( 2025-12-04T09:44:49.7295063Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7295130Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7295398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7295462Z layer_outputs = layer_module( 2025-12-04T09:44:49.7295734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.7295814Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.7296077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.7296196Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.7296458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.7296572Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7296839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7296921Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7296925Z 2025-12-04T09:44:49.7297005Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7297126Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7297311Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7297375Z res = mod(**inputs) 2025-12-04T09:44:49.7297633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7297700Z outputs = self.mobilebert( 2025-12-04T09:44:49.7297964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7298050Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7298320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7298385Z layer_outputs = layer_module( 2025-12-04T09:44:49.7298664Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7298761Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7299022Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7299133Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7299395Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7299498Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7299501Z 2025-12-04T09:44:49.7299582Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7299677Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7299865Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7299927Z res = mod(**inputs) 2025-12-04T09:44:49.7300195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7300270Z outputs = self.mobilebert( 2025-12-04T09:44:49.7300534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7300599Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7300868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7300936Z layer_outputs = layer_module( 2025-12-04T09:44:49.7301210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7301297Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7301562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7301686Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7301947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7302066Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7302331Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7302414Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7302417Z 2025-12-04T09:44:49.7302497Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7302590Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7302778Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7302840Z res = mod(**inputs) 2025-12-04T09:44:49.7303132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7303201Z outputs = self.mobilebert( 2025-12-04T09:44:49.7303462Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7303527Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7303798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7303885Z layer_outputs = layer_module( 2025-12-04T09:44:49.7304156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7304240Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7304523Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7304633Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7304894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7305003Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7305006Z 2025-12-04T09:44:49.7305078Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7305171Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7305364Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7305423Z res = mod(**inputs) 2025-12-04T09:44:49.7305802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7305889Z outputs = self.mobilebert( 2025-12-04T09:44:49.7306155Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7306230Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7306490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7306555Z layer_outputs = layer_module( 2025-12-04T09:44:49.7306827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7306916Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7307180Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7307299Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7307563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7307677Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7307938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7308021Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7308030Z 2025-12-04T09:44:49.7308102Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7308200Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7308391Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7308451Z res = mod(**inputs) 2025-12-04T09:44:49.7308713Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7308787Z outputs = self.mobilebert( 2025-12-04T09:44:49.7309103Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7309178Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7309447Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7309511Z layer_outputs = layer_module( 2025-12-04T09:44:49.7309781Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7309893Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7310159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7310302Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7310567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7310677Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7310680Z 2025-12-04T09:44:49.7310751Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7310846Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7311037Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7311095Z res = mod(**inputs) 2025-12-04T09:44:49.7311365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7311430Z outputs = self.mobilebert( 2025-12-04T09:44:49.7311694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7311768Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7312035Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7312100Z layer_outputs = layer_module( 2025-12-04T09:44:49.7312374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7312460Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7312734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7312849Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7313111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7313277Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7313556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7313648Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7313652Z 2025-12-04T09:44:49.7313724Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7313822Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7314024Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7314087Z res = mod(**inputs) 2025-12-04T09:44:49.7314353Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7314427Z outputs = self.mobilebert( 2025-12-04T09:44:49.7314695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7314772Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7315062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7315131Z layer_outputs = layer_module( 2025-12-04T09:44:49.7315406Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.7315519Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.7315790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7315910Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7315913Z 2025-12-04T09:44:49.7315986Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7316091Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7316299Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7316361Z res = mod(**inputs) 2025-12-04T09:44:49.7316640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7316708Z outputs = self.mobilebert( 2025-12-04T09:44:49.7316983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7317052Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7317323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7317399Z layer_outputs = layer_module( 2025-12-04T09:44:49.7317670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7317832Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7318104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.7318219Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.7318492Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7318580Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7318585Z 2025-12-04T09:44:49.7318666Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7318763Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7318952Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7319021Z res = mod(**inputs) 2025-12-04T09:44:49.7319297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7319365Z outputs = self.mobilebert( 2025-12-04T09:44:49.7319646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7319715Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7319990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7320059Z layer_outputs = layer_module( 2025-12-04T09:44:49.7320332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7320491Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7320760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.7320911Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.7321181Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.7321293Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7321560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7321658Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7321661Z 2025-12-04T09:44:49.7321732Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7321831Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7322010Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7322118Z res = mod(**inputs) 2025-12-04T09:44:49.7322384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7322450Z outputs = self.mobilebert( 2025-12-04T09:44:49.7322718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7322782Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7323043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7323115Z layer_outputs = layer_module( 2025-12-04T09:44:49.7323377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.7323533Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.7323803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.7323904Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.7324173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.7324250Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.7324519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7324603Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7324606Z 2025-12-04T09:44:49.7324678Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7324756Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7324824Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7324892Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7324969Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7325041Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7325114Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7325182Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7325249Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7325323Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7325419Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7325600Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7325667Z res = mod(**inputs) 2025-12-04T09:44:49.7325933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7326003Z outputs = self.mobilebert( 2025-12-04T09:44:49.7326268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7326366Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7326635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7326700Z layer_outputs = layer_module( 2025-12-04T09:44:49.7326962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.7327047Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.7327312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.7327452Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.7327712Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.7327845Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7328122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7328205Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7328208Z 2025-12-04T09:44:49.7328284Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7328377Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7328561Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7328630Z res = mod(**inputs) 2025-12-04T09:44:49.7328896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7328961Z outputs = self.mobilebert( 2025-12-04T09:44:49.7329234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7329304Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7329576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7329642Z layer_outputs = layer_module( 2025-12-04T09:44:49.7329904Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7330001Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7330264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7330371Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7330634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7330733Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7330739Z 2025-12-04T09:44:49.7330820Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7331224Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7331414Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7331472Z res = mod(**inputs) 2025-12-04T09:44:49.7331733Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7331808Z outputs = self.mobilebert( 2025-12-04T09:44:49.7332073Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7332141Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7332414Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7332517Z layer_outputs = layer_module( 2025-12-04T09:44:49.7332794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7332882Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7333146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7333270Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7333556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7333676Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7333938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7334042Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7334044Z 2025-12-04T09:44:49.7334124Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7334219Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7334404Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7334467Z res = mod(**inputs) 2025-12-04T09:44:49.7334731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7334803Z outputs = self.mobilebert( 2025-12-04T09:44:49.7335069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7335135Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7335406Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7335475Z layer_outputs = layer_module( 2025-12-04T09:44:49.7335748Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7335837Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7336106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7336217Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7336495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7336596Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7336606Z 2025-12-04T09:44:49.7336678Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7336774Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7336970Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7337030Z res = mod(**inputs) 2025-12-04T09:44:49.7337294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7337366Z outputs = self.mobilebert( 2025-12-04T09:44:49.7337632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7337705Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7337971Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7338035Z layer_outputs = layer_module( 2025-12-04T09:44:49.7338307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7338422Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7338686Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7338806Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7339069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7339188Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7339467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7339547Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7339550Z 2025-12-04T09:44:49.7339622Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7339731Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7339922Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7339981Z res = mod(**inputs) 2025-12-04T09:44:49.7340245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7340320Z outputs = self.mobilebert( 2025-12-04T09:44:49.7340582Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7340649Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7340917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7340982Z layer_outputs = layer_module( 2025-12-04T09:44:49.7341248Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7341337Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7341599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7341708Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7341969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7342074Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7342079Z 2025-12-04T09:44:49.7342150Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7342243Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7342432Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7342494Z res = mod(**inputs) 2025-12-04T09:44:49.7342759Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7342832Z outputs = self.mobilebert( 2025-12-04T09:44:49.7343093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7343163Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7343423Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7343490Z layer_outputs = layer_module( 2025-12-04T09:44:49.7343758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7343843Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7344115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7344262Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7344527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7344644Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7344908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7345015Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7345018Z 2025-12-04T09:44:49.7345089Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7345182Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7345369Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7345450Z res = mod(**inputs) 2025-12-04T09:44:49.7345714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7345786Z outputs = self.mobilebert( 2025-12-04T09:44:49.7346049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7346122Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7346387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7346454Z layer_outputs = layer_module( 2025-12-04T09:44:49.7346727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.7346839Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.7347114Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7347214Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7347217Z 2025-12-04T09:44:49.7347288Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7347389Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7347577Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7347635Z res = mod(**inputs) 2025-12-04T09:44:49.7347910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7347976Z outputs = self.mobilebert( 2025-12-04T09:44:49.7348245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7348313Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7348578Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7348651Z layer_outputs = layer_module( 2025-12-04T09:44:49.7348916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7349072Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7349336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.7349449Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.7349717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7349800Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7349805Z 2025-12-04T09:44:49.7349878Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7350009Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7350195Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7350263Z res = mod(**inputs) 2025-12-04T09:44:49.7350528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7350594Z outputs = self.mobilebert( 2025-12-04T09:44:49.7350861Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7350943Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7351216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7351297Z layer_outputs = layer_module( 2025-12-04T09:44:49.7351560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7351710Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7351974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.7352085Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.7352354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.7352464Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7352731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7352817Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7352820Z 2025-12-04T09:44:49.7352895Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7352996Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7353245Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7353320Z res = mod(**inputs) 2025-12-04T09:44:49.7353596Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7353664Z outputs = self.mobilebert( 2025-12-04T09:44:49.7353937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7354005Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7354277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7354354Z layer_outputs = layer_module( 2025-12-04T09:44:49.7354636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.7354790Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.7355047Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.7355148Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.7355413Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.7355493Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.7355763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7355849Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7355894Z 2025-12-04T09:44:49.7355970Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7356050Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7356121Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7356193Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7356272Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7356342Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7356419Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7356489Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7356580Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7356657Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7356753Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7356938Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7357024Z res = mod(**inputs) 2025-12-04T09:44:49.7357305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7357379Z outputs = self.mobilebert( 2025-12-04T09:44:49.7357652Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7357718Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7358000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7358068Z layer_outputs = layer_module( 2025-12-04T09:44:49.7358342Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.7358432Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.7358716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.7358840Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.7359115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.7359233Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7359514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7359597Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7359601Z 2025-12-04T09:44:49.7359677Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7359772Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7359963Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7360029Z res = mod(**inputs) 2025-12-04T09:44:49.7360309Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7360376Z outputs = self.mobilebert( 2025-12-04T09:44:49.7360656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7360722Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7361002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7361067Z layer_outputs = layer_module( 2025-12-04T09:44:49.7361340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7361436Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7361741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7361854Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7362122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7362224Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7362228Z 2025-12-04T09:44:49.7362307Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7362403Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7362612Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7362681Z res = mod(**inputs) 2025-12-04T09:44:49.7362952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7363041Z outputs = self.mobilebert( 2025-12-04T09:44:49.7363311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7363380Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7363664Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7363731Z layer_outputs = layer_module( 2025-12-04T09:44:49.7364011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7364101Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7364378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7364506Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7364782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7364898Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7365175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7365260Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7365263Z 2025-12-04T09:44:49.7365346Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7365443Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7365635Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7365705Z res = mod(**inputs) 2025-12-04T09:44:49.7365978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7366052Z outputs = self.mobilebert( 2025-12-04T09:44:49.7366322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7366391Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7366674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7366738Z layer_outputs = layer_module( 2025-12-04T09:44:49.7367000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7367097Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7367358Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7367465Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7367758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7367860Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7367864Z 2025-12-04T09:44:49.7367943Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7368035Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7368222Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7368277Z res = mod(**inputs) 2025-12-04T09:44:49.7368560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7368632Z outputs = self.mobilebert( 2025-12-04T09:44:49.7368894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7368977Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7369250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7369314Z layer_outputs = layer_module( 2025-12-04T09:44:49.7369586Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7369673Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7369940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7370063Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7370328Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7370447Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7370720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7370803Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7370806Z 2025-12-04T09:44:49.7370884Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7370979Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7371162Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7371229Z res = mod(**inputs) 2025-12-04T09:44:49.7371502Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7371573Z outputs = self.mobilebert( 2025-12-04T09:44:49.7371838Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7371907Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7372181Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7372244Z layer_outputs = layer_module( 2025-12-04T09:44:49.7372514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7372601Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7372865Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7372976Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7373247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7373348Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7373358Z 2025-12-04T09:44:49.7373459Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7373556Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7373748Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7373807Z res = mod(**inputs) 2025-12-04T09:44:49.7374071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7374145Z outputs = self.mobilebert( 2025-12-04T09:44:49.7374424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7374501Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7374766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7374851Z layer_outputs = layer_module( 2025-12-04T09:44:49.7375121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7375209Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7375467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7375591Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7375854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7375974Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7376236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7376320Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7376323Z 2025-12-04T09:44:49.7376405Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7376498Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7376687Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7376746Z res = mod(**inputs) 2025-12-04T09:44:49.7377006Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7377078Z outputs = self.mobilebert( 2025-12-04T09:44:49.7377341Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7377407Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7377675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7377741Z layer_outputs = layer_module( 2025-12-04T09:44:49.7378012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.7378123Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.7378385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7378493Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7378496Z 2025-12-04T09:44:49.7378567Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7378666Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7378847Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7378905Z res = mod(**inputs) 2025-12-04T09:44:49.7379181Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7379282Z outputs = self.mobilebert( 2025-12-04T09:44:49.7379546Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7379621Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7379880Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7379953Z layer_outputs = layer_module( 2025-12-04T09:44:49.7380245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7380396Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7380664Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.7380795Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.7381069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7381151Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7381154Z 2025-12-04T09:44:49.7381223Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7381323Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7381506Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7381574Z res = mod(**inputs) 2025-12-04T09:44:49.7381842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7381906Z outputs = self.mobilebert( 2025-12-04T09:44:49.7382182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7382247Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7382511Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7382583Z layer_outputs = layer_module( 2025-12-04T09:44:49.7382845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7382996Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7383260Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.7383374Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.7383648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.7383754Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7384020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7384103Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7384106Z 2025-12-04T09:44:49.7384176Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7384278Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7384462Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7384521Z res = mod(**inputs) 2025-12-04T09:44:49.7384790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7384856Z outputs = self.mobilebert( 2025-12-04T09:44:49.7385160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7385226Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7385490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7385564Z layer_outputs = layer_module( 2025-12-04T09:44:49.7385827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.7386001Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.7386266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.7386367Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.7386666Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.7386746Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.7387011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7387096Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7387098Z 2025-12-04T09:44:49.7387167Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7387242Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7387310Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7387380Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7387457Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7387525Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7387594Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7387674Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7387743Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7387823Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7387917Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7388100Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7388168Z res = mod(**inputs) 2025-12-04T09:44:49.7388432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7388496Z outputs = self.mobilebert( 2025-12-04T09:44:49.7388771Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7388839Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7389111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7389180Z layer_outputs = layer_module( 2025-12-04T09:44:49.7389446Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.7389533Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.7389794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.7389914Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.7390178Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.7390293Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7390567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7390650Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7390683Z 2025-12-04T09:44:49.7390755Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7390858Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7391041Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7391106Z res = mod(**inputs) 2025-12-04T09:44:49.7391369Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7391450Z outputs = self.mobilebert( 2025-12-04T09:44:49.7391719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7391785Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7392054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7392136Z layer_outputs = layer_module( 2025-12-04T09:44:49.7392401Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7392495Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7392759Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7392861Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7393139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7393308Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7393312Z 2025-12-04T09:44:49.7393390Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7393491Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7393686Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7393756Z res = mod(**inputs) 2025-12-04T09:44:49.7394029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7394105Z outputs = self.mobilebert( 2025-12-04T09:44:49.7394427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7394494Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7394767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7394834Z layer_outputs = layer_module( 2025-12-04T09:44:49.7395096Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7395197Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7395459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7395579Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7395846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7395957Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7396228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7396309Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7396312Z 2025-12-04T09:44:49.7396388Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7396482Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7396699Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7396768Z res = mod(**inputs) 2025-12-04T09:44:49.7397032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7397096Z outputs = self.mobilebert( 2025-12-04T09:44:49.7397366Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7397448Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7397714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7397779Z layer_outputs = layer_module( 2025-12-04T09:44:49.7398036Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7398159Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7398420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7398529Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7398788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7398887Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7398891Z 2025-12-04T09:44:49.7398969Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7399063Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7399244Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7399309Z res = mod(**inputs) 2025-12-04T09:44:49.7399572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7399643Z outputs = self.mobilebert( 2025-12-04T09:44:49.7399903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7399969Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7400237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7400302Z layer_outputs = layer_module( 2025-12-04T09:44:49.7400568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7400654Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7400914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7401040Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7401305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7401415Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7401686Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7401768Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7401772Z 2025-12-04T09:44:49.7401850Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7401944Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7402127Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7402195Z res = mod(**inputs) 2025-12-04T09:44:49.7402484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7402558Z outputs = self.mobilebert( 2025-12-04T09:44:49.7402825Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7402890Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7403160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7403243Z layer_outputs = layer_module( 2025-12-04T09:44:49.7403509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7403603Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7403867Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7403996Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7404258Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7404358Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7404361Z 2025-12-04T09:44:49.7404440Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7404534Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7404727Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7404784Z res = mod(**inputs) 2025-12-04T09:44:49.7405047Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7405117Z outputs = self.mobilebert( 2025-12-04T09:44:49.7405383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7405454Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7405914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7405987Z layer_outputs = layer_module( 2025-12-04T09:44:49.7406261Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7406351Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7406614Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7406740Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7407005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7407125Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7407387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7407470Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7407473Z 2025-12-04T09:44:49.7407555Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7407652Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7407845Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7407906Z res = mod(**inputs) 2025-12-04T09:44:49.7408168Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7408244Z outputs = self.mobilebert( 2025-12-04T09:44:49.7408556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7408624Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7408895Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7408961Z layer_outputs = layer_module( 2025-12-04T09:44:49.7409230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.7409367Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.7409627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7409736Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7409762Z 2025-12-04T09:44:49.7409836Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7409942Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7410125Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7410184Z res = mod(**inputs) 2025-12-04T09:44:49.7410458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7410525Z outputs = self.mobilebert( 2025-12-04T09:44:49.7410787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7410865Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7411128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7411200Z layer_outputs = layer_module( 2025-12-04T09:44:49.7411469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7411617Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7411889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.7412001Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.7412274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7412358Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7412361Z 2025-12-04T09:44:49.7412431Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7412530Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7412717Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7412777Z res = mod(**inputs) 2025-12-04T09:44:49.7413050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7413114Z outputs = self.mobilebert( 2025-12-04T09:44:49.7413382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7413448Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7413711Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7413783Z layer_outputs = layer_module( 2025-12-04T09:44:49.7414047Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7414202Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7414496Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.7414608Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.7414877Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.7414988Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7415255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7415389Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7415392Z 2025-12-04T09:44:49.7415464Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7415566Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7415774Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7415833Z res = mod(**inputs) 2025-12-04T09:44:49.7416107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7416171Z outputs = self.mobilebert( 2025-12-04T09:44:49.7416443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7416509Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7416777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7416852Z layer_outputs = layer_module( 2025-12-04T09:44:49.7417115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.7417267Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.7417542Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.7417643Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.7417917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.7417998Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.7418264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7418355Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7418358Z 2025-12-04T09:44:49.7418429Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7418509Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7418580Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7418651Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7418731Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7418799Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7418868Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7418944Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7419013Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7419082Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7419185Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7419375Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7419441Z res = mod(**inputs) 2025-12-04T09:44:49.7419706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7419772Z outputs = self.mobilebert( 2025-12-04T09:44:49.7420075Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7420142Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7420403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7420476Z layer_outputs = layer_module( 2025-12-04T09:44:49.7420738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.7420849Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.7421107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.7421219Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.7421513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.7421625Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7421894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7421976Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7421979Z 2025-12-04T09:44:49.7422049Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7422151Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7422336Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7422401Z res = mod(**inputs) 2025-12-04T09:44:49.7422666Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7422732Z outputs = self.mobilebert( 2025-12-04T09:44:49.7423004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7423071Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7423330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7423404Z layer_outputs = layer_module( 2025-12-04T09:44:49.7423671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7423765Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7424028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7424129Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7424402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7424506Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7424509Z 2025-12-04T09:44:49.7424589Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7424682Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7424863Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7424927Z res = mod(**inputs) 2025-12-04T09:44:49.7425189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7425256Z outputs = self.mobilebert( 2025-12-04T09:44:49.7425525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7425592Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7425891Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7425959Z layer_outputs = layer_module( 2025-12-04T09:44:49.7426219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7426314Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7426577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7426714Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7426973Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7427083Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7427372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7427455Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7427458Z 2025-12-04T09:44:49.7427529Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7427629Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7427810Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7427874Z res = mod(**inputs) 2025-12-04T09:44:49.7428136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7428200Z outputs = self.mobilebert( 2025-12-04T09:44:49.7428470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7428537Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7428806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7428871Z layer_outputs = layer_module( 2025-12-04T09:44:49.7429131Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7429225Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7429487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7429590Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7429863Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7429965Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7429970Z 2025-12-04T09:44:49.7430049Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7430144Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7430326Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7430394Z res = mod(**inputs) 2025-12-04T09:44:49.7430657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7430729Z outputs = self.mobilebert( 2025-12-04T09:44:49.7430990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7431055Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7431325Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7431391Z layer_outputs = layer_module( 2025-12-04T09:44:49.7431681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7431779Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7432041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7432162Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7432426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7432555Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7432840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7432940Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7432943Z 2025-12-04T09:44:49.7433026Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7433119Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7433360Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7433431Z res = mod(**inputs) 2025-12-04T09:44:49.7433700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7433765Z outputs = self.mobilebert( 2025-12-04T09:44:49.7434038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7434107Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7434379Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7434446Z layer_outputs = layer_module( 2025-12-04T09:44:49.7434713Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7434812Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7435075Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7435190Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7435456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7435559Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7435563Z 2025-12-04T09:44:49.7435644Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7435739Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7435924Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7435996Z res = mod(**inputs) 2025-12-04T09:44:49.7436262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7436338Z outputs = self.mobilebert( 2025-12-04T09:44:49.7436601Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7436669Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7436945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7437010Z layer_outputs = layer_module( 2025-12-04T09:44:49.7437279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7437367Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7437672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7437800Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7438062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7438175Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7438446Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7438546Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7438549Z 2025-12-04T09:44:49.7438627Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7438722Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7438924Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7438993Z res = mod(**inputs) 2025-12-04T09:44:49.7439258Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7439328Z outputs = self.mobilebert( 2025-12-04T09:44:49.7439590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7439657Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7439927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7439991Z layer_outputs = layer_module( 2025-12-04T09:44:49.7440253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.7440373Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.7440637Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7440745Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7440749Z 2025-12-04T09:44:49.7440820Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7440911Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7441104Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7441165Z res = mod(**inputs) 2025-12-04T09:44:49.7441438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7441503Z outputs = self.mobilebert( 2025-12-04T09:44:49.7441769Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7441845Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7442108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7442173Z layer_outputs = layer_module( 2025-12-04T09:44:49.7442438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7442586Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7442857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.7442967Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.7443230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7443359Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7443362Z 2025-12-04T09:44:49.7443435Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7443534Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7443717Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7443776Z res = mod(**inputs) 2025-12-04T09:44:49.7444047Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7444128Z outputs = self.mobilebert( 2025-12-04T09:44:49.7444393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7444465Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7444754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7444826Z layer_outputs = layer_module( 2025-12-04T09:44:49.7445089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7445234Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7445507Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.7445620Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.7445889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.7445999Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7446270Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7446361Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7446364Z 2025-12-04T09:44:49.7446437Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7446538Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7446721Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7446779Z res = mod(**inputs) 2025-12-04T09:44:49.7447053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7447120Z outputs = self.mobilebert( 2025-12-04T09:44:49.7447381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7447457Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7447722Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7447795Z layer_outputs = layer_module( 2025-12-04T09:44:49.7448059Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.7448209Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.7448482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.7448584Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.7448854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.7448934Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.7449228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7449320Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7449323Z 2025-12-04T09:44:49.7449397Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7449468Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7449547Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7449616Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7449692Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7449777Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7449846Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7449921Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7449989Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7450059Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7450159Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7450361Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7450421Z res = mod(**inputs) 2025-12-04T09:44:49.7450693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7450760Z outputs = self.mobilebert( 2025-12-04T09:44:49.7451032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7451099Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7451364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7451438Z layer_outputs = layer_module( 2025-12-04T09:44:49.7451702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.7451790Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.7452054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.7452167Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.7452441Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.7452554Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7452828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7452912Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7452915Z 2025-12-04T09:44:49.7452987Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7453088Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7453277Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7453336Z res = mod(**inputs) 2025-12-04T09:44:49.7453610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7453675Z outputs = self.mobilebert( 2025-12-04T09:44:49.7453946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7454011Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7454278Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7454349Z layer_outputs = layer_module( 2025-12-04T09:44:49.7454613Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7454730Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7455001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7455102Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7455371Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7455471Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7455492Z 2025-12-04T09:44:49.7455564Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7455667Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7455850Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7455916Z res = mod(**inputs) 2025-12-04T09:44:49.7456197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7456262Z outputs = self.mobilebert( 2025-12-04T09:44:49.7456530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7456594Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7456855Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7456927Z layer_outputs = layer_module( 2025-12-04T09:44:49.7457191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7457286Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7457550Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7457668Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7457936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7458048Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7458315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7458397Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7458401Z 2025-12-04T09:44:49.7458472Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7458573Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7458756Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7458821Z res = mod(**inputs) 2025-12-04T09:44:49.7459088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7459153Z outputs = self.mobilebert( 2025-12-04T09:44:49.7459422Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7459487Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7459749Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7459824Z layer_outputs = layer_module( 2025-12-04T09:44:49.7460086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7460179Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7460441Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7460579Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7460849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7460949Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7460952Z 2025-12-04T09:44:49.7461032Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7461125Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7461310Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7461396Z res = mod(**inputs) 2025-12-04T09:44:49.7461657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7461721Z outputs = self.mobilebert( 2025-12-04T09:44:49.7462014Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7462080Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7462351Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7462417Z layer_outputs = layer_module( 2025-12-04T09:44:49.7462677Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7462772Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7463034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7463159Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7463421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7463538Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7463812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7463894Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7463897Z 2025-12-04T09:44:49.7463968Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7464072Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7464256Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7464322Z res = mod(**inputs) 2025-12-04T09:44:49.7464583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7464648Z outputs = self.mobilebert( 2025-12-04T09:44:49.7464919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7464984Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7465250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7465315Z layer_outputs = layer_module( 2025-12-04T09:44:49.7465575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7465669Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7465931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7466032Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7466301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7466441Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7466445Z 2025-12-04T09:44:49.7466525Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7466618Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7466805Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7466871Z res = mod(**inputs) 2025-12-04T09:44:49.7467141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7467229Z outputs = self.mobilebert( 2025-12-04T09:44:49.7467495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7467561Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7467849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7467915Z layer_outputs = layer_module( 2025-12-04T09:44:49.7468177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7468271Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7468546Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7468671Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7468935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7469047Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7469332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7469414Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7469417Z 2025-12-04T09:44:49.7469497Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7469591Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7469771Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7469837Z res = mod(**inputs) 2025-12-04T09:44:49.7470097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7470164Z outputs = self.mobilebert( 2025-12-04T09:44:49.7470433Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7470500Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7470770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7470835Z layer_outputs = layer_module( 2025-12-04T09:44:49.7471095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.7471210Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.7471470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7471577Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7471580Z 2025-12-04T09:44:49.7471652Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7471745Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7471934Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7471993Z res = mod(**inputs) 2025-12-04T09:44:49.7472287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7472361Z outputs = self.mobilebert( 2025-12-04T09:44:49.7472621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7472695Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7472958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7473040Z layer_outputs = layer_module( 2025-12-04T09:44:49.7473371Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7473525Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7473820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.7473936Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.7474202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7474294Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7474298Z 2025-12-04T09:44:49.7474371Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7474475Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7474659Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7474719Z res = mod(**inputs) 2025-12-04T09:44:49.7474989Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7475056Z outputs = self.mobilebert( 2025-12-04T09:44:49.7475320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7475396Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7475657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7475731Z layer_outputs = layer_module( 2025-12-04T09:44:49.7475991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7476136Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7476407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.7476520Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.7476794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.7476907Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7477166Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7477257Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7477260Z 2025-12-04T09:44:49.7477333Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7477427Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7477615Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7477675Z res = mod(**inputs) 2025-12-04T09:44:49.7477942Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7478039Z outputs = self.mobilebert( 2025-12-04T09:44:49.7478306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7478379Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7478645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7478717Z layer_outputs = layer_module( 2025-12-04T09:44:49.7478997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.7479147Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.7479415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.7479537Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.7479795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.7479886Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.7480147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7480236Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7480241Z 2025-12-04T09:44:49.7480315Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7480386Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7480463Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7480533Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7480602Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7480679Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7480752Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7480829Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7480898Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7480967Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7481068Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7481252Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7481310Z res = mod(**inputs) 2025-12-04T09:44:49.7481578Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7481646Z outputs = self.mobilebert( 2025-12-04T09:44:49.7481908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7481978Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7482238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7482311Z layer_outputs = layer_module( 2025-12-04T09:44:49.7482569Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.7482649Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.7482914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.7483027Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.7483290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.7483402Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7483688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7483781Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7483784Z 2025-12-04T09:44:49.7483854Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7483954Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7484135Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7484193Z res = mod(**inputs) 2025-12-04T09:44:49.7484458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7484540Z outputs = self.mobilebert( 2025-12-04T09:44:49.7484803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7484904Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7485164Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7485237Z layer_outputs = layer_module( 2025-12-04T09:44:49.7485496Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7485582Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7485851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7485953Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7486216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7486318Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7486322Z 2025-12-04T09:44:49.7486393Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7486495Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7486678Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7486739Z res = mod(**inputs) 2025-12-04T09:44:49.7487003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7487067Z outputs = self.mobilebert( 2025-12-04T09:44:49.7487332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7487400Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7487658Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7487732Z layer_outputs = layer_module( 2025-12-04T09:44:49.7487990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7488084Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7488345Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7488460Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7488727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7488838Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7489108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7489191Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7489196Z 2025-12-04T09:44:49.7489297Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7489403Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7489588Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7489647Z res = mod(**inputs) 2025-12-04T09:44:49.7489922Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7489987Z outputs = self.mobilebert( 2025-12-04T09:44:49.7490260Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7490346Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7490610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7490704Z layer_outputs = layer_module( 2025-12-04T09:44:49.7490972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7491067Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7491336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7491438Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7491711Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7491813Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7491817Z 2025-12-04T09:44:49.7491888Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7491989Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7492177Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7492246Z res = mod(**inputs) 2025-12-04T09:44:49.7492512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7492578Z outputs = self.mobilebert( 2025-12-04T09:44:49.7492850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7492916Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7493191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7493257Z layer_outputs = layer_module( 2025-12-04T09:44:49.7493521Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7493616Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7493880Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7493995Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7494266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7494378Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7494650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7494735Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7494739Z 2025-12-04T09:44:49.7494809Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7494913Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7495103Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7495202Z res = mod(**inputs) 2025-12-04T09:44:49.7495466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7495532Z outputs = self.mobilebert( 2025-12-04T09:44:49.7495801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7495868Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7496151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7496224Z layer_outputs = layer_module( 2025-12-04T09:44:49.7496488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7496600Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7496863Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7496967Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7497240Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7497341Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7497344Z 2025-12-04T09:44:49.7497425Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7497519Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7497700Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7497765Z res = mod(**inputs) 2025-12-04T09:44:49.7498028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7498098Z outputs = self.mobilebert( 2025-12-04T09:44:49.7498369Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7498434Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7498703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7498768Z layer_outputs = layer_module( 2025-12-04T09:44:49.7499032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7499124Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7499388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7499509Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7499770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7499881Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7500149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7500232Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7500234Z 2025-12-04T09:44:49.7500314Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7500408Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7500588Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7500653Z res = mod(**inputs) 2025-12-04T09:44:49.7500921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7501013Z outputs = self.mobilebert( 2025-12-04T09:44:49.7501287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7501352Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7501620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7501684Z layer_outputs = layer_module( 2025-12-04T09:44:49.7501967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.7502083Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.7502346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7502468Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7502480Z 2025-12-04T09:44:49.7502550Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7502643Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7502833Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7502891Z res = mod(**inputs) 2025-12-04T09:44:49.7503154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7503228Z outputs = self.mobilebert( 2025-12-04T09:44:49.7503491Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7503563Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7503823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7503892Z layer_outputs = layer_module( 2025-12-04T09:44:49.7504165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7504312Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7504572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.7504691Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.7504963Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7505055Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7505058Z 2025-12-04T09:44:49.7505132Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7505226Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7505420Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7505479Z res = mod(**inputs) 2025-12-04T09:44:49.7505912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7505983Z outputs = self.mobilebert( 2025-12-04T09:44:49.7506251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7506332Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7506597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7506663Z layer_outputs = layer_module( 2025-12-04T09:44:49.7506935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7507162Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7507436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.7507551Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.7507811Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.7507962Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7508222Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7508314Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7508338Z 2025-12-04T09:44:49.7508412Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7508510Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7508702Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7508761Z res = mod(**inputs) 2025-12-04T09:44:49.7509031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7509096Z outputs = self.mobilebert( 2025-12-04T09:44:49.7509358Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7509431Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7509694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7509759Z layer_outputs = layer_module( 2025-12-04T09:44:49.7510030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.7510178Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.7510448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.7510550Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.7510814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.7510903Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.7511167Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7511257Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7511261Z 2025-12-04T09:44:49.7511335Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7511409Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7511487Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7511557Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7511626Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7511706Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7511774Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7511841Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7511918Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7511988Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7512089Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7512271Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7512330Z res = mod(**inputs) 2025-12-04T09:44:49.7512634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7512702Z outputs = self.mobilebert( 2025-12-04T09:44:49.7512968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7513043Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7513352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7513428Z layer_outputs = layer_module( 2025-12-04T09:44:49.7513713Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.7513793Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.7514066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.7514202Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.7514465Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.7514576Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7514833Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7514922Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7514927Z 2025-12-04T09:44:49.7514998Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7515099Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7515280Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7515339Z res = mod(**inputs) 2025-12-04T09:44:49.7515608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7515674Z outputs = self.mobilebert( 2025-12-04T09:44:49.7515931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7516007Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7516265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7516340Z layer_outputs = layer_module( 2025-12-04T09:44:49.7516608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7516694Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7516959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7517067Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7517324Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7517432Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7517435Z 2025-12-04T09:44:49.7517507Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7517607Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7517789Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7517849Z res = mod(**inputs) 2025-12-04T09:44:49.7518114Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7518180Z outputs = self.mobilebert( 2025-12-04T09:44:49.7518479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7518545Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7518804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7518877Z layer_outputs = layer_module( 2025-12-04T09:44:49.7519133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7519219Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7519510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7519625Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7519895Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7520026Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7520284Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7520375Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7520378Z 2025-12-04T09:44:49.7520451Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7520554Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7520737Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7520799Z res = mod(**inputs) 2025-12-04T09:44:49.7521064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7521129Z outputs = self.mobilebert( 2025-12-04T09:44:49.7521391Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7521466Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7521722Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7521795Z layer_outputs = layer_module( 2025-12-04T09:44:49.7522052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7522139Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7522407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7522509Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7522779Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7522883Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7522886Z 2025-12-04T09:44:49.7522956Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7523056Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7523237Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7523303Z res = mod(**inputs) 2025-12-04T09:44:49.7523565Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7523631Z outputs = self.mobilebert( 2025-12-04T09:44:49.7523896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7523962Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7524251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7524327Z layer_outputs = layer_module( 2025-12-04T09:44:49.7524584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7524679Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7524939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7525052Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7525339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7525451Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7525736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7525822Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7525825Z 2025-12-04T09:44:49.7525896Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7525995Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7526180Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7526239Z res = mod(**inputs) 2025-12-04T09:44:49.7526508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7526577Z outputs = self.mobilebert( 2025-12-04T09:44:49.7526846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7526912Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7527179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7527253Z layer_outputs = layer_module( 2025-12-04T09:44:49.7527514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7527605Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7527864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7527966Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7528234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7528333Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7528337Z 2025-12-04T09:44:49.7528407Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7528510Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7528696Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7528760Z res = mod(**inputs) 2025-12-04T09:44:49.7529021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7529086Z outputs = self.mobilebert( 2025-12-04T09:44:49.7529356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7529424Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7529691Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7529758Z layer_outputs = layer_module( 2025-12-04T09:44:49.7530061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7530155Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7530419Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7530533Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7530801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7530929Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7531200Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7531282Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7531302Z 2025-12-04T09:44:49.7531376Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7531480Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7531663Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7531728Z res = mod(**inputs) 2025-12-04T09:44:49.7531990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7532057Z outputs = self.mobilebert( 2025-12-04T09:44:49.7532326Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7532394Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7532657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7532732Z layer_outputs = layer_module( 2025-12-04T09:44:49.7532999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.7533120Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.7533383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7533485Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7533488Z 2025-12-04T09:44:49.7533569Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7533663Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7533858Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7533918Z res = mod(**inputs) 2025-12-04T09:44:49.7534184Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7534259Z outputs = self.mobilebert( 2025-12-04T09:44:49.7534524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7534591Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7534864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7534932Z layer_outputs = layer_module( 2025-12-04T09:44:49.7535202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7535355Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7535616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.7535737Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.7536029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7536124Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7536127Z 2025-12-04T09:44:49.7536203Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7536299Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7536490Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7536550Z res = mod(**inputs) 2025-12-04T09:44:49.7536827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7536900Z outputs = self.mobilebert( 2025-12-04T09:44:49.7537171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7537262Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7537530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7537595Z layer_outputs = layer_module( 2025-12-04T09:44:49.7537867Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7538012Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7538279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.7538392Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.7538654Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.7538778Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7539044Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7539136Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7539139Z 2025-12-04T09:44:49.7539211Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7539304Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7539492Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7539552Z res = mod(**inputs) 2025-12-04T09:44:49.7539812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7539885Z outputs = self.mobilebert( 2025-12-04T09:44:49.7540151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7540230Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7540494Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7540559Z layer_outputs = layer_module( 2025-12-04T09:44:49.7540831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.7540980Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.7541253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.7541356Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.7541625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.7541744Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.7542011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7542094Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7542106Z 2025-12-04T09:44:49.7542178Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7542251Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7542327Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7542414Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7542485Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7542562Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7542630Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7542700Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7542775Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7542859Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7542955Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7543146Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7543206Z res = mod(**inputs) 2025-12-04T09:44:49.7543475Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7543540Z outputs = self.mobilebert( 2025-12-04T09:44:49.7543804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7543881Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7544144Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7544216Z layer_outputs = layer_module( 2025-12-04T09:44:49.7544484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.7544563Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.7544839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.7544952Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.7545218Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.7545341Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7545607Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7545697Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7545702Z 2025-12-04T09:44:49.7545773Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7545868Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7546061Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7546120Z res = mod(**inputs) 2025-12-04T09:44:49.7546391Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7546456Z outputs = self.mobilebert( 2025-12-04T09:44:49.7546720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7546796Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7547060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7547128Z layer_outputs = layer_module( 2025-12-04T09:44:49.7547430Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7547519Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7547789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7547891Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7548153Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7548282Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7548285Z 2025-12-04T09:44:49.7548356Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7548459Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7548640Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7548717Z res = mod(**inputs) 2025-12-04T09:44:49.7548990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7549055Z outputs = self.mobilebert( 2025-12-04T09:44:49.7549319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7549392Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7549655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7549729Z layer_outputs = layer_module( 2025-12-04T09:44:49.7549996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7550081Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7550359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7550475Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7550747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7550859Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7551122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7551217Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7551220Z 2025-12-04T09:44:49.7551292Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7551390Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7551576Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7551638Z res = mod(**inputs) 2025-12-04T09:44:49.7551909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7551973Z outputs = self.mobilebert( 2025-12-04T09:44:49.7552236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7552309Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7552575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7552649Z layer_outputs = layer_module( 2025-12-04T09:44:49.7552914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7553000Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7553382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7553492Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7553763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7553865Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7553868Z 2025-12-04T09:44:49.7553942Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7554061Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7554243Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7554304Z res = mod(**inputs) 2025-12-04T09:44:49.7554578Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7554664Z outputs = self.mobilebert( 2025-12-04T09:44:49.7554938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7555005Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7555268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7555342Z layer_outputs = layer_module( 2025-12-04T09:44:49.7555606Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7555693Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7555964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7556080Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7556351Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7556464Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7556730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7556821Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7556824Z 2025-12-04T09:44:49.7556897Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7556999Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7557181Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7557240Z res = mod(**inputs) 2025-12-04T09:44:49.7557513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7557582Z outputs = self.mobilebert( 2025-12-04T09:44:49.7557851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7557919Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7558180Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7558253Z layer_outputs = layer_module( 2025-12-04T09:44:49.7558515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7558601Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7558870Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7558975Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7559276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7559377Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7559380Z 2025-12-04T09:44:49.7559451Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7559551Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7559735Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7559817Z res = mod(**inputs) 2025-12-04T09:44:49.7560080Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7560145Z outputs = self.mobilebert( 2025-12-04T09:44:49.7560413Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7560499Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7560759Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7560832Z layer_outputs = layer_module( 2025-12-04T09:44:49.7561093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7561185Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7561448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7561562Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7561830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7561946Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7562216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7562299Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7562302Z 2025-12-04T09:44:49.7562373Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7562476Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7562659Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7562720Z res = mod(**inputs) 2025-12-04T09:44:49.7562988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7563054Z outputs = self.mobilebert( 2025-12-04T09:44:49.7563322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7563390Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7563653Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7563726Z layer_outputs = layer_module( 2025-12-04T09:44:49.7563986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.7564105Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.7564369Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7564470Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7564473Z 2025-12-04T09:44:49.7564553Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7564646Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7564858Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7564928Z res = mod(**inputs) 2025-12-04T09:44:49.7565202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7565273Z outputs = self.mobilebert( 2025-12-04T09:44:49.7565544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7565628Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7565895Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7565959Z layer_outputs = layer_module( 2025-12-04T09:44:49.7566225Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7566394Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7566654Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.7566775Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.7567051Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7567135Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7567146Z 2025-12-04T09:44:49.7567218Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7567309Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7567497Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7567558Z res = mod(**inputs) 2025-12-04T09:44:49.7567822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7567894Z outputs = self.mobilebert( 2025-12-04T09:44:49.7568156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7568229Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7568494Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7568562Z layer_outputs = layer_module( 2025-12-04T09:44:49.7568832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7568978Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7569247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.7569368Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.7569630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.7569750Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7570016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7570101Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7570104Z 2025-12-04T09:44:49.7570183Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7570276Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7570464Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7570525Z res = mod(**inputs) 2025-12-04T09:44:49.7570883Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7570960Z outputs = self.mobilebert( 2025-12-04T09:44:49.7571227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7571292Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7571561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7571642Z layer_outputs = layer_module( 2025-12-04T09:44:49.7571908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.7572056Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.7572344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.7572454Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.7572718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.7572806Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.7573067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7573152Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7573155Z 2025-12-04T09:44:49.7573236Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7573308Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7573386Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7573457Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7573530Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7573608Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7573678Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7573749Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7573829Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7573896Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7573988Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7574177Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7574237Z res = mod(**inputs) 2025-12-04T09:44:49.7574505Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7574570Z outputs = self.mobilebert( 2025-12-04T09:44:49.7574834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7574911Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7575176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7575240Z layer_outputs = layer_module( 2025-12-04T09:44:49.7575509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.7575589Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.7575859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.7575972Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.7576231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.7576383Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7576649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7576739Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7576742Z 2025-12-04T09:44:49.7576814Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7576909Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7577099Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7577176Z res = mod(**inputs) 2025-12-04T09:44:49.7577441Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7577515Z outputs = self.mobilebert( 2025-12-04T09:44:49.7577778Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7577872Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7578128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7578195Z layer_outputs = layer_module( 2025-12-04T09:44:49.7578460Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7578547Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7578811Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7578914Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7579169Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7579282Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7579285Z 2025-12-04T09:44:49.7579357Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7579450Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7579637Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7579695Z res = mod(**inputs) 2025-12-04T09:44:49.7579958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7580025Z outputs = self.mobilebert( 2025-12-04T09:44:49.7580280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7580353Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7580609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7580685Z layer_outputs = layer_module( 2025-12-04T09:44:49.7580946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7581032Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7581296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7581412Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7581667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7581786Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7582039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7582158Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7582162Z 2025-12-04T09:44:49.7582235Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7582330Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7582524Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7582584Z res = mod(**inputs) 2025-12-04T09:44:49.7582853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7582936Z outputs = self.mobilebert( 2025-12-04T09:44:49.7583209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7583284Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7583567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7583634Z layer_outputs = layer_module( 2025-12-04T09:44:49.7583906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7583992Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7584260Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7584362Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7584622Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7584731Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7584734Z 2025-12-04T09:44:49.7584807Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7584908Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7585093Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7585152Z res = mod(**inputs) 2025-12-04T09:44:49.7585422Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7585486Z outputs = self.mobilebert( 2025-12-04T09:44:49.7585748Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7585824Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7586084Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7586156Z layer_outputs = layer_module( 2025-12-04T09:44:49.7586422Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7586509Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7586777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7586893Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7587163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7587276Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7587539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7587628Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7587633Z 2025-12-04T09:44:49.7587704Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7587830Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7588016Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7588075Z res = mod(**inputs) 2025-12-04T09:44:49.7588346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7588409Z outputs = self.mobilebert( 2025-12-04T09:44:49.7588671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7588765Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7589030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7589102Z layer_outputs = layer_module( 2025-12-04T09:44:49.7589383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7589468Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7589735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7589837Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7590106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7590207Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7590210Z 2025-12-04T09:44:49.7590282Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7590381Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7590564Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7590625Z res = mod(**inputs) 2025-12-04T09:44:49.7590897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7590963Z outputs = self.mobilebert( 2025-12-04T09:44:49.7591232Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7591298Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7591560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7591635Z layer_outputs = layer_module( 2025-12-04T09:44:49.7591896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7591988Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7592252Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7592368Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7592637Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7592749Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7593011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7593103Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7593106Z 2025-12-04T09:44:49.7593238Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7593352Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7593538Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7593602Z res = mod(**inputs) 2025-12-04T09:44:49.7593911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7597260Z outputs = self.mobilebert( 2025-12-04T09:44:49.7597562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7597639Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7597906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7598007Z layer_outputs = layer_module( 2025-12-04T09:44:49.7598280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.7598395Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.7598719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7598822Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7598826Z 2025-12-04T09:44:49.7598939Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7599035Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7599227Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7599286Z res = mod(**inputs) 2025-12-04T09:44:49.7599550Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7599624Z outputs = self.mobilebert( 2025-12-04T09:44:49.7599891Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7599967Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7600232Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7600298Z layer_outputs = layer_module( 2025-12-04T09:44:49.7600571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7600722Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7600985Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.7601107Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.7601370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7601463Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7601466Z 2025-12-04T09:44:49.7601540Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7601635Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7601827Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7601887Z res = mod(**inputs) 2025-12-04T09:44:49.7602156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7602224Z outputs = self.mobilebert( 2025-12-04T09:44:49.7602488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7602565Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7602829Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7602896Z layer_outputs = layer_module( 2025-12-04T09:44:49.7603186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7603387Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7603658Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.7603772Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.7604053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.7604175Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7604439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7604548Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7604551Z 2025-12-04T09:44:49.7604626Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7604721Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7604912Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7604970Z res = mod(**inputs) 2025-12-04T09:44:49.7605226Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7605302Z outputs = self.mobilebert( 2025-12-04T09:44:49.7605560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7605635Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7606050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7606125Z layer_outputs = layer_module( 2025-12-04T09:44:49.7606399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:44:49.7606550Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:44:49.7606821Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:44:49.7606923Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:44:49.7607190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:44:49.7607281Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:44:49.7607544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7607639Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7607643Z 2025-12-04T09:44:49.7607715Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7607787Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7607868Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7607940Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7608009Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7608086Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7608156Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7608228Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7608304Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7608373Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7608473Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7608654Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7608715Z res = mod(**inputs) 2025-12-04T09:44:49.7609017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7609126Z outputs = self.mobilebert( 2025-12-04T09:44:49.7609389Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7609462Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7609736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7609834Z layer_outputs = layer_module( 2025-12-04T09:44:49.7610093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:44:49.7610173Z self_attention_outputs = self.attention( 2025-12-04T09:44:49.7610468Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:44:49.7610581Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:44:49.7610850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:44:49.7610962Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7611225Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7611316Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7611319Z 2025-12-04T09:44:49.7611389Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7611485Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7611677Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7611737Z res = mod(**inputs) 2025-12-04T09:44:49.7612008Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7612076Z outputs = self.mobilebert( 2025-12-04T09:44:49.7612343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7612418Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7612683Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7612756Z layer_outputs = layer_module( 2025-12-04T09:44:49.7613020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7613107Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7613379Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7613483Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7613745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7613856Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7613859Z 2025-12-04T09:44:49.7613930Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7614035Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7614220Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7614279Z res = mod(**inputs) 2025-12-04T09:44:49.7614552Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7614618Z outputs = self.mobilebert( 2025-12-04T09:44:49.7614901Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7614991Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7615255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7615328Z layer_outputs = layer_module( 2025-12-04T09:44:49.7615589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7615692Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7615966Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7616102Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7616375Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7616489Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7616757Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7616849Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7616852Z 2025-12-04T09:44:49.7616923Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7617023Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7617212Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7617270Z res = mod(**inputs) 2025-12-04T09:44:49.7617541Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7617608Z outputs = self.mobilebert( 2025-12-04T09:44:49.7617868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7617943Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7618207Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7618279Z layer_outputs = layer_module( 2025-12-04T09:44:49.7618546Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7618633Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7618904Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7619009Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7619286Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7619391Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7619395Z 2025-12-04T09:44:49.7619466Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7619570Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7619756Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7619817Z res = mod(**inputs) 2025-12-04T09:44:49.7620088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7620152Z outputs = self.mobilebert( 2025-12-04T09:44:49.7620424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7620490Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7620777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7620867Z layer_outputs = layer_module( 2025-12-04T09:44:49.7621127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7621220Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7621481Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7621611Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7621882Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7622006Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7622275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7622360Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7622363Z 2025-12-04T09:44:49.7622433Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7622531Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7622713Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7622773Z res = mod(**inputs) 2025-12-04T09:44:49.7623041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7623105Z outputs = self.mobilebert( 2025-12-04T09:44:49.7623378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7623446Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7623709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7623785Z layer_outputs = layer_module( 2025-12-04T09:44:49.7624049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7624140Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7624402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:44:49.7624504Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:44:49.7624771Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7624872Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7624876Z 2025-12-04T09:44:49.7624946Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7625046Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7625228Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7625293Z res = mod(**inputs) 2025-12-04T09:44:49.7625553Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7625620Z outputs = self.mobilebert( 2025-12-04T09:44:49.7625887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7625953Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7626223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7626307Z layer_outputs = layer_module( 2025-12-04T09:44:49.7626570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:44:49.7626679Z attention_output = ffn_module(attention_output) 2025-12-04T09:44:49.7626946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:44:49.7627058Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:44:49.7627346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:44:49.7627456Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7627730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7627830Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7627833Z 2025-12-04T09:44:49.7627905Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7628007Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7628190Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7628259Z res = mod(**inputs) 2025-12-04T09:44:49.7628516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7628583Z outputs = self.mobilebert( 2025-12-04T09:44:49.7628845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7628910Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7629168Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7629241Z layer_outputs = layer_module( 2025-12-04T09:44:49.7629495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:44:49.7629614Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:44:49.7629870Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:44:49.7629969Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:44:49.7629973Z 2025-12-04T09:44:49.7630053Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7630146Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7630331Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7630388Z res = mod(**inputs) 2025-12-04T09:44:49.7630648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7630720Z outputs = self.mobilebert( 2025-12-04T09:44:49.7630977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7631045Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7631307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7631375Z layer_outputs = layer_module( 2025-12-04T09:44:49.7631639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7631781Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7632037Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:44:49.7632171Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:44:49.7632451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7632542Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7632545Z 2025-12-04T09:44:49.7632617Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7632710Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7632921Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7632979Z res = mod(**inputs) 2025-12-04T09:44:49.7633325Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:44:49.7633421Z outputs = self.mobilebert( 2025-12-04T09:44:49.7633700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:44:49.7633777Z encoder_outputs = self.encoder( 2025-12-04T09:44:49.7634081Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:44:49.7634147Z layer_outputs = layer_module( 2025-12-04T09:44:49.7634426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:44:49.7634573Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:44:49.7634848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:44:49.7634961Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:44:49.7635231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:44:49.7635349Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:44:49.7635620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:44:49.7635713Z return input_tensor * self.weight + self.bias 2025-12-04T09:44:49.7635716Z 2025-12-04T09:44:49.7635789Z cudagraph partition due to non gpu ops 2025-12-04T09:44:49.7635884Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7636081Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7636141Z res = mod(**inputs) 2025-12-04T09:44:49.7636408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 989, in forward 2025-12-04T09:44:49.7636502Z prediction_scores = self.cls(sequence_output) 2025-12-04T09:44:49.7636771Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 643, in forward 2025-12-04T09:44:49.7636884Z prediction_scores = self.predictions(sequence_output) 2025-12-04T09:44:49.7637150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 632, in forward 2025-12-04T09:44:49.7637342Z hidden_states = hidden_states.matmul(torch.cat([self.decoder.weight.t(), self.dense.weight], dim=0)) 2025-12-04T09:44:49.7637347Z 2025-12-04T09:44:49.7637446Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7637632Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7637700Z res = mod(**inputs) 2025-12-04T09:44:49.7637964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 994, in forward 2025-12-04T09:44:49.7638157Z masked_lm_loss = loss_fct(prediction_scores.view(-1, self.config.vocab_size), labels.view(-1)) 2025-12-04T09:44:49.7638161Z 2025-12-04T09:44:49.7638278Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:49.7638463Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:49.7638521Z res = mod(**inputs) 2025-12-04T09:44:49.7638792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 989, in forward 2025-12-04T09:44:49.7638895Z prediction_scores = self.cls(sequence_output) 2025-12-04T09:44:49.7639169Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 643, in forward 2025-12-04T09:44:49.7639271Z prediction_scores = self.predictions(sequence_output) 2025-12-04T09:44:49.7639561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 633, in forward 2025-12-04T09:44:49.7639640Z hidden_states += self.decoder.bias 2025-12-04T09:44:49.7639643Z 2025-12-04T09:45:02.1757950Z Compilation time (from dynamo_timed): 41.953640635 2025-12-04T09:45:02.1759498Z pass 2025-12-04T09:45:02.1759919Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:45:02.1765383Z TIMING: _recursive_pre_grad_passes:0.1194 _recursive_joint_graph_passes:1.28419 _recursive_post_grad_passes:0.14269 async_compile.wait:0.76673 code_gen:11.43641 inductor_compile:15.87073 backend_compile:31.73683 gc:0.00051 entire_frame_compile:41.95364 total_wall_time:41.95364 2025-12-04T09:45:02.1766457Z STATS: call_* op count: 1449 | FakeTensorMode.__torch_dispatch__:76974 | FakeTensor.__torch_dispatch__:11426 | ProxyTorchDispatchMode.__torch_dispatch__:10397 2025-12-04T09:45:02.1766973Z Dynamo produced 1 graphs covering 1449 ops with 0 graph breaks (0 unique) 2025-12-04T09:45:05.6309544Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:45:05.6311046Z import pynvml # type: ignore[import] 2025-12-04T09:45:08.7016824Z 2025-12-04T09:45:10.1571309Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:45:10.1572316Z loading model: 0it [00:01, ?it/s] 2025-12-04T09:45:10.1582302Z cpu eval OPTForCausalLM 2025-12-04T09:45:11.7547084Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:45:12.4278938Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:45:13.1131615Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:45:21.0867194Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0867743Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0868669Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0868953Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0869207Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0869423Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0869655Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0869848Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0870046Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0870254Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0870445Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0870645Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0870847Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0871041Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0871246Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0871445Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0871956Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0872161Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0872360Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0872690Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.0873070Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.0873537Z res = mod(**inputs) 2025-12-04T09:45:21.0873909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0874409Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0874800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.0875190Z outputs = self.model.decoder( 2025-12-04T09:45:21.0875608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0875959Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0876336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.0876710Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.0877056Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.0877421Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.0877810Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.0878188Z return func(*args, **kwargs) 2025-12-04T09:45:21.0878543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:45:21.0878948Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:45:21.0879353Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.0879723Z return func(*args, **kwargs) 2025-12-04T09:45:21.0880075Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:45:21.0880482Z attn_output, attn_weights = attention_interface( 2025-12-04T09:45:21.0880928Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:45:21.0881411Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:45:21.0881598Z 2025-12-04T09:45:21.0881679Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0881882Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0882114Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.0882468Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.0882784Z res = mod(**inputs) 2025-12-04T09:45:21.0883110Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0883460Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0883823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.0884202Z outputs = self.model.decoder( 2025-12-04T09:45:21.0884544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0884886Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0885254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.0885624Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.0886002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.0886359Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.0886757Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.0887130Z return func(*args, **kwargs) 2025-12-04T09:45:21.0887480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:45:21.0887878Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:45:21.0888060Z 2025-12-04T09:45:21.0888138Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0888340Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0888533Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0888732Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0888957Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0889154Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0889352Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0889550Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0889764Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.0890123Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.0890438Z res = mod(**inputs) 2025-12-04T09:45:21.0890765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0891108Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0891473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.0891850Z outputs = self.model.decoder( 2025-12-04T09:45:21.0892183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0892527Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0892893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.0893269Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.0893617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.0893979Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.0894351Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.0894721Z return func(*args, **kwargs) 2025-12-04T09:45:21.0895070Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:45:21.0895463Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:45:21.0895859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.0896217Z return func(*args, **kwargs) 2025-12-04T09:45:21.0896574Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:45:21.0896972Z attn_output, attn_weights = attention_interface( 2025-12-04T09:45:21.0897409Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:45:21.0897885Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:45:21.0898067Z 2025-12-04T09:45:21.0898142Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0898339Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0898551Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.0898891Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.0899198Z res = mod(**inputs) 2025-12-04T09:45:21.0899528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0899866Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0900287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.0900680Z outputs = self.model.decoder( 2025-12-04T09:45:21.0901011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0901370Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0901724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.0902087Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.0902426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.0902793Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.0903163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.0903522Z return func(*args, **kwargs) 2025-12-04T09:45:21.0903871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:45:21.0904250Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:45:21.0904406Z 2025-12-04T09:45:21.0904484Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0904783Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0904981Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0905168Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0905362Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0905553Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0905940Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0906144Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0906369Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.0906709Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.0907022Z res = mod(**inputs) 2025-12-04T09:45:21.0907340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0907684Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0908035Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.0908400Z outputs = self.model.decoder( 2025-12-04T09:45:21.0908731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0909106Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0909469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.0909830Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.0910167Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.0910507Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.0910874Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.0911240Z return func(*args, **kwargs) 2025-12-04T09:45:21.0911580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:45:21.0911962Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:45:21.0912347Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.0912707Z return func(*args, **kwargs) 2025-12-04T09:45:21.0913087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:45:21.0913573Z attn_output, attn_weights = attention_interface( 2025-12-04T09:45:21.0914027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:45:21.0914514Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:45:21.0914691Z 2025-12-04T09:45:21.0914797Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0915003Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0915230Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.0915569Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.0915914Z res = mod(**inputs) 2025-12-04T09:45:21.0916238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0916580Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0916935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.0917303Z outputs = self.model.decoder( 2025-12-04T09:45:21.0917642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0917973Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0918334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.0918694Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.0919038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.0919389Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.0919756Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.0920114Z return func(*args, **kwargs) 2025-12-04T09:45:21.0920454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:45:21.0920837Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:45:21.0920989Z 2025-12-04T09:45:21.0921064Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0921264Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0921455Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0921647Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0921842Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0922030Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0922222Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0922418Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0922632Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.0922974Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.0923284Z res = mod(**inputs) 2025-12-04T09:45:21.0923596Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0923926Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0924282Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.0924646Z outputs = self.model.decoder( 2025-12-04T09:45:21.0924971Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0925307Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0925671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.0926054Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.0926395Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.0926775Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.0927133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.0927482Z return func(*args, **kwargs) 2025-12-04T09:45:21.0927817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:45:21.0928220Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:45:21.0928615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.0928990Z return func(*args, **kwargs) 2025-12-04T09:45:21.0929340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:45:21.0929731Z attn_output, attn_weights = attention_interface( 2025-12-04T09:45:21.0930168Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:45:21.0930625Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:45:21.0930806Z 2025-12-04T09:45:21.0930880Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0931087Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0931301Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.0931645Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.0931955Z res = mod(**inputs) 2025-12-04T09:45:21.0932268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0932603Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0932963Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.0933327Z outputs = self.model.decoder( 2025-12-04T09:45:21.0933655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0933994Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0934353Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.0934715Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.0935042Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.0935390Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.0935760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.0936121Z return func(*args, **kwargs) 2025-12-04T09:45:21.0936465Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:45:21.0936853Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:45:21.0936998Z 2025-12-04T09:45:21.0937076Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0937267Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0937462Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0937659Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0937851Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0938037Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0938231Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0938421Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0938635Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.0938992Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.0939304Z res = mod(**inputs) 2025-12-04T09:45:21.0939629Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0939971Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0940336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.0940700Z outputs = self.model.decoder( 2025-12-04T09:45:21.0941942Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0942289Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0942651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.0943025Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.0943365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.0943718Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.0944084Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.0944436Z return func(*args, **kwargs) 2025-12-04T09:45:21.0944787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:45:21.0945174Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:45:21.0945548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.0945907Z return func(*args, **kwargs) 2025-12-04T09:45:21.0946253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:45:21.0946643Z attn_output, attn_weights = attention_interface( 2025-12-04T09:45:21.0947064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:45:21.0947534Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:45:21.0947712Z 2025-12-04T09:45:21.0947788Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0947986Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0948201Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.0948543Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.0948853Z res = mod(**inputs) 2025-12-04T09:45:21.0949163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0949509Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0949868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.0950228Z outputs = self.model.decoder( 2025-12-04T09:45:21.0950558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0950900Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0951253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.0951604Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.0951944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.0952293Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.0952658Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.0953019Z return func(*args, **kwargs) 2025-12-04T09:45:21.0953493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:45:21.0953922Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:45:21.0954083Z 2025-12-04T09:45:21.0954166Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0954359Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0954559Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0954762Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0954971Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0955165Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0955361Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0955545Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0955774Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.0956154Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.0956464Z res = mod(**inputs) 2025-12-04T09:45:21.0956789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0957139Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0957507Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.0957873Z outputs = self.model.decoder( 2025-12-04T09:45:21.0958215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0958567Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0958934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.0959296Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.0959644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.0960008Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.0960378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.0960752Z return func(*args, **kwargs) 2025-12-04T09:45:21.0961105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:45:21.0961505Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:45:21.0961895Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.0962272Z return func(*args, **kwargs) 2025-12-04T09:45:21.0962626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:45:21.0963013Z attn_output, attn_weights = attention_interface( 2025-12-04T09:45:21.0963454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:45:21.0963925Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:45:21.0964103Z 2025-12-04T09:45:21.0964190Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0964387Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0964614Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.0964969Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.0965284Z res = mod(**inputs) 2025-12-04T09:45:21.0965597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0965946Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0966335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.0966707Z outputs = self.model.decoder( 2025-12-04T09:45:21.0967068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0967409Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0967765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.0968119Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.0968465Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.0968806Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.0969155Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.0969533Z return func(*args, **kwargs) 2025-12-04T09:45:21.0969879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:45:21.0970271Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:45:21.0970416Z 2025-12-04T09:45:21.0970490Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0970687Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0970882Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0971068Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0971265Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0971460Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0971645Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0971839Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0972067Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.0972410Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.0972720Z res = mod(**inputs) 2025-12-04T09:45:21.0973041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0973386Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0973743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.0974109Z outputs = self.model.decoder( 2025-12-04T09:45:21.0974445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0974791Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0975147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.0975515Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.0975858Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.0976203Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.0976571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.0976936Z return func(*args, **kwargs) 2025-12-04T09:45:21.0977290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:45:21.0977671Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:45:21.0978056Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.0978419Z return func(*args, **kwargs) 2025-12-04T09:45:21.0978769Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:45:21.0979148Z attn_output, attn_weights = attention_interface( 2025-12-04T09:45:21.0979601Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:45:21.0980066Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:45:21.0980240Z 2025-12-04T09:45:21.0980333Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0980532Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0980756Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.0981103Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.0981424Z res = mod(**inputs) 2025-12-04T09:45:21.0981746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0982095Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0982451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.0982827Z outputs = self.model.decoder( 2025-12-04T09:45:21.0983159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0983496Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0983844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.0984205Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.0984536Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.0984880Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.0985243Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.0985603Z return func(*args, **kwargs) 2025-12-04T09:45:21.0985951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:45:21.0986330Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:45:21.0986481Z 2025-12-04T09:45:21.0986554Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0986756Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0986952Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0987137Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0987332Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0987525Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0987714Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0987909Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.0988131Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.0988463Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.0988774Z res = mod(**inputs) 2025-12-04T09:45:21.0989094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0989436Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0989790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.0990149Z outputs = self.model.decoder( 2025-12-04T09:45:21.0990481Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.0990813Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.0991172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.0991534Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.0991869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.0992213Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.0992600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.0993083Z return func(*args, **kwargs) 2025-12-04T09:45:21.1020399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:45:21.1020862Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:45:21.1021275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.1021904Z return func(*args, **kwargs) 2025-12-04T09:45:21.1022352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:45:21.1022743Z attn_output, attn_weights = attention_interface( 2025-12-04T09:45:21.1023183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:45:21.1023710Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:45:21.1023892Z 2025-12-04T09:45:21.1023985Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1024193Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1024433Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.1024831Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.1025172Z res = mod(**inputs) 2025-12-04T09:45:21.1025510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.1025869Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.1026244Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.1026617Z outputs = self.model.decoder( 2025-12-04T09:45:21.1026993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.1027343Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.1027718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.1028081Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.1028426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.1028804Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.1029174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.1029541Z return func(*args, **kwargs) 2025-12-04T09:45:21.1029896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:45:21.1030290Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:45:21.1030442Z 2025-12-04T09:45:21.1030520Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1030725Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1030927Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1031119Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1031316Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1031513Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1031701Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1031897Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1032128Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.1032478Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.1032785Z res = mod(**inputs) 2025-12-04T09:45:21.1033103Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.1033638Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.1034015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.1034431Z outputs = self.model.decoder( 2025-12-04T09:45:21.1034786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.1035136Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.1035501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.1035895Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.1036248Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.1036612Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.1037018Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.1037393Z return func(*args, **kwargs) 2025-12-04T09:45:21.1058604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:45:21.1059205Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:45:21.1059644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.1060023Z return func(*args, **kwargs) 2025-12-04T09:45:21.1060388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:45:21.1060779Z attn_output, attn_weights = attention_interface( 2025-12-04T09:45:21.1061213Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:45:21.1061674Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:45:21.1061854Z 2025-12-04T09:45:21.1061933Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1062129Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1062344Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.1062689Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.1062992Z res = mod(**inputs) 2025-12-04T09:45:21.1063297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.1063633Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.1063981Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.1064334Z outputs = self.model.decoder( 2025-12-04T09:45:21.1064656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.1064983Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.1065328Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.1065677Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.1066002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.1066341Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.1066693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.1067034Z return func(*args, **kwargs) 2025-12-04T09:45:21.1067365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:45:21.1067729Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:45:21.1067877Z 2025-12-04T09:45:21.1068069Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1068266Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1068449Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1068660Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1068840Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1069018Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1069196Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1069371Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1069578Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.1069935Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.1070234Z res = mod(**inputs) 2025-12-04T09:45:21.1070538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.1070924Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.1071274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.1071624Z outputs = self.model.decoder( 2025-12-04T09:45:21.1071945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.1072274Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.1072621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.1073017Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.1073481Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.1073855Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.1074240Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.1074611Z return func(*args, **kwargs) 2025-12-04T09:45:21.1074959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:45:21.1075395Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:45:21.1075823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.1076241Z return func(*args, **kwargs) 2025-12-04T09:45:21.1076626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:45:21.1077063Z attn_output, attn_weights = attention_interface( 2025-12-04T09:45:21.1077549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:45:21.1078085Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:45:21.1078282Z 2025-12-04T09:45:21.1078374Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1078587Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1078833Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.1079229Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.1079582Z res = mod(**inputs) 2025-12-04T09:45:21.1079946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.1080352Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.1080762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.1081195Z outputs = self.model.decoder( 2025-12-04T09:45:21.1081551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.1081910Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.1082285Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.1082645Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.1082994Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.1083333Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.1083700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.1084076Z return func(*args, **kwargs) 2025-12-04T09:45:21.1084424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:45:21.1084801Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:45:21.1084951Z 2025-12-04T09:45:21.1085040Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1085239Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1085426Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1085619Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1085810Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1086001Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1086181Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1086369Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1086585Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.1086915Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.1087222Z res = mod(**inputs) 2025-12-04T09:45:21.1087532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.1087861Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.1088219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.1088579Z outputs = self.model.decoder( 2025-12-04T09:45:21.1088907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.1089238Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.1089593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.1089951Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.1090286Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.1090627Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.1090993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.1091355Z return func(*args, **kwargs) 2025-12-04T09:45:21.1091696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:45:21.1092079Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:45:21.1092461Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.1092817Z return func(*args, **kwargs) 2025-12-04T09:45:21.1093157Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:45:21.1093545Z attn_output, attn_weights = attention_interface( 2025-12-04T09:45:21.1093974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:45:21.1094435Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:45:21.1094612Z 2025-12-04T09:45:21.1094688Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1094891Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1095129Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.1095487Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.1095795Z res = mod(**inputs) 2025-12-04T09:45:21.1096106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.1096441Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.1096797Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.1097171Z outputs = self.model.decoder( 2025-12-04T09:45:21.1097501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.1097833Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.1098225Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.1098575Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.1098906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.1099243Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.1099599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.1099949Z return func(*args, **kwargs) 2025-12-04T09:45:21.1100284Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:45:21.1100655Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:45:21.1100802Z 2025-12-04T09:45:21.1100875Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1101071Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1101262Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1101451Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1101640Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1101820Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1102010Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1102199Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1102406Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.1102736Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.1103037Z res = mod(**inputs) 2025-12-04T09:45:21.1103347Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.1103668Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.1104013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.1104365Z outputs = self.model.decoder( 2025-12-04T09:45:21.1104680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.1105008Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.1105360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.1105928Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.1106265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.1106622Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.1106991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.1107344Z return func(*args, **kwargs) 2025-12-04T09:45:21.1107702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:45:21.1108191Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:45:21.1108596Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.1108939Z return func(*args, **kwargs) 2025-12-04T09:45:21.1109279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:45:21.1109657Z attn_output, attn_weights = attention_interface( 2025-12-04T09:45:21.1110117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:45:21.1110574Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:45:21.1110760Z 2025-12-04T09:45:21.1110834Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1111061Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1111277Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.1111620Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.1111930Z res = mod(**inputs) 2025-12-04T09:45:21.1112251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.1112583Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.1112948Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:45:21.1113381Z outputs = self.model.decoder( 2025-12-04T09:45:21.1113720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.1114068Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.1114432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:45:21.1114808Z layer_outputs = decoder_layer( 2025-12-04T09:45:21.1115132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:21.1115477Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:21.1115839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:21.1116194Z return func(*args, **kwargs) 2025-12-04T09:45:21.1116532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:45:21.1116914Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:45:21.1117058Z 2025-12-04T09:45:21.1117140Z cudagraph partition due to non gpu ops 2025-12-04T09:45:21.1117353Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.1117693Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.1117998Z res = mod(**inputs) 2025-12-04T09:45:21.1118305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.1118629Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.1118980Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 833, in forward 2025-12-04T09:45:21.1119352Z logits = self.lm_head(outputs[0]).contiguous() 2025-12-04T09:45:21.1119487Z 2025-12-04T09:45:21.1119583Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:21.1119920Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:21.1120226Z res = mod(**inputs) 2025-12-04T09:45:21.1120530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:45:21.1120859Z output = func(self, *args, **kwargs) 2025-12-04T09:45:21.1121225Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 839, in forward 2025-12-04T09:45:21.1121591Z loss = self.loss_function( 2025-12-04T09:45:21.1121948Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 67, in ForCausalLMLoss 2025-12-04T09:45:21.1122410Z loss = fixed_cross_entropy(logits, shift_labels, num_items_in_batch, ignore_index, **kwargs) 2025-12-04T09:45:21.1122876Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 36, in fixed_cross_entropy 2025-12-04T09:45:21.1123386Z loss = nn.functional.cross_entropy(source, target, ignore_index=ignore_index, reduction=reduction) 2025-12-04T09:45:21.1123628Z 2025-12-04T09:45:31.5937540Z Compilation time (from dynamo_timed): 17.121226688 2025-12-04T09:45:31.6394706Z pass 2025-12-04T09:45:31.6396809Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:45:31.6397689Z TIMING: _recursive_pre_grad_passes:0.03119 _recursive_joint_graph_passes:0.61667 _recursive_post_grad_passes:0.06578 async_compile.wait:0.83265 code_gen:9.81417 inductor_compile:11.40717 backend_compile:14.88141 gc:0.00037 entire_frame_compile:17.12123 total_wall_time:17.12123 2025-12-04T09:45:31.6401910Z STATS: call_* op count: 379 | FakeTensorMode.__torch_dispatch__:17830 | FakeTensor.__torch_dispatch__:3248 | ProxyTorchDispatchMode.__torch_dispatch__:2777 2025-12-04T09:45:31.6403728Z Dynamo produced 1 graphs covering 379 ops with 0 graph breaks (0 unique) 2025-12-04T09:45:33.9482584Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:45:33.9484854Z import pynvml # type: ignore[import] 2025-12-04T09:45:37.1083891Z 2025-12-04T09:45:38.1260363Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:45:38.1261825Z loading model: 0it [00:01, ?it/s] 2025-12-04T09:45:38.1268814Z cpu eval PLBartForCausalLM 2025-12-04T09:45:38.7896458Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:45:39.0581149Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:45:39.3218393Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:45:44.2675625Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2678492Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2678708Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2678920Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2679124Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2679350Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2679546Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2679744Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2679967Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2680168Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2680364Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2680553Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2680760Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2680995Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:44.2681388Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:44.2681724Z res = mod(**inputs) 2025-12-04T09:45:44.2682127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:45:44.2682534Z outputs = self.model.decoder( 2025-12-04T09:45:44.2683193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:45:44.2683596Z layer_outputs = decoder_layer( 2025-12-04T09:45:44.2684005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:44.2684367Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:44.2684753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:44.2685126Z return func(*args, **kwargs) 2025-12-04T09:45:44.2685619Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 766, in forward 2025-12-04T09:45:44.2686099Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:45:44.2686508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:44.2686936Z return func(*args, **kwargs) 2025-12-04T09:45:44.2687313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 443, in forward 2025-12-04T09:45:44.2687731Z attn_output, attn_weights = attention_interface( 2025-12-04T09:45:44.2688179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:45:44.2688660Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:45:44.2688844Z 2025-12-04T09:45:44.2688929Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2689127Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2689357Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:44.2689715Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:44.2690035Z res = mod(**inputs) 2025-12-04T09:45:44.2690407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:45:44.2690819Z outputs = self.model.decoder( 2025-12-04T09:45:44.2691199Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:45:44.2691590Z layer_outputs = decoder_layer( 2025-12-04T09:45:44.2691932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:44.2692287Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:44.2692651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:44.2693011Z return func(*args, **kwargs) 2025-12-04T09:45:44.2693377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 798, in forward 2025-12-04T09:45:44.2693802Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:45:44.2694173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:44.2694509Z return self.act(input) 2025-12-04T09:45:44.2694615Z 2025-12-04T09:45:44.2694699Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2694890Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2695083Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2695275Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2695459Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2695651Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2695840Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2696028Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2696241Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:44.2696583Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:44.2696892Z res = mod(**inputs) 2025-12-04T09:45:44.2697258Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:45:44.2697679Z outputs = self.model.decoder( 2025-12-04T09:45:44.2698055Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:45:44.2698436Z layer_outputs = decoder_layer( 2025-12-04T09:45:44.2698770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:44.2699136Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:44.2699498Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:44.2699849Z return func(*args, **kwargs) 2025-12-04T09:45:44.2700235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 766, in forward 2025-12-04T09:45:44.2700640Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:45:44.2701024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:44.2701374Z return func(*args, **kwargs) 2025-12-04T09:45:44.2701728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 443, in forward 2025-12-04T09:45:44.2702125Z attn_output, attn_weights = attention_interface( 2025-12-04T09:45:44.2702549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:45:44.2703002Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:45:44.2703180Z 2025-12-04T09:45:44.2703254Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2703452Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2703666Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:44.2704002Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:44.2704312Z res = mod(**inputs) 2025-12-04T09:45:44.2704662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:45:44.2705033Z outputs = self.model.decoder( 2025-12-04T09:45:44.2705399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:45:44.2706001Z layer_outputs = decoder_layer( 2025-12-04T09:45:44.2706330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:44.2706677Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:44.2707046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:44.2707406Z return func(*args, **kwargs) 2025-12-04T09:45:44.2707794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 798, in forward 2025-12-04T09:45:44.2708204Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:45:44.2708578Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:44.2708905Z return self.act(input) 2025-12-04T09:45:44.2709010Z 2025-12-04T09:45:44.2709090Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2709278Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2709470Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2709661Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2709842Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2710035Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2710264Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2710452Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2710672Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:44.2711039Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:44.2711349Z res = mod(**inputs) 2025-12-04T09:45:44.2711694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:45:44.2712072Z outputs = self.model.decoder( 2025-12-04T09:45:44.2712465Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:45:44.2712842Z layer_outputs = decoder_layer( 2025-12-04T09:45:44.2713171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:44.2713695Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:44.2714074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:44.2714496Z return func(*args, **kwargs) 2025-12-04T09:45:44.2714912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 766, in forward 2025-12-04T09:45:44.2715329Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:45:44.2715720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:44.2716081Z return func(*args, **kwargs) 2025-12-04T09:45:44.2716451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 443, in forward 2025-12-04T09:45:44.2716857Z attn_output, attn_weights = attention_interface( 2025-12-04T09:45:44.2717287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:45:44.2717761Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:45:44.2717953Z 2025-12-04T09:45:44.2718032Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2718234Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2718453Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:44.2718804Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:44.2719124Z res = mod(**inputs) 2025-12-04T09:45:44.2719478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:45:44.2719866Z outputs = self.model.decoder( 2025-12-04T09:45:44.2720246Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:45:44.2720634Z layer_outputs = decoder_layer( 2025-12-04T09:45:44.2720973Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:44.2721330Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:44.2721704Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:44.2722073Z return func(*args, **kwargs) 2025-12-04T09:45:44.2722435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 798, in forward 2025-12-04T09:45:44.2722861Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:45:44.2723243Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:44.2723570Z return self.act(input) 2025-12-04T09:45:44.2723687Z 2025-12-04T09:45:44.2723762Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2732688Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2734511Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2739837Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2745397Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2745966Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2746181Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2746417Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2746648Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:44.2747020Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:44.2747393Z res = mod(**inputs) 2025-12-04T09:45:44.2747786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:45:44.2748182Z outputs = self.model.decoder( 2025-12-04T09:45:44.2748605Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:45:44.2748992Z layer_outputs = decoder_layer( 2025-12-04T09:45:44.2749340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:44.2749689Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:44.2750065Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:44.2750432Z return func(*args, **kwargs) 2025-12-04T09:45:44.2750803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 766, in forward 2025-12-04T09:45:44.2751203Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:45:44.2751591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:44.2751965Z return func(*args, **kwargs) 2025-12-04T09:45:44.2752344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 443, in forward 2025-12-04T09:45:44.2752755Z attn_output, attn_weights = attention_interface( 2025-12-04T09:45:44.2753324Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:45:44.2753830Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:45:44.2754018Z 2025-12-04T09:45:44.2754101Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2754313Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2754562Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:44.2754909Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:44.2755216Z res = mod(**inputs) 2025-12-04T09:45:44.2755575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:45:44.2755962Z outputs = self.model.decoder( 2025-12-04T09:45:44.2756334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:45:44.2756713Z layer_outputs = decoder_layer( 2025-12-04T09:45:44.2757056Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:44.2757409Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:44.2757772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:44.2758131Z return func(*args, **kwargs) 2025-12-04T09:45:44.2758494Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 798, in forward 2025-12-04T09:45:44.2758915Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:45:44.2759315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:44.2759643Z return self.act(input) 2025-12-04T09:45:44.2759775Z 2025-12-04T09:45:44.2759861Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2760054Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2760251Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2760446Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2760628Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2760841Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2761030Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2761221Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2761436Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:44.2761800Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:44.2762138Z res = mod(**inputs) 2025-12-04T09:45:44.2762494Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:45:44.2762870Z outputs = self.model.decoder( 2025-12-04T09:45:44.2763242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:45:44.2763614Z layer_outputs = decoder_layer( 2025-12-04T09:45:44.2763951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:44.2764295Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:44.2764658Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:44.2765017Z return func(*args, **kwargs) 2025-12-04T09:45:44.2765378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 766, in forward 2025-12-04T09:45:44.2765780Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:45:44.2766163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:44.2766518Z return func(*args, **kwargs) 2025-12-04T09:45:44.2766872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 443, in forward 2025-12-04T09:45:44.2767270Z attn_output, attn_weights = attention_interface( 2025-12-04T09:45:44.2767698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:45:44.2768161Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:45:44.2768334Z 2025-12-04T09:45:44.2768408Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2768605Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2768825Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:44.2769158Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:44.2769465Z res = mod(**inputs) 2025-12-04T09:45:44.2769819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:45:44.2770198Z outputs = self.model.decoder( 2025-12-04T09:45:44.2770561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:45:44.2770940Z layer_outputs = decoder_layer( 2025-12-04T09:45:44.2771276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:44.2771613Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:44.2771995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:44.2772354Z return func(*args, **kwargs) 2025-12-04T09:45:44.2772739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 798, in forward 2025-12-04T09:45:44.2773155Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:45:44.2773533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:44.2773867Z return self.act(input) 2025-12-04T09:45:44.2773993Z 2025-12-04T09:45:44.2774073Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2774266Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2774464Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2774656Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2774841Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2775053Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2775245Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2775429Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2775647Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:44.2775989Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:44.2776288Z res = mod(**inputs) 2025-12-04T09:45:44.2776643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:45:44.2777020Z outputs = self.model.decoder( 2025-12-04T09:45:44.2777388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:45:44.2777759Z layer_outputs = decoder_layer( 2025-12-04T09:45:44.2778095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:44.2778449Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:44.2778812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:44.2779163Z return func(*args, **kwargs) 2025-12-04T09:45:44.2779526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 766, in forward 2025-12-04T09:45:44.2779928Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:45:44.2780303Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:44.2780663Z return func(*args, **kwargs) 2025-12-04T09:45:44.2781021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 443, in forward 2025-12-04T09:45:44.2781422Z attn_output, attn_weights = attention_interface( 2025-12-04T09:45:44.2781843Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:45:44.2782303Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:45:44.2782474Z 2025-12-04T09:45:44.2782557Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2782756Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2782964Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:44.2783301Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:44.2783610Z res = mod(**inputs) 2025-12-04T09:45:44.2783953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:45:44.2784335Z outputs = self.model.decoder( 2025-12-04T09:45:44.2784701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:45:44.2785080Z layer_outputs = decoder_layer( 2025-12-04T09:45:44.2785453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:44.2785818Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:44.2786174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:44.2786515Z return func(*args, **kwargs) 2025-12-04T09:45:44.2786868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 798, in forward 2025-12-04T09:45:44.2787312Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:45:44.2787691Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:44.2788015Z return self.act(input) 2025-12-04T09:45:44.2788149Z 2025-12-04T09:45:44.2788223Z cudagraph partition due to non gpu ops 2025-12-04T09:45:44.2788451Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:44.2788785Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:44.2789090Z res = mod(**inputs) 2025-12-04T09:45:44.2789441Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1696, in forward 2025-12-04T09:45:44.2789822Z logits = self.lm_head(outputs[0]) 2025-12-04T09:45:44.2789944Z 2025-12-04T09:45:44.2790054Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:44.2790395Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:44.2790701Z res = mod(**inputs) 2025-12-04T09:45:44.2791043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1702, in forward 2025-12-04T09:45:44.2791494Z loss = loss_fct(logits.view(-1, self.config.vocab_size), labels.view(-1)) 2025-12-04T09:45:44.2791686Z 2025-12-04T09:45:53.0440769Z Compilation time (from dynamo_timed): 12.940837839 2025-12-04T09:45:53.0767222Z pass 2025-12-04T09:45:53.0769463Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:45:53.0770336Z TIMING: _recursive_pre_grad_passes:0.01742 _recursive_joint_graph_passes:0.23969 _recursive_post_grad_passes:0.03575 async_compile.wait:0.73012 code_gen:8.35225 inductor_compile:9.76419 backend_compile:11.56996 gc:0.00088 entire_frame_compile:12.94084 total_wall_time:12.94084 2025-12-04T09:45:53.0774973Z STATS: call_* op count: 180 | FakeTensorMode.__torch_dispatch__:10061 | FakeTensor.__torch_dispatch__:1881 | ProxyTorchDispatchMode.__torch_dispatch__:1534 2025-12-04T09:45:53.0779570Z Dynamo produced 1 graphs covering 180 ops with 0 graph breaks (0 unique) 2025-12-04T09:45:55.2592887Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:45:55.2593771Z import pynvml # type: ignore[import] 2025-12-04T09:45:58.3654048Z 2025-12-04T09:46:01.2205661Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:46:01.2206300Z loading model: 0it [00:02, ?it/s] 2025-12-04T09:46:01.2224318Z cpu eval PegasusForCausalLM 2025-12-04T09:46:01.5841021Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:46:01.7159198Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:46:01.8322771Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:46:09.9724540Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9726268Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9726670Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9731962Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9736102Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9738126Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9738725Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9742945Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9745151Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9745464Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9745668Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9746065Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9746252Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9746451Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9746649Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9746833Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9747083Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9747280Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9747471Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9747714Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9748102Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9748454Z res = mod(**inputs) 2025-12-04T09:46:09.9748862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9749277Z outputs = self.model.decoder( 2025-12-04T09:46:09.9749684Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9750073Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9750427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9750796Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9751183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9751542Z return func(*args, **kwargs) 2025-12-04T09:46:09.9751919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:46:09.9752339Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:46:09.9752746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9753123Z return func(*args, **kwargs) 2025-12-04T09:46:09.9753687Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:46:09.9754126Z attn_output, attn_weights = attention_interface( 2025-12-04T09:46:09.9754580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:46:09.9755063Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:09.9755247Z 2025-12-04T09:46:09.9755325Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9755523Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9755741Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9756087Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9756403Z res = mod(**inputs) 2025-12-04T09:46:09.9756752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9757139Z outputs = self.model.decoder( 2025-12-04T09:46:09.9757516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9757898Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9758267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9758625Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9759019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9759383Z return func(*args, **kwargs) 2025-12-04T09:46:09.9759750Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:46:09.9760224Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:46:09.9760613Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:09.9760935Z return self.act(input) 2025-12-04T09:46:09.9761067Z 2025-12-04T09:46:09.9761148Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9761384Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9761577Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9761759Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9761947Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9762135Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9762315Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9762500Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9762758Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9763134Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9763444Z res = mod(**inputs) 2025-12-04T09:46:09.9763807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9764192Z outputs = self.model.decoder( 2025-12-04T09:46:09.9764567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9764947Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9765311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9765657Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9766025Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9766383Z return func(*args, **kwargs) 2025-12-04T09:46:09.9766752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:46:09.9767148Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:46:09.9767530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9767885Z return func(*args, **kwargs) 2025-12-04T09:46:09.9768253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:46:09.9768651Z attn_output, attn_weights = attention_interface( 2025-12-04T09:46:09.9769082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:46:09.9769539Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:09.9769712Z 2025-12-04T09:46:09.9769786Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9769982Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9770204Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9770544Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9770841Z res = mod(**inputs) 2025-12-04T09:46:09.9771217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9771682Z outputs = self.model.decoder( 2025-12-04T09:46:09.9772092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9772468Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9772801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9773151Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9773525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9773877Z return func(*args, **kwargs) 2025-12-04T09:46:09.9774241Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:46:09.9774675Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:46:09.9775036Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:09.9775363Z return self.act(input) 2025-12-04T09:46:09.9775467Z 2025-12-04T09:46:09.9775548Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9775735Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9775923Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9776110Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9776289Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9776478Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9776667Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9776854Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9777060Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9777394Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9777698Z res = mod(**inputs) 2025-12-04T09:46:09.9778046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9778428Z outputs = self.model.decoder( 2025-12-04T09:46:09.9778798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9779174Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9779497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9779844Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9780203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9780547Z return func(*args, **kwargs) 2025-12-04T09:46:09.9780927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:46:09.9781336Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:46:09.9781719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9782068Z return func(*args, **kwargs) 2025-12-04T09:46:09.9782434Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:46:09.9782831Z attn_output, attn_weights = attention_interface( 2025-12-04T09:46:09.9783255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:46:09.9783707Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:09.9783888Z 2025-12-04T09:46:09.9783961Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9784154Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9784383Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9784719Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9785038Z res = mod(**inputs) 2025-12-04T09:46:09.9785396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9785770Z outputs = self.model.decoder( 2025-12-04T09:46:09.9786143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9786537Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9786864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9787211Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9787586Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9787935Z return func(*args, **kwargs) 2025-12-04T09:46:09.9788290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:46:09.9788702Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:46:09.9789068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:09.9789390Z return self.act(input) 2025-12-04T09:46:09.9789493Z 2025-12-04T09:46:09.9789566Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9789788Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9789974Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9790166Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9790357Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9790543Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9790736Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9790924Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9791133Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9791473Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9791779Z res = mod(**inputs) 2025-12-04T09:46:09.9792131Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9792499Z outputs = self.model.decoder( 2025-12-04T09:46:09.9792869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9793317Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9793658Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9794024Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9794426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9794794Z return func(*args, **kwargs) 2025-12-04T09:46:09.9795163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:46:09.9795577Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:46:09.9795976Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9796337Z return func(*args, **kwargs) 2025-12-04T09:46:09.9796689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:46:09.9797088Z attn_output, attn_weights = attention_interface( 2025-12-04T09:46:09.9797538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:46:09.9797991Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:09.9798170Z 2025-12-04T09:46:09.9798261Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9798457Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9798677Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9799012Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9799331Z res = mod(**inputs) 2025-12-04T09:46:09.9799688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9800069Z outputs = self.model.decoder( 2025-12-04T09:46:09.9800443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9800839Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9801176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9801520Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9801891Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9802250Z return func(*args, **kwargs) 2025-12-04T09:46:09.9802616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:46:09.9803028Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:46:09.9803396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:09.9803721Z return self.act(input) 2025-12-04T09:46:09.9803826Z 2025-12-04T09:46:09.9803898Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9804098Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9804290Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9804483Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9804668Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9804857Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9805047Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9805231Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9805449Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9806064Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9806374Z res = mod(**inputs) 2025-12-04T09:46:09.9806738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9807124Z outputs = self.model.decoder( 2025-12-04T09:46:09.9807500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9807875Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9808211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9808563Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9808922Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9809285Z return func(*args, **kwargs) 2025-12-04T09:46:09.9809654Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:46:09.9810059Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:46:09.9810436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9810839Z return func(*args, **kwargs) 2025-12-04T09:46:09.9811208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:46:09.9811654Z attn_output, attn_weights = attention_interface( 2025-12-04T09:46:09.9812072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:46:09.9812531Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:09.9812733Z 2025-12-04T09:46:09.9812817Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9813004Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9813224Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9813561Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9813892Z res = mod(**inputs) 2025-12-04T09:46:09.9814243Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9814628Z outputs = self.model.decoder( 2025-12-04T09:46:09.9815005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9815378Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9815710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9816057Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9816423Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9816776Z return func(*args, **kwargs) 2025-12-04T09:46:09.9817142Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:46:09.9817566Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:46:09.9817938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:09.9818261Z return self.act(input) 2025-12-04T09:46:09.9818369Z 2025-12-04T09:46:09.9818442Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9818638Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9818825Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9819016Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9819211Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9819393Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9819585Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9819778Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9819988Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9820332Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9820643Z res = mod(**inputs) 2025-12-04T09:46:09.9821005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9821379Z outputs = self.model.decoder( 2025-12-04T09:46:09.9821754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9822137Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9822475Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9822813Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9823177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9823539Z return func(*args, **kwargs) 2025-12-04T09:46:09.9823911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:46:09.9824319Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:46:09.9824714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9825073Z return func(*args, **kwargs) 2025-12-04T09:46:09.9825424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:46:09.9825839Z attn_output, attn_weights = attention_interface( 2025-12-04T09:46:09.9826260Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:46:09.9826716Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:09.9826907Z 2025-12-04T09:46:09.9826980Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9827176Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9827393Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9827726Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9828036Z res = mod(**inputs) 2025-12-04T09:46:09.9828396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9828779Z outputs = self.model.decoder( 2025-12-04T09:46:09.9829147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9829526Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9829856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9830194Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9830558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9830915Z return func(*args, **kwargs) 2025-12-04T09:46:09.9831281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:46:09.9831695Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:46:09.9832070Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:09.9832404Z return self.act(input) 2025-12-04T09:46:09.9832508Z 2025-12-04T09:46:09.9832582Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9832779Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9832973Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9833169Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9833435Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9833642Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9833843Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9834031Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9834260Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9834615Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9834925Z res = mod(**inputs) 2025-12-04T09:46:09.9835295Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9835692Z outputs = self.model.decoder( 2025-12-04T09:46:09.9836085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9836466Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9836806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9837182Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9837568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9837938Z return func(*args, **kwargs) 2025-12-04T09:46:09.9838311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:46:09.9838724Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:46:09.9839127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9839496Z return func(*args, **kwargs) 2025-12-04T09:46:09.9839872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:46:09.9840308Z attn_output, attn_weights = attention_interface( 2025-12-04T09:46:09.9840736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:46:09.9841206Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:09.9841382Z 2025-12-04T09:46:09.9841464Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9841658Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9841885Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9842229Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9842541Z res = mod(**inputs) 2025-12-04T09:46:09.9842896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9843285Z outputs = self.model.decoder( 2025-12-04T09:46:09.9843672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9844056Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9844402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9844756Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9845131Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9845491Z return func(*args, **kwargs) 2025-12-04T09:46:09.9845878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:46:09.9846297Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:46:09.9846671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:09.9846995Z return self.act(input) 2025-12-04T09:46:09.9847103Z 2025-12-04T09:46:09.9847177Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9847376Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9847561Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9847755Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9847947Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9848132Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9848324Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9848515Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9848735Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9849065Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9849368Z res = mod(**inputs) 2025-12-04T09:46:09.9849721Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9850095Z outputs = self.model.decoder( 2025-12-04T09:46:09.9850493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9850871Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9851228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9851566Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9851931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9852310Z return func(*args, **kwargs) 2025-12-04T09:46:09.9852676Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:46:09.9853084Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:46:09.9853484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9853848Z return func(*args, **kwargs) 2025-12-04T09:46:09.9854212Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:46:09.9854624Z attn_output, attn_weights = attention_interface( 2025-12-04T09:46:09.9855055Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:46:09.9855519Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:09.9855694Z 2025-12-04T09:46:09.9855770Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9855975Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9856201Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9856536Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9856850Z res = mod(**inputs) 2025-12-04T09:46:09.9857212Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9857599Z outputs = self.model.decoder( 2025-12-04T09:46:09.9857970Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9858352Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9858690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9859034Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9859404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9859764Z return func(*args, **kwargs) 2025-12-04T09:46:09.9860135Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:46:09.9860554Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:46:09.9860935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:09.9861268Z return self.act(input) 2025-12-04T09:46:09.9861372Z 2025-12-04T09:46:09.9861454Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9861647Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9861846Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9862046Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9862238Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9862438Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9862637Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9862826Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9863054Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9863428Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9863727Z res = mod(**inputs) 2025-12-04T09:46:09.9864100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9864488Z outputs = self.model.decoder( 2025-12-04T09:46:09.9864868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9865243Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9865601Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9865948Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9866314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9866681Z return func(*args, **kwargs) 2025-12-04T09:46:09.9867051Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:46:09.9867454Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:46:09.9867829Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9868186Z return func(*args, **kwargs) 2025-12-04T09:46:09.9868550Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:46:09.9868953Z attn_output, attn_weights = attention_interface( 2025-12-04T09:46:09.9869370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:46:09.9869834Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:09.9870008Z 2025-12-04T09:46:09.9870089Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9870288Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9870502Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9870846Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9871152Z res = mod(**inputs) 2025-12-04T09:46:09.9871502Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9871886Z outputs = self.model.decoder( 2025-12-04T09:46:09.9872263Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9872640Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9872964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9873382Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9873752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9874107Z return func(*args, **kwargs) 2025-12-04T09:46:09.9874473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:46:09.9874895Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:46:09.9875267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:09.9875589Z return self.act(input) 2025-12-04T09:46:09.9875701Z 2025-12-04T09:46:09.9875774Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9875972Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9876160Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9876355Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9876548Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9876748Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9876940Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9877132Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9877365Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9877697Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9878000Z res = mod(**inputs) 2025-12-04T09:46:09.9878356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9878751Z outputs = self.model.decoder( 2025-12-04T09:46:09.9879130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9879518Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9879869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9880213Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9880576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9880937Z return func(*args, **kwargs) 2025-12-04T09:46:09.9881298Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:46:09.9881705Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:46:09.9882088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9882445Z return func(*args, **kwargs) 2025-12-04T09:46:09.9882800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:46:09.9883206Z attn_output, attn_weights = attention_interface( 2025-12-04T09:46:09.9883635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:46:09.9884096Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:09.9884268Z 2025-12-04T09:46:09.9884340Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9884538Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9884755Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9885090Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9885399Z res = mod(**inputs) 2025-12-04T09:46:09.9885758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9886139Z outputs = self.model.decoder( 2025-12-04T09:46:09.9886510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9886892Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9887226Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9887565Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9887927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9888287Z return func(*args, **kwargs) 2025-12-04T09:46:09.9888653Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:46:09.9889064Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:46:09.9889435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:09.9889763Z return self.act(input) 2025-12-04T09:46:09.9889882Z 2025-12-04T09:46:09.9889962Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9890153Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9890360Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9890552Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9890737Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9890925Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9891118Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9891298Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9891530Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9891868Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9892170Z res = mod(**inputs) 2025-12-04T09:46:09.9892520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9892922Z outputs = self.model.decoder( 2025-12-04T09:46:09.9893297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9893669Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9894005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9894353Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9894714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9895063Z return func(*args, **kwargs) 2025-12-04T09:46:09.9895424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:46:09.9895828Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:46:09.9896207Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9896561Z return func(*args, **kwargs) 2025-12-04T09:46:09.9896926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:46:09.9897327Z attn_output, attn_weights = attention_interface( 2025-12-04T09:46:09.9897743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:46:09.9898203Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:09.9898382Z 2025-12-04T09:46:09.9898454Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9898651Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9898862Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9899201Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9899506Z res = mod(**inputs) 2025-12-04T09:46:09.9899854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9900240Z outputs = self.model.decoder( 2025-12-04T09:46:09.9900616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9900997Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9901322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9901667Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9902026Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9902376Z return func(*args, **kwargs) 2025-12-04T09:46:09.9902758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:46:09.9903180Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:46:09.9903563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:09.9903884Z return self.act(input) 2025-12-04T09:46:09.9903994Z 2025-12-04T09:46:09.9904066Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9904261Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9904447Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9904654Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9904843Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9905031Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9905213Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9905405Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9905820Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9906168Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9906479Z res = mod(**inputs) 2025-12-04T09:46:09.9906839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9907214Z outputs = self.model.decoder( 2025-12-04T09:46:09.9907590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9907974Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9908311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9908653Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9909017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9909377Z return func(*args, **kwargs) 2025-12-04T09:46:09.9909744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:46:09.9910140Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:46:09.9910520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9910874Z return func(*args, **kwargs) 2025-12-04T09:46:09.9911226Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:46:09.9911635Z attn_output, attn_weights = attention_interface( 2025-12-04T09:46:09.9912053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:46:09.9912505Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:09.9912676Z 2025-12-04T09:46:09.9912750Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9912948Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9913166Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9913555Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9913874Z res = mod(**inputs) 2025-12-04T09:46:09.9914245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:46:09.9914629Z outputs = self.model.decoder( 2025-12-04T09:46:09.9914998Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:46:09.9915375Z layer_outputs = decoder_layer( 2025-12-04T09:46:09.9915714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:09.9916100Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:09.9916454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:09.9916831Z return func(*args, **kwargs) 2025-12-04T09:46:09.9917202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:46:09.9917615Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:46:09.9917991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:09.9918347Z return self.act(input) 2025-12-04T09:46:09.9918451Z 2025-12-04T09:46:09.9918531Z cudagraph partition due to non gpu ops 2025-12-04T09:46:09.9918750Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9919141Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9919440Z res = mod(**inputs) 2025-12-04T09:46:09.9919777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1649, in forward 2025-12-04T09:46:09.9920152Z logits = self.lm_head(outputs[0]) 2025-12-04T09:46:09.9920279Z 2025-12-04T09:46:09.9920375Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:09.9920701Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:09.9920990Z res = mod(**inputs) 2025-12-04T09:46:09.9921332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1655, in forward 2025-12-04T09:46:09.9921771Z loss = loss_fct(logits.view(-1, self.config.vocab_size), labels.view(-1)) 2025-12-04T09:46:09.9921955Z 2025-12-04T09:46:19.2692293Z Compilation time (from dynamo_timed): 16.888565185 2025-12-04T09:46:19.2712921Z pass 2025-12-04T09:46:19.2715307Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:46:19.2716261Z TIMING: _recursive_pre_grad_passes:0.02918 _recursive_joint_graph_passes:0.63766 _recursive_post_grad_passes:0.05228 async_compile.wait:0.7631 code_gen:9.43524 inductor_compile:11.03172 backend_compile:14.6945 gc:0.00031 entire_frame_compile:16.88857 total_wall_time:16.88857 2025-12-04T09:46:19.2720652Z STATS: call_* op count: 333 | FakeTensorMode.__torch_dispatch__:18822 | FakeTensor.__torch_dispatch__:3465 | ProxyTorchDispatchMode.__torch_dispatch__:2851 2025-12-04T09:46:19.2725122Z Dynamo produced 1 graphs covering 333 ops with 0 graph breaks (0 unique) 2025-12-04T09:46:21.5776468Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:46:21.5781110Z import pynvml # type: ignore[import] 2025-12-04T09:46:24.6185264Z 2025-12-04T09:46:24.6294149Z loading model: 0it [00:00, ?it/s]If you want to use `RobertaLMHeadModel` as a standalone, add `is_decoder=True.` 2025-12-04T09:46:24.6296043Z WARNING:transformers.models.roberta.modeling_roberta:If you want to use `RobertaLMHeadModel` as a standalone, add `is_decoder=True.` 2025-12-04T09:46:25.7098473Z We strongly recommend passing in an `attention_mask` since your input_ids may be padded. See https://huggingface.co/docs/transformers/troubleshooting#incorrect-output-when-padding-tokens-arent-masked. 2025-12-04T09:46:25.7099617Z You may ignore this warning if your `pad_token_id` (0) is identical to the `bos_token_id` (0), `eos_token_id` (2), or the `sep_token_id` (None), and your input is not padded. 2025-12-04T09:46:25.7100802Z WARNING:transformers.modeling_utils:We strongly recommend passing in an `attention_mask` since your input_ids may be padded. See https://huggingface.co/docs/transformers/troubleshooting#incorrect-output-when-padding-tokens-arent-masked. 2025-12-04T09:46:25.7101775Z You may ignore this warning if your `pad_token_id` (0) is identical to the `bos_token_id` (0), `eos_token_id` (2), or the `sep_token_id` (None), and your input is not padded. 2025-12-04T09:46:25.8583658Z 2025-12-04T09:46:25.8584458Z loading model: 0it [00:01, ?it/s] 2025-12-04T09:46:25.8593296Z cpu eval RobertaForCausalLM 2025-12-04T09:46:26.3389728Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:46:26.5763369Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:46:26.8471888Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:46:35.2443834Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2448840Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2450426Z res = mod(**inputs) 2025-12-04T09:46:35.2451088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2454169Z outputs = self.roberta( 2025-12-04T09:46:35.2454740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 798, in forward 2025-12-04T09:46:35.2459747Z embedding_output = self.embeddings( 2025-12-04T09:46:35.2462174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 87, in forward 2025-12-04T09:46:35.2462848Z position_ids = create_position_ids_from_input_ids(input_ids, self.padding_idx, past_key_values_length) 2025-12-04T09:46:35.2468394Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 1548, in create_position_ids_from_input_ids 2025-12-04T09:46:35.2469088Z mask = input_ids.ne(padding_idx).int() 2025-12-04T09:46:35.2469343Z 2025-12-04T09:46:35.2469550Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2469762Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2470038Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2470244Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2470453Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2470654Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2470856Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2471056Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2471252Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2471441Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2471636Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2471833Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2472060Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2472428Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2472762Z res = mod(**inputs) 2025-12-04T09:46:35.2473151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2473702Z outputs = self.roberta( 2025-12-04T09:46:35.2474116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 798, in forward 2025-12-04T09:46:35.2474540Z embedding_output = self.embeddings( 2025-12-04T09:46:35.2474959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 87, in forward 2025-12-04T09:46:35.2475485Z position_ids = create_position_ids_from_input_ids(input_ids, self.padding_idx, past_key_values_length) 2025-12-04T09:46:35.2476082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 1549, in create_position_ids_from_input_ids 2025-12-04T09:46:35.2476893Z incremental_indices = (torch.cumsum(mask, dim=1).type_as(mask) + past_key_values_length) * mask 2025-12-04T09:46:35.2477144Z 2025-12-04T09:46:35.2477304Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2477676Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2477996Z res = mod(**inputs) 2025-12-04T09:46:35.2478352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2478791Z outputs = self.roberta( 2025-12-04T09:46:35.2479173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 798, in forward 2025-12-04T09:46:35.2479564Z embedding_output = self.embeddings( 2025-12-04T09:46:35.2480041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 87, in forward 2025-12-04T09:46:35.2480552Z position_ids = create_position_ids_from_input_ids(input_ids, self.padding_idx, past_key_values_length) 2025-12-04T09:46:35.2481133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 1549, in create_position_ids_from_input_ids 2025-12-04T09:46:35.2481691Z incremental_indices = (torch.cumsum(mask, dim=1).type_as(mask) + past_key_values_length) * mask 2025-12-04T09:46:35.2481923Z 2025-12-04T09:46:35.2482002Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2482211Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2482537Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2482732Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2482921Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2483112Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2483310Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2483530Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2483886Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2484205Z res = mod(**inputs) 2025-12-04T09:46:35.2484569Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2484956Z outputs = self.roberta( 2025-12-04T09:46:35.2485329Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2485717Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2486095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2486483Z layer_outputs = layer_module( 2025-12-04T09:46:35.2486845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2487204Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2487573Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2487938Z return func(*args, **kwargs) 2025-12-04T09:46:35.2488307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:46:35.2488689Z self_attention_outputs = self.attention( 2025-12-04T09:46:35.2489066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2489425Z return func(*args, **kwargs) 2025-12-04T09:46:35.2489788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:46:35.2490158Z self_outputs = self.self( 2025-12-04T09:46:35.2490530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2490891Z return func(*args, **kwargs) 2025-12-04T09:46:35.2491270Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:46:35.2491708Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:35.2491890Z 2025-12-04T09:46:35.2491966Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2492164Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2492397Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2492738Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2493043Z res = mod(**inputs) 2025-12-04T09:46:35.2493392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2493788Z outputs = self.roberta( 2025-12-04T09:46:35.2494151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2494532Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2494906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2495297Z layer_outputs = layer_module( 2025-12-04T09:46:35.2495645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2496007Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2496377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2496745Z return func(*args, **kwargs) 2025-12-04T09:46:35.2497118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:46:35.2497522Z layer_output = apply_chunking_to_forward( 2025-12-04T09:46:35.2497929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:46:35.2498306Z return forward_fn(*input_tensors) 2025-12-04T09:46:35.2498710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:46:35.2499159Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:35.2499575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:46:35.2499988Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:35.2500354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:35.2500681Z return self.act(input) 2025-12-04T09:46:35.2500818Z 2025-12-04T09:46:35.2500893Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2501094Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2501293Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2501480Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2501673Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2501863Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2502044Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2502238Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2502459Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2502803Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2503102Z res = mod(**inputs) 2025-12-04T09:46:35.2503461Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2503850Z outputs = self.roberta( 2025-12-04T09:46:35.2504219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2504596Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2504963Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2505337Z layer_outputs = layer_module( 2025-12-04T09:46:35.2506020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2506384Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2506752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2507161Z return func(*args, **kwargs) 2025-12-04T09:46:35.2507518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:46:35.2507912Z self_attention_outputs = self.attention( 2025-12-04T09:46:35.2508290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2508648Z return func(*args, **kwargs) 2025-12-04T09:46:35.2509013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:46:35.2509403Z self_outputs = self.self( 2025-12-04T09:46:35.2509762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2510117Z return func(*args, **kwargs) 2025-12-04T09:46:35.2510486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:46:35.2510930Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:35.2511111Z 2025-12-04T09:46:35.2511194Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2511389Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2511617Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2511964Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2512278Z res = mod(**inputs) 2025-12-04T09:46:35.2512640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2513026Z outputs = self.roberta( 2025-12-04T09:46:35.2513448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2513847Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2514252Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2514643Z layer_outputs = layer_module( 2025-12-04T09:46:35.2514986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2515345Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2515725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2516105Z return func(*args, **kwargs) 2025-12-04T09:46:35.2516469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:46:35.2516865Z layer_output = apply_chunking_to_forward( 2025-12-04T09:46:35.2517266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:46:35.2517653Z return forward_fn(*input_tensors) 2025-12-04T09:46:35.2518099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:46:35.2518613Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:35.2519040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:46:35.2519454Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:35.2519826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:35.2520191Z return self.act(input) 2025-12-04T09:46:35.2520299Z 2025-12-04T09:46:35.2520381Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2520577Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2520777Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2520998Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2521190Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2521386Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2521581Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2521769Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2521996Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2522349Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2522661Z res = mod(**inputs) 2025-12-04T09:46:35.2523018Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2523404Z outputs = self.roberta( 2025-12-04T09:46:35.2523770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2524158Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2524529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2524901Z layer_outputs = layer_module( 2025-12-04T09:46:35.2525240Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2525581Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2525944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2526304Z return func(*args, **kwargs) 2025-12-04T09:46:35.2526663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:46:35.2527052Z self_attention_outputs = self.attention( 2025-12-04T09:46:35.2527418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2527774Z return func(*args, **kwargs) 2025-12-04T09:46:35.2528131Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:46:35.2528511Z self_outputs = self.self( 2025-12-04T09:46:35.2528858Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2529209Z return func(*args, **kwargs) 2025-12-04T09:46:35.2529574Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:46:35.2530005Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:35.2530177Z 2025-12-04T09:46:35.2530259Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2530450Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2530677Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2531036Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2531348Z res = mod(**inputs) 2025-12-04T09:46:35.2531712Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2532085Z outputs = self.roberta( 2025-12-04T09:46:35.2532444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2532812Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2533196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2533571Z layer_outputs = layer_module( 2025-12-04T09:46:35.2533905Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2534259Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2534624Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2534986Z return func(*args, **kwargs) 2025-12-04T09:46:35.2535353Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:46:35.2535740Z layer_output = apply_chunking_to_forward( 2025-12-04T09:46:35.2536123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:46:35.2536503Z return forward_fn(*input_tensors) 2025-12-04T09:46:35.2536899Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:46:35.2537353Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:35.2537774Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:46:35.2538185Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:35.2538546Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:35.2538873Z return self.act(input) 2025-12-04T09:46:35.2538978Z 2025-12-04T09:46:35.2539060Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2539250Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2539443Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2539638Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2539820Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2540011Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2540203Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2540393Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2540607Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2540948Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2541253Z res = mod(**inputs) 2025-12-04T09:46:35.2541606Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2541980Z outputs = self.roberta( 2025-12-04T09:46:35.2542340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2542718Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2543082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2543453Z layer_outputs = layer_module( 2025-12-04T09:46:35.2543787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2544125Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2544502Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2544873Z return func(*args, **kwargs) 2025-12-04T09:46:35.2545237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:46:35.2545613Z self_attention_outputs = self.attention( 2025-12-04T09:46:35.2545983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2546356Z return func(*args, **kwargs) 2025-12-04T09:46:35.2546720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:46:35.2547092Z self_outputs = self.self( 2025-12-04T09:46:35.2547456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2547813Z return func(*args, **kwargs) 2025-12-04T09:46:35.2548168Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:46:35.2548603Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:35.2548783Z 2025-12-04T09:46:35.2548856Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2549052Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2549265Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2549610Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2549921Z res = mod(**inputs) 2025-12-04T09:46:35.2550262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2550643Z outputs = self.roberta( 2025-12-04T09:46:35.2551006Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2551381Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2551752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2552132Z layer_outputs = layer_module( 2025-12-04T09:46:35.2552469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2552810Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2553180Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2553617Z return func(*args, **kwargs) 2025-12-04T09:46:35.2553996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:46:35.2554398Z layer_output = apply_chunking_to_forward( 2025-12-04T09:46:35.2554796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:46:35.2555165Z return forward_fn(*input_tensors) 2025-12-04T09:46:35.2555567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:46:35.2556017Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:35.2556450Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:46:35.2556877Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:35.2557245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:35.2557585Z return self.act(input) 2025-12-04T09:46:35.2557704Z 2025-12-04T09:46:35.2557797Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2558002Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2558198Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2558414Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2558618Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2558810Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2559013Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2559213Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2559453Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2559817Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2560142Z res = mod(**inputs) 2025-12-04T09:46:35.2560515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2560918Z outputs = self.roberta( 2025-12-04T09:46:35.2561299Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2561702Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2562098Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2562494Z layer_outputs = layer_module( 2025-12-04T09:46:35.2562844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2563214Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2563638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2564036Z return func(*args, **kwargs) 2025-12-04T09:46:35.2564428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:46:35.2564817Z self_attention_outputs = self.attention( 2025-12-04T09:46:35.2565178Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2565535Z return func(*args, **kwargs) 2025-12-04T09:46:35.2565899Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:46:35.2566266Z self_outputs = self.self( 2025-12-04T09:46:35.2566617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2566970Z return func(*args, **kwargs) 2025-12-04T09:46:35.2567330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:46:35.2567754Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:35.2567933Z 2025-12-04T09:46:35.2568006Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2568208Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2568422Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2568767Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2569073Z res = mod(**inputs) 2025-12-04T09:46:35.2569426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2569794Z outputs = self.roberta( 2025-12-04T09:46:35.2570153Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2570526Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2570887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2571284Z layer_outputs = layer_module( 2025-12-04T09:46:35.2571610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2571965Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2572319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2572673Z return func(*args, **kwargs) 2025-12-04T09:46:35.2573033Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:46:35.2573443Z layer_output = apply_chunking_to_forward( 2025-12-04T09:46:35.2573818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:46:35.2574207Z return forward_fn(*input_tensors) 2025-12-04T09:46:35.2574612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:46:35.2575060Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:35.2575479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:46:35.2575895Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:35.2576259Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:35.2576581Z return self.act(input) 2025-12-04T09:46:35.2576694Z 2025-12-04T09:46:35.2576768Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2576967Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2577155Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2577349Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2577547Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2577740Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2577926Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2578117Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2578337Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2578675Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2578985Z res = mod(**inputs) 2025-12-04T09:46:35.2579341Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2579716Z outputs = self.roberta( 2025-12-04T09:46:35.2580069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2580443Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2580816Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2581180Z layer_outputs = layer_module( 2025-12-04T09:46:35.2581517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2581867Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2582232Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2582585Z return func(*args, **kwargs) 2025-12-04T09:46:35.2582951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:46:35.2583340Z self_attention_outputs = self.attention( 2025-12-04T09:46:35.2583697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2584057Z return func(*args, **kwargs) 2025-12-04T09:46:35.2584432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:46:35.2584812Z self_outputs = self.self( 2025-12-04T09:46:35.2585167Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2585523Z return func(*args, **kwargs) 2025-12-04T09:46:35.2585883Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:46:35.2586331Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:35.2586503Z 2025-12-04T09:46:35.2586577Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2586771Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2586996Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2587345Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2587651Z res = mod(**inputs) 2025-12-04T09:46:35.2588001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2588371Z outputs = self.roberta( 2025-12-04T09:46:35.2588719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2589092Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2589459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2589823Z layer_outputs = layer_module( 2025-12-04T09:46:35.2590154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2590498Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2590863Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2591211Z return func(*args, **kwargs) 2025-12-04T09:46:35.2591572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:46:35.2591957Z layer_output = apply_chunking_to_forward( 2025-12-04T09:46:35.2592330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:46:35.2592704Z return forward_fn(*input_tensors) 2025-12-04T09:46:35.2593108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:46:35.2593635Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:35.2594067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:46:35.2594497Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:35.2594880Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:35.2595213Z return self.act(input) 2025-12-04T09:46:35.2595320Z 2025-12-04T09:46:35.2595395Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2595599Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2595804Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2595996Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2596198Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2596400Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2596589Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2596789Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2597017Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2597394Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2597704Z res = mod(**inputs) 2025-12-04T09:46:35.2598082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2598464Z outputs = self.roberta( 2025-12-04T09:46:35.2598826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2599212Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2599613Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2599999Z layer_outputs = layer_module( 2025-12-04T09:46:35.2600336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2600710Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2601085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2601448Z return func(*args, **kwargs) 2025-12-04T09:46:35.2601823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:46:35.2602222Z self_attention_outputs = self.attention( 2025-12-04T09:46:35.2602596Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2602958Z return func(*args, **kwargs) 2025-12-04T09:46:35.2603330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:46:35.2603714Z self_outputs = self.self( 2025-12-04T09:46:35.2604066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2604423Z return func(*args, **kwargs) 2025-12-04T09:46:35.2604793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:46:35.2605237Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:35.2605414Z 2025-12-04T09:46:35.2605489Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2605852Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2606091Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2606451Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2606764Z res = mod(**inputs) 2025-12-04T09:46:35.2607127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2607514Z outputs = self.roberta( 2025-12-04T09:46:35.2607877Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2608269Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2608642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2609019Z layer_outputs = layer_module( 2025-12-04T09:46:35.2609347Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2609699Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2610071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2610420Z return func(*args, **kwargs) 2025-12-04T09:46:35.2610787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:46:35.2611179Z layer_output = apply_chunking_to_forward( 2025-12-04T09:46:35.2611604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:46:35.2611974Z return forward_fn(*input_tensors) 2025-12-04T09:46:35.2612401Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:46:35.2612852Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:35.2613273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:46:35.2613704Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:35.2614077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:35.2614433Z return self.act(input) 2025-12-04T09:46:35.2614539Z 2025-12-04T09:46:35.2614612Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2614811Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2615003Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2615192Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2615378Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2615570Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2615762Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2615947Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2616170Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2616515Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2616816Z res = mod(**inputs) 2025-12-04T09:46:35.2617171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2617549Z outputs = self.roberta( 2025-12-04T09:46:35.2617908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2618272Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2618642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2619014Z layer_outputs = layer_module( 2025-12-04T09:46:35.2619338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2619682Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2620045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2620404Z return func(*args, **kwargs) 2025-12-04T09:46:35.2620759Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:46:35.2621148Z self_attention_outputs = self.attention( 2025-12-04T09:46:35.2621514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2621873Z return func(*args, **kwargs) 2025-12-04T09:46:35.2622230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:46:35.2622601Z self_outputs = self.self( 2025-12-04T09:46:35.2622944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2623293Z return func(*args, **kwargs) 2025-12-04T09:46:35.2623652Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:46:35.2624081Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:35.2624254Z 2025-12-04T09:46:35.2624336Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2624544Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2624766Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2625135Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2625437Z res = mod(**inputs) 2025-12-04T09:46:35.2625793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2626168Z outputs = self.roberta( 2025-12-04T09:46:35.2626550Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2626920Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2627291Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2627681Z layer_outputs = layer_module( 2025-12-04T09:46:35.2628006Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2628351Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2628716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2629072Z return func(*args, **kwargs) 2025-12-04T09:46:35.2629430Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:46:35.2629821Z layer_output = apply_chunking_to_forward( 2025-12-04T09:46:35.2630213Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:46:35.2630589Z return forward_fn(*input_tensors) 2025-12-04T09:46:35.2630987Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:46:35.2631443Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:35.2631864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:46:35.2632265Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:35.2632632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:35.2632959Z return self.act(input) 2025-12-04T09:46:35.2633065Z 2025-12-04T09:46:35.2633146Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2633399Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2633600Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2633801Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2633989Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2634193Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2634394Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2634591Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2634810Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2635166Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2635488Z res = mod(**inputs) 2025-12-04T09:46:35.2635876Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2636261Z outputs = self.roberta( 2025-12-04T09:46:35.2636630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2637017Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2637398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2637805Z layer_outputs = layer_module( 2025-12-04T09:46:35.2638152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2638526Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2638902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2639269Z return func(*args, **kwargs) 2025-12-04T09:46:35.2639644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:46:35.2640052Z self_attention_outputs = self.attention( 2025-12-04T09:46:35.2640436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2640816Z return func(*args, **kwargs) 2025-12-04T09:46:35.2641196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:46:35.2641581Z self_outputs = self.self( 2025-12-04T09:46:35.2641939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2642306Z return func(*args, **kwargs) 2025-12-04T09:46:35.2642671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:46:35.2643116Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:35.2643295Z 2025-12-04T09:46:35.2643380Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2643584Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2643802Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2644147Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2644463Z res = mod(**inputs) 2025-12-04T09:46:35.2644824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2645209Z outputs = self.roberta( 2025-12-04T09:46:35.2645577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2645967Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2646341Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2646729Z layer_outputs = layer_module( 2025-12-04T09:46:35.2647070Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2647418Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2647788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2648158Z return func(*args, **kwargs) 2025-12-04T09:46:35.2648532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:46:35.2648914Z layer_output = apply_chunking_to_forward( 2025-12-04T09:46:35.2649298Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:46:35.2649672Z return forward_fn(*input_tensors) 2025-12-04T09:46:35.2650066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:46:35.2650517Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:35.2650934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:46:35.2651341Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:35.2651711Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:35.2652038Z return self.act(input) 2025-12-04T09:46:35.2652162Z 2025-12-04T09:46:35.2652236Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2652432Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2652616Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2652805Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2652994Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2653196Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2653388Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2653577Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2653792Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2654136Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2654460Z res = mod(**inputs) 2025-12-04T09:46:35.2654822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2655190Z outputs = self.roberta( 2025-12-04T09:46:35.2655554Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2655934Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2656300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2656680Z layer_outputs = layer_module( 2025-12-04T09:46:35.2657014Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2657365Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2657728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2658091Z return func(*args, **kwargs) 2025-12-04T09:46:35.2658461Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:46:35.2658840Z self_attention_outputs = self.attention( 2025-12-04T09:46:35.2659209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2659566Z return func(*args, **kwargs) 2025-12-04T09:46:35.2659933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:46:35.2660303Z self_outputs = self.self( 2025-12-04T09:46:35.2660654Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2661018Z return func(*args, **kwargs) 2025-12-04T09:46:35.2661385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:46:35.2661811Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:35.2661994Z 2025-12-04T09:46:35.2662070Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2662273Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2662493Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2662839Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2663149Z res = mod(**inputs) 2025-12-04T09:46:35.2663504Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2663871Z outputs = self.roberta( 2025-12-04T09:46:35.2664230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2664649Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2665011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2665434Z layer_outputs = layer_module( 2025-12-04T09:46:35.2665770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2666132Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2666492Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2666868Z return func(*args, **kwargs) 2025-12-04T09:46:35.2667229Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:46:35.2667626Z layer_output = apply_chunking_to_forward( 2025-12-04T09:46:35.2668011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:46:35.2668380Z return forward_fn(*input_tensors) 2025-12-04T09:46:35.2668782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:46:35.2669228Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:35.2669646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:46:35.2670057Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:35.2670421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:35.2670741Z return self.act(input) 2025-12-04T09:46:35.2670855Z 2025-12-04T09:46:35.2670930Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2671129Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2671317Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2671511Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2671703Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2671889Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2672079Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2672271Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2672494Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2672830Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2673143Z res = mod(**inputs) 2025-12-04T09:46:35.2673563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2673985Z outputs = self.roberta( 2025-12-04T09:46:35.2674360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2674757Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2675128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2675494Z layer_outputs = layer_module( 2025-12-04T09:46:35.2675829Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2676178Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2676538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2676892Z return func(*args, **kwargs) 2025-12-04T09:46:35.2677253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:46:35.2677637Z self_attention_outputs = self.attention( 2025-12-04T09:46:35.2678022Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2678383Z return func(*args, **kwargs) 2025-12-04T09:46:35.2678763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:46:35.2679137Z self_outputs = self.self( 2025-12-04T09:46:35.2679472Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2679839Z return func(*args, **kwargs) 2025-12-04T09:46:35.2680204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:46:35.2680624Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:35.2680804Z 2025-12-04T09:46:35.2680896Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2681089Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2681314Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2681648Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2681959Z res = mod(**inputs) 2025-12-04T09:46:35.2682309Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2682668Z outputs = self.roberta( 2025-12-04T09:46:35.2683025Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2683400Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2683766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2684134Z layer_outputs = layer_module( 2025-12-04T09:46:35.2684467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2684813Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2685168Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2685522Z return func(*args, **kwargs) 2025-12-04T09:46:35.2685880Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:46:35.2686266Z layer_output = apply_chunking_to_forward( 2025-12-04T09:46:35.2686642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:46:35.2687014Z return forward_fn(*input_tensors) 2025-12-04T09:46:35.2687418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:46:35.2687874Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:35.2688288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:46:35.2688700Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:35.2689072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:35.2689392Z return self.act(input) 2025-12-04T09:46:35.2689504Z 2025-12-04T09:46:35.2689580Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2689779Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2689973Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2690164Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2690356Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2690547Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2690733Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2690941Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2691162Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2691507Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2691816Z res = mod(**inputs) 2025-12-04T09:46:35.2692170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2692544Z outputs = self.roberta( 2025-12-04T09:46:35.2692913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2693288Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2693658Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2694044Z layer_outputs = layer_module( 2025-12-04T09:46:35.2694368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2694712Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2695074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2695426Z return func(*args, **kwargs) 2025-12-04T09:46:35.2695793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:46:35.2696175Z self_attention_outputs = self.attention( 2025-12-04T09:46:35.2696539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2696883Z return func(*args, **kwargs) 2025-12-04T09:46:35.2697247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:46:35.2697620Z self_outputs = self.self( 2025-12-04T09:46:35.2697956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2698308Z return func(*args, **kwargs) 2025-12-04T09:46:35.2698669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:46:35.2699093Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:46:35.2699268Z 2025-12-04T09:46:35.2699342Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2699539Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2699760Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2700092Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2700399Z res = mod(**inputs) 2025-12-04T09:46:35.2700751Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:46:35.2701122Z outputs = self.roberta( 2025-12-04T09:46:35.2701474Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:46:35.2701849Z encoder_outputs = self.encoder( 2025-12-04T09:46:35.2702218Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:46:35.2702587Z layer_outputs = layer_module( 2025-12-04T09:46:35.2702912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:46:35.2703264Z return super().__call__(*args, **kwargs) 2025-12-04T09:46:35.2703625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:46:35.2703973Z return func(*args, **kwargs) 2025-12-04T09:46:35.2704346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:46:35.2704732Z layer_output = apply_chunking_to_forward( 2025-12-04T09:46:35.2705127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:46:35.2705499Z return forward_fn(*input_tensors) 2025-12-04T09:46:35.2706063Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:46:35.2706574Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:35.2706997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:46:35.2707407Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:35.2707808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:46:35.2708136Z return self.act(input) 2025-12-04T09:46:35.2708239Z 2025-12-04T09:46:35.2708313Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2708512Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2708710Z cudagraph partition due to non gpu ops 2025-12-04T09:46:35.2708925Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:35.2709267Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:35.2709578Z res = mod(**inputs) 2025-12-04T09:46:35.2709932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 994, in forward 2025-12-04T09:46:35.2710299Z lm_loss = self.loss_function( 2025-12-04T09:46:35.2710661Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 67, in ForCausalLMLoss 2025-12-04T09:46:35.2711127Z loss = fixed_cross_entropy(logits, shift_labels, num_items_in_batch, ignore_index, **kwargs) 2025-12-04T09:46:35.2711589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 36, in fixed_cross_entropy 2025-12-04T09:46:35.2712063Z loss = nn.functional.cross_entropy(source, target, ignore_index=ignore_index, reduction=reduction) 2025-12-04T09:46:35.2712308Z 2025-12-04T09:46:44.6944033Z Compilation time (from dynamo_timed): 17.189366769 2025-12-04T09:46:44.7048865Z pass 2025-12-04T09:46:44.7050811Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:46:44.7051681Z TIMING: _recursive_pre_grad_passes:0.02765 _recursive_joint_graph_passes:0.3504 _recursive_post_grad_passes:0.05457 async_compile.wait:0.72153 code_gen:9.3657 inductor_compile:11.00477 backend_compile:14.59913 gc:0.00064 entire_frame_compile:17.18937 total_wall_time:17.18937 2025-12-04T09:46:44.7056401Z STATS: call_* op count: 303 | FakeTensorMode.__torch_dispatch__:19079 | FakeTensor.__torch_dispatch__:3569 | ProxyTorchDispatchMode.__torch_dispatch__:2812 2025-12-04T09:46:44.7058048Z Dynamo produced 1 graphs covering 303 ops with 0 graph breaks (0 unique) 2025-12-04T09:46:46.9724072Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:46:46.9724880Z import pynvml # type: ignore[import] 2025-12-04T09:46:50.0973250Z 2025-12-04T09:46:50.9014445Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:46:50.9015665Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:46:50.9027674Z cpu eval T5ForConditionalGeneration 2025-12-04T09:46:52.2934487Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:46:52.7160858Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:46:53.0872737Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:47:02.9341747Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9346141Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9351393Z res = mod(**inputs) 2025-12-04T09:47:02.9351851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9352542Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9352934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9353458Z layer_outputs = layer_module( 2025-12-04T09:47:02.9355320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9355876Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9356328Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9356793Z return func(*args, **kwargs) 2025-12-04T09:47:02.9361688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9362267Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9362727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9363652Z return func(*args, **kwargs) 2025-12-04T09:47:02.9364267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9364859Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9365718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9366421Z return func(*args, **kwargs) 2025-12-04T09:47:02.9366869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 549, in forward 2025-12-04T09:47:02.9367296Z position_bias = position_bias + causal_mask 2025-12-04T09:47:02.9367480Z 2025-12-04T09:47:02.9367607Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9368053Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9368428Z res = mod(**inputs) 2025-12-04T09:47:02.9368823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9369234Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9369642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9370042Z layer_outputs = layer_module( 2025-12-04T09:47:02.9370480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9370903Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9371288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9371696Z return func(*args, **kwargs) 2025-12-04T09:47:02.9372079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9372524Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9372968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9373367Z return func(*args, **kwargs) 2025-12-04T09:47:02.9374543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9374962Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9375421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9375805Z return func(*args, **kwargs) 2025-12-04T09:47:02.9376179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:02.9376593Z query_states = self.q(hidden_states) 2025-12-04T09:47:02.9376737Z 2025-12-04T09:47:02.9376844Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9377205Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9377527Z res = mod(**inputs) 2025-12-04T09:47:02.9377893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9378262Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9378633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9378993Z layer_outputs = layer_module( 2025-12-04T09:47:02.9379341Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9379703Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9380077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9380474Z return func(*args, **kwargs) 2025-12-04T09:47:02.9380850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9381212Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9381585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9381942Z return func(*args, **kwargs) 2025-12-04T09:47:02.9382294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9382664Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9383042Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9383405Z return func(*args, **kwargs) 2025-12-04T09:47:02.9383766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:02.9384134Z key_states = self.k(current_states) 2025-12-04T09:47:02.9384261Z 2025-12-04T09:47:02.9384362Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9384724Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9385045Z res = mod(**inputs) 2025-12-04T09:47:02.9385388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9385748Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9386108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9386473Z layer_outputs = layer_module( 2025-12-04T09:47:02.9386814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9387167Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9387537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9387908Z return func(*args, **kwargs) 2025-12-04T09:47:02.9388271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9388646Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9389064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9389438Z return func(*args, **kwargs) 2025-12-04T09:47:02.9389778Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9390152Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9390548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9390904Z return func(*args, **kwargs) 2025-12-04T09:47:02.9391254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:02.9391644Z value_states = self.v(current_states) 2025-12-04T09:47:02.9391779Z 2025-12-04T09:47:02.9391866Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9392071Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9392310Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9392679Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9393000Z res = mod(**inputs) 2025-12-04T09:47:02.9393513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9393905Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9394279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9394654Z layer_outputs = layer_module( 2025-12-04T09:47:02.9395008Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9395382Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9395762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9396143Z return func(*args, **kwargs) 2025-12-04T09:47:02.9396501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9396887Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9397260Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9397627Z return func(*args, **kwargs) 2025-12-04T09:47:02.9397972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9398346Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9398717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9399084Z return func(*args, **kwargs) 2025-12-04T09:47:02.9399432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:02.9399789Z attn_output = self.o(attn_output) 2025-12-04T09:47:02.9399923Z 2025-12-04T09:47:02.9400024Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9400373Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9400689Z res = mod(**inputs) 2025-12-04T09:47:02.9401019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9401386Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9401745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9402120Z layer_outputs = layer_module( 2025-12-04T09:47:02.9402453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9403242Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9403619Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9403977Z return func(*args, **kwargs) 2025-12-04T09:47:02.9404328Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9404739Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9405120Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9405518Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9406211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:02.9406591Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:02.9406724Z 2025-12-04T09:47:02.9406827Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9407177Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9407495Z res = mod(**inputs) 2025-12-04T09:47:02.9407835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9408197Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9408557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9408923Z layer_outputs = layer_module( 2025-12-04T09:47:02.9409259Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9409626Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9409998Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9410364Z return func(*args, **kwargs) 2025-12-04T09:47:02.9410706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9411095Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9411463Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9411860Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9412244Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:02.9412609Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:02.9412738Z 2025-12-04T09:47:02.9412846Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9413179Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9413488Z res = mod(**inputs) 2025-12-04T09:47:02.9413820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9414175Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9414520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9414877Z layer_outputs = layer_module( 2025-12-04T09:47:02.9415207Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9415547Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9415913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9416319Z return func(*args, **kwargs) 2025-12-04T09:47:02.9416664Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9417054Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9417421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9417806Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9418220Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:02.9418572Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:02.9418707Z 2025-12-04T09:47:02.9418806Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9419175Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9419474Z res = mod(**inputs) 2025-12-04T09:47:02.9419804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9420162Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9420506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9420849Z layer_outputs = layer_module( 2025-12-04T09:47:02.9421181Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9421530Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9421881Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9422239Z return func(*args, **kwargs) 2025-12-04T09:47:02.9422581Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9422943Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9423304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9423661Z return func(*args, **kwargs) 2025-12-04T09:47:02.9423999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9424360Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9424720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9425078Z return func(*args, **kwargs) 2025-12-04T09:47:02.9425417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:02.9425767Z query_states = self.q(hidden_states) 2025-12-04T09:47:02.9425905Z 2025-12-04T09:47:02.9426005Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9426343Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9426651Z res = mod(**inputs) 2025-12-04T09:47:02.9426974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9427330Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9427680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9428029Z layer_outputs = layer_module( 2025-12-04T09:47:02.9428360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9428710Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9429073Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9429440Z return func(*args, **kwargs) 2025-12-04T09:47:02.9429789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9430144Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9430493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9430843Z return func(*args, **kwargs) 2025-12-04T09:47:02.9431173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9431557Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9431917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9432286Z return func(*args, **kwargs) 2025-12-04T09:47:02.9432616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:02.9433014Z key_states = self.k(current_states) 2025-12-04T09:47:02.9433147Z 2025-12-04T09:47:02.9433333Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9433711Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9434053Z res = mod(**inputs) 2025-12-04T09:47:02.9434407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9434784Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9435160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9435549Z layer_outputs = layer_module( 2025-12-04T09:47:02.9435907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9436286Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9436684Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9437070Z return func(*args, **kwargs) 2025-12-04T09:47:02.9437446Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9437837Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9438235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9438618Z return func(*args, **kwargs) 2025-12-04T09:47:02.9438987Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9439380Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9439772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9440160Z return func(*args, **kwargs) 2025-12-04T09:47:02.9440525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:02.9440917Z value_states = self.v(current_states) 2025-12-04T09:47:02.9441055Z 2025-12-04T09:47:02.9441138Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9441358Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9441602Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9441967Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9442301Z res = mod(**inputs) 2025-12-04T09:47:02.9442661Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9443052Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9443457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9443832Z layer_outputs = layer_module( 2025-12-04T09:47:02.9444179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9444528Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9444885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9445260Z return func(*args, **kwargs) 2025-12-04T09:47:02.9445611Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9445965Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9446333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9446714Z return func(*args, **kwargs) 2025-12-04T09:47:02.9447052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9447410Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9447778Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9448134Z return func(*args, **kwargs) 2025-12-04T09:47:02.9448464Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:02.9448820Z attn_output = self.o(attn_output) 2025-12-04T09:47:02.9448951Z 2025-12-04T09:47:02.9449050Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9449393Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9449694Z res = mod(**inputs) 2025-12-04T09:47:02.9450027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9450385Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9450736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9451089Z layer_outputs = layer_module( 2025-12-04T09:47:02.9451420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9451769Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9452130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9452488Z return func(*args, **kwargs) 2025-12-04T09:47:02.9452829Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9453205Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9453568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9453970Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9454363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:02.9454728Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:02.9454861Z 2025-12-04T09:47:02.9454961Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9455300Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9455607Z res = mod(**inputs) 2025-12-04T09:47:02.9455929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9456289Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9456653Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9457011Z layer_outputs = layer_module( 2025-12-04T09:47:02.9457348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9457700Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9458065Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9458433Z return func(*args, **kwargs) 2025-12-04T09:47:02.9458774Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9459146Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9459512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9459922Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9460317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:02.9460683Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:02.9460812Z 2025-12-04T09:47:02.9460912Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9461261Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9461575Z res = mod(**inputs) 2025-12-04T09:47:02.9461909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9462269Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9462621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9462980Z layer_outputs = layer_module( 2025-12-04T09:47:02.9463307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9463659Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9464031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9464388Z return func(*args, **kwargs) 2025-12-04T09:47:02.9464722Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9465097Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9465468Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9465863Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9466249Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:02.9466611Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:02.9466735Z 2025-12-04T09:47:02.9466843Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9467187Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9467499Z res = mod(**inputs) 2025-12-04T09:47:02.9467832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9468195Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9468540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9468895Z layer_outputs = layer_module( 2025-12-04T09:47:02.9469230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9469599Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9469948Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9470316Z return func(*args, **kwargs) 2025-12-04T09:47:02.9470657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9471011Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9471379Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9471757Z return func(*args, **kwargs) 2025-12-04T09:47:02.9472095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9472448Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9472834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9473247Z return func(*args, **kwargs) 2025-12-04T09:47:02.9473597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:02.9473965Z query_states = self.q(hidden_states) 2025-12-04T09:47:02.9474102Z 2025-12-04T09:47:02.9474201Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9474547Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9474848Z res = mod(**inputs) 2025-12-04T09:47:02.9475189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9475550Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9475894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9476253Z layer_outputs = layer_module( 2025-12-04T09:47:02.9476598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9476951Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9477307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9477663Z return func(*args, **kwargs) 2025-12-04T09:47:02.9478003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9478362Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9478724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9479079Z return func(*args, **kwargs) 2025-12-04T09:47:02.9479418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9479778Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9480145Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9480495Z return func(*args, **kwargs) 2025-12-04T09:47:02.9480831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:02.9481184Z key_states = self.k(current_states) 2025-12-04T09:47:02.9481313Z 2025-12-04T09:47:02.9481413Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9481756Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9482061Z res = mod(**inputs) 2025-12-04T09:47:02.9482399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9482756Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9483121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9483473Z layer_outputs = layer_module( 2025-12-04T09:47:02.9483822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9484175Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9484532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9484913Z return func(*args, **kwargs) 2025-12-04T09:47:02.9485249Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9485611Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9485969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9486357Z return func(*args, **kwargs) 2025-12-04T09:47:02.9486696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9487058Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9487418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9487772Z return func(*args, **kwargs) 2025-12-04T09:47:02.9488110Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:02.9488463Z value_states = self.v(current_states) 2025-12-04T09:47:02.9488598Z 2025-12-04T09:47:02.9488675Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9488878Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9489105Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9489439Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9489747Z res = mod(**inputs) 2025-12-04T09:47:02.9490079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9490435Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9490789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9491144Z layer_outputs = layer_module( 2025-12-04T09:47:02.9491479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9491819Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9492190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9492554Z return func(*args, **kwargs) 2025-12-04T09:47:02.9492888Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9493251Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9493620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9493977Z return func(*args, **kwargs) 2025-12-04T09:47:02.9494310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9494678Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9495045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9495407Z return func(*args, **kwargs) 2025-12-04T09:47:02.9495738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:02.9496112Z attn_output = self.o(attn_output) 2025-12-04T09:47:02.9496237Z 2025-12-04T09:47:02.9496346Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9496701Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9497007Z res = mod(**inputs) 2025-12-04T09:47:02.9497338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9497698Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9498050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9498394Z layer_outputs = layer_module( 2025-12-04T09:47:02.9498717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9499077Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9499445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9499803Z return func(*args, **kwargs) 2025-12-04T09:47:02.9500146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9500513Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9500886Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9501287Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9501674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:02.9502037Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:02.9502171Z 2025-12-04T09:47:02.9502273Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9502614Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9502915Z res = mod(**inputs) 2025-12-04T09:47:02.9503247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9503600Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9503953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9504301Z layer_outputs = layer_module( 2025-12-04T09:47:02.9504636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9504988Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9505346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9505875Z return func(*args, **kwargs) 2025-12-04T09:47:02.9506223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9506586Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9506941Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9507329Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9507714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:02.9508068Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:02.9508200Z 2025-12-04T09:47:02.9508299Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9508634Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9508944Z res = mod(**inputs) 2025-12-04T09:47:02.9509300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9509663Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9510038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9510405Z layer_outputs = layer_module( 2025-12-04T09:47:02.9510735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9511087Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9511471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9511814Z return func(*args, **kwargs) 2025-12-04T09:47:02.9512150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9512548Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9512927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9513372Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9513763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:02.9514118Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:02.9514244Z 2025-12-04T09:47:02.9514349Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9514679Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9514986Z res = mod(**inputs) 2025-12-04T09:47:02.9515312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9515660Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9516012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9533382Z layer_outputs = layer_module( 2025-12-04T09:47:02.9533820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9534206Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9534599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9534981Z return func(*args, **kwargs) 2025-12-04T09:47:02.9535344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9535716Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9536092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9536450Z return func(*args, **kwargs) 2025-12-04T09:47:02.9536800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9537170Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9537540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9537901Z return func(*args, **kwargs) 2025-12-04T09:47:02.9538242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:02.9538614Z query_states = self.q(hidden_states) 2025-12-04T09:47:02.9538747Z 2025-12-04T09:47:02.9538854Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9539209Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9539534Z res = mod(**inputs) 2025-12-04T09:47:02.9539936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9540306Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9540693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9541055Z layer_outputs = layer_module( 2025-12-04T09:47:02.9541384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9541742Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9542129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9542485Z return func(*args, **kwargs) 2025-12-04T09:47:02.9542822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9543216Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9543586Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9543939Z return func(*args, **kwargs) 2025-12-04T09:47:02.9544280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9544649Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9545020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9545368Z return func(*args, **kwargs) 2025-12-04T09:47:02.9545705Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:02.9546062Z key_states = self.k(current_states) 2025-12-04T09:47:02.9546190Z 2025-12-04T09:47:02.9546292Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9546645Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9546958Z res = mod(**inputs) 2025-12-04T09:47:02.9547292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9547645Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9547997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9548352Z layer_outputs = layer_module( 2025-12-04T09:47:02.9548678Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9549028Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9549393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9549751Z return func(*args, **kwargs) 2025-12-04T09:47:02.9550082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9550443Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9550813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9551171Z return func(*args, **kwargs) 2025-12-04T09:47:02.9551502Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9551866Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9552233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9552582Z return func(*args, **kwargs) 2025-12-04T09:47:02.9552916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:02.9553384Z value_states = self.v(current_states) 2025-12-04T09:47:02.9553514Z 2025-12-04T09:47:02.9553604Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9553814Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9554043Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9554385Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9554686Z res = mod(**inputs) 2025-12-04T09:47:02.9555019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9555397Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9555746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9556093Z layer_outputs = layer_module( 2025-12-04T09:47:02.9556452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9556805Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9557172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9557520Z return func(*args, **kwargs) 2025-12-04T09:47:02.9557860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9558221Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9558583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9558937Z return func(*args, **kwargs) 2025-12-04T09:47:02.9559277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9559647Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9560008Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9560365Z return func(*args, **kwargs) 2025-12-04T09:47:02.9560708Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:02.9561058Z attn_output = self.o(attn_output) 2025-12-04T09:47:02.9561188Z 2025-12-04T09:47:02.9561287Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9561633Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9561946Z res = mod(**inputs) 2025-12-04T09:47:02.9562270Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9562630Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9562989Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9563342Z layer_outputs = layer_module( 2025-12-04T09:47:02.9563666Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9564017Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9564378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9564725Z return func(*args, **kwargs) 2025-12-04T09:47:02.9565066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9565425Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9565789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9566140Z return func(*args, **kwargs) 2025-12-04T09:47:02.9566499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 613, in forward 2025-12-04T09:47:02.9566910Z hidden_states = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:47:02.9567115Z 2025-12-04T09:47:02.9567215Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9567555Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9567865Z res = mod(**inputs) 2025-12-04T09:47:02.9568193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9568561Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9568913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9569269Z layer_outputs = layer_module( 2025-12-04T09:47:02.9569604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9569942Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9570291Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9570635Z return func(*args, **kwargs) 2025-12-04T09:47:02.9570956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9571321Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9571681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9572066Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9572444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:02.9572795Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:02.9572922Z 2025-12-04T09:47:02.9573028Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9573354Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9573655Z res = mod(**inputs) 2025-12-04T09:47:02.9573976Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9574322Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9574651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9574996Z layer_outputs = layer_module( 2025-12-04T09:47:02.9575320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9575652Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9576010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9576356Z return func(*args, **kwargs) 2025-12-04T09:47:02.9576689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9577049Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9577411Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9577798Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9578182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:02.9578524Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:02.9578658Z 2025-12-04T09:47:02.9578756Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9579112Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9579414Z res = mod(**inputs) 2025-12-04T09:47:02.9579761Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9580121Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9580469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9580814Z layer_outputs = layer_module( 2025-12-04T09:47:02.9581160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9581508Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9581859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9582232Z return func(*args, **kwargs) 2025-12-04T09:47:02.9582572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9582942Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9583303Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9583700Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9584094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:02.9584453Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:02.9584580Z 2025-12-04T09:47:02.9584680Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9585020Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9585328Z res = mod(**inputs) 2025-12-04T09:47:02.9585654Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9586004Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9586355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9586709Z layer_outputs = layer_module( 2025-12-04T09:47:02.9587032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9587377Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9587743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9588092Z return func(*args, **kwargs) 2025-12-04T09:47:02.9588432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9588793Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9589162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9589508Z return func(*args, **kwargs) 2025-12-04T09:47:02.9589851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9590214Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9590583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9590933Z return func(*args, **kwargs) 2025-12-04T09:47:02.9591269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:02.9591625Z query_states = self.q(hidden_states) 2025-12-04T09:47:02.9591749Z 2025-12-04T09:47:02.9591846Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9592208Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9592517Z res = mod(**inputs) 2025-12-04T09:47:02.9592858Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9593290Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9593663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9594043Z layer_outputs = layer_module( 2025-12-04T09:47:02.9594393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9594744Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9595108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9595481Z return func(*args, **kwargs) 2025-12-04T09:47:02.9595818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9596182Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9596551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9596904Z return func(*args, **kwargs) 2025-12-04T09:47:02.9597233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9597597Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9597963Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9598313Z return func(*args, **kwargs) 2025-12-04T09:47:02.9598650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:02.9599006Z key_states = self.k(current_states) 2025-12-04T09:47:02.9599132Z 2025-12-04T09:47:02.9599237Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9599571Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9599878Z res = mod(**inputs) 2025-12-04T09:47:02.9600480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9600830Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9601182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9601536Z layer_outputs = layer_module( 2025-12-04T09:47:02.9601867Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9602211Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9602577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9602931Z return func(*args, **kwargs) 2025-12-04T09:47:02.9603268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9603634Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9604000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9604354Z return func(*args, **kwargs) 2025-12-04T09:47:02.9604685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9605046Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9605408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9605933Z return func(*args, **kwargs) 2025-12-04T09:47:02.9606331Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:02.9606693Z value_states = self.v(current_states) 2025-12-04T09:47:02.9606856Z 2025-12-04T09:47:02.9606943Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9607139Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9607366Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9607710Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9608028Z res = mod(**inputs) 2025-12-04T09:47:02.9608353Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9608701Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9609040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9609409Z layer_outputs = layer_module( 2025-12-04T09:47:02.9609738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9610084Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9610438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9610788Z return func(*args, **kwargs) 2025-12-04T09:47:02.9611138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9611497Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9611852Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9612203Z return func(*args, **kwargs) 2025-12-04T09:47:02.9612545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9612900Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9613267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9613622Z return func(*args, **kwargs) 2025-12-04T09:47:02.9613959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:02.9614308Z attn_output = self.o(attn_output) 2025-12-04T09:47:02.9614440Z 2025-12-04T09:47:02.9614537Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9614882Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9615189Z res = mod(**inputs) 2025-12-04T09:47:02.9615516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9615876Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9616230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9616577Z layer_outputs = layer_module( 2025-12-04T09:47:02.9616910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9617262Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9617623Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9617974Z return func(*args, **kwargs) 2025-12-04T09:47:02.9618308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9618683Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9619072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9619465Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9619868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:02.9620242Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:02.9620369Z 2025-12-04T09:47:02.9620466Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9620812Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9621133Z res = mod(**inputs) 2025-12-04T09:47:02.9621462Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9621809Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9622154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9622527Z layer_outputs = layer_module( 2025-12-04T09:47:02.9622852Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9623199Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9623562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9623916Z return func(*args, **kwargs) 2025-12-04T09:47:02.9624248Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9624622Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9624994Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9625381Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9625773Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:02.9626132Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:02.9626258Z 2025-12-04T09:47:02.9626364Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9626698Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9627005Z res = mod(**inputs) 2025-12-04T09:47:02.9627336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9627690Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9628030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9628385Z layer_outputs = layer_module( 2025-12-04T09:47:02.9628719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9629059Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9629427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9629782Z return func(*args, **kwargs) 2025-12-04T09:47:02.9630123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9630486Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9630859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9631250Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9631630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:02.9631994Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:02.9632128Z 2025-12-04T09:47:02.9632245Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9632590Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9632907Z res = mod(**inputs) 2025-12-04T09:47:02.9633311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9633668Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9634015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9634385Z layer_outputs = layer_module( 2025-12-04T09:47:02.9634724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9635074Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9635453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9635819Z return func(*args, **kwargs) 2025-12-04T09:47:02.9636163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9636529Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9636884Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9637243Z return func(*args, **kwargs) 2025-12-04T09:47:02.9637583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9637942Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9638313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9638674Z return func(*args, **kwargs) 2025-12-04T09:47:02.9639011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:02.9639361Z query_states = self.q(hidden_states) 2025-12-04T09:47:02.9639495Z 2025-12-04T09:47:02.9639593Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9639936Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9640233Z res = mod(**inputs) 2025-12-04T09:47:02.9640564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9640920Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9641269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9641614Z layer_outputs = layer_module( 2025-12-04T09:47:02.9641957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9642304Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9642667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9643015Z return func(*args, **kwargs) 2025-12-04T09:47:02.9643349Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9643712Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9644074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9644431Z return func(*args, **kwargs) 2025-12-04T09:47:02.9644768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9645136Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9645512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9645870Z return func(*args, **kwargs) 2025-12-04T09:47:02.9646221Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:02.9646573Z key_states = self.k(current_states) 2025-12-04T09:47:02.9646705Z 2025-12-04T09:47:02.9646802Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9647140Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9647462Z res = mod(**inputs) 2025-12-04T09:47:02.9647782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9648153Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9648521Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9648867Z layer_outputs = layer_module( 2025-12-04T09:47:02.9649195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9649539Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9649899Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9650244Z return func(*args, **kwargs) 2025-12-04T09:47:02.9650580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9650946Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9651308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9651658Z return func(*args, **kwargs) 2025-12-04T09:47:02.9651997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9652357Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9652720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9653074Z return func(*args, **kwargs) 2025-12-04T09:47:02.9653409Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:02.9653763Z value_states = self.v(current_states) 2025-12-04T09:47:02.9653891Z 2025-12-04T09:47:02.9653966Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9654165Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9654384Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9654716Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9655022Z res = mod(**inputs) 2025-12-04T09:47:02.9655352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9655706Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9656046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9656400Z layer_outputs = layer_module( 2025-12-04T09:47:02.9656729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9657076Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9657437Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9657793Z return func(*args, **kwargs) 2025-12-04T09:47:02.9658135Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9658503Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9658874Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9659243Z return func(*args, **kwargs) 2025-12-04T09:47:02.9659577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9659940Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9660301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9660675Z return func(*args, **kwargs) 2025-12-04T09:47:02.9661008Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:02.9661363Z query_states = self.q(hidden_states) 2025-12-04T09:47:02.9661514Z 2025-12-04T09:47:02.9661621Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9661959Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9662261Z res = mod(**inputs) 2025-12-04T09:47:02.9662598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9662962Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9663305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9663659Z layer_outputs = layer_module( 2025-12-04T09:47:02.9663991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9664335Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9664691Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9665051Z return func(*args, **kwargs) 2025-12-04T09:47:02.9665388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9665741Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9666106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9666462Z return func(*args, **kwargs) 2025-12-04T09:47:02.9666800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9667156Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9667524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9667879Z return func(*args, **kwargs) 2025-12-04T09:47:02.9668215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:02.9668573Z key_states = self.k(current_states) 2025-12-04T09:47:02.9668704Z 2025-12-04T09:47:02.9668800Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9669143Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9669440Z res = mod(**inputs) 2025-12-04T09:47:02.9669769Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9670126Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9670472Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9670820Z layer_outputs = layer_module( 2025-12-04T09:47:02.9671148Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9671510Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9671856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9672215Z return func(*args, **kwargs) 2025-12-04T09:47:02.9672554Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9672917Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9673354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9673736Z return func(*args, **kwargs) 2025-12-04T09:47:02.9674074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9674427Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9674816Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9675171Z return func(*args, **kwargs) 2025-12-04T09:47:02.9675509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:02.9675864Z value_states = self.v(current_states) 2025-12-04T09:47:02.9676002Z 2025-12-04T09:47:02.9676078Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9676278Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9676492Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9676836Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9677145Z res = mod(**inputs) 2025-12-04T09:47:02.9677477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9677830Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9678185Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9678542Z layer_outputs = layer_module( 2025-12-04T09:47:02.9678864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9679209Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9679570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9679926Z return func(*args, **kwargs) 2025-12-04T09:47:02.9680255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9680612Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9680978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9681339Z return func(*args, **kwargs) 2025-12-04T09:47:02.9681669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9682031Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9682397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9682747Z return func(*args, **kwargs) 2025-12-04T09:47:02.9683084Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:02.9683439Z attn_output = self.o(attn_output) 2025-12-04T09:47:02.9683559Z 2025-12-04T09:47:02.9683666Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9683998Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9684306Z res = mod(**inputs) 2025-12-04T09:47:02.9684651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9685002Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9685372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9685725Z layer_outputs = layer_module( 2025-12-04T09:47:02.9686054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9686392Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9686768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9687119Z return func(*args, **kwargs) 2025-12-04T09:47:02.9687453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9687820Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9688185Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9688540Z return func(*args, **kwargs) 2025-12-04T09:47:02.9688870Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9689235Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9689600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9689954Z return func(*args, **kwargs) 2025-12-04T09:47:02.9690284Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:02.9690640Z query_states = self.q(hidden_states) 2025-12-04T09:47:02.9690766Z 2025-12-04T09:47:02.9690874Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9691211Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9691517Z res = mod(**inputs) 2025-12-04T09:47:02.9691848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9692205Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9692551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9692905Z layer_outputs = layer_module( 2025-12-04T09:47:02.9693336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9693679Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9694037Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9694440Z return func(*args, **kwargs) 2025-12-04T09:47:02.9694787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9695145Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9695521Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9695881Z return func(*args, **kwargs) 2025-12-04T09:47:02.9696223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9696583Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9696946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9697300Z return func(*args, **kwargs) 2025-12-04T09:47:02.9697631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:02.9698010Z attn_output = self.o(attn_output) 2025-12-04T09:47:02.9698139Z 2025-12-04T09:47:02.9698237Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9698583Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9698885Z res = mod(**inputs) 2025-12-04T09:47:02.9699215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9699568Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9699928Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9700280Z layer_outputs = layer_module( 2025-12-04T09:47:02.9700610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9700974Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9701325Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9701686Z return func(*args, **kwargs) 2025-12-04T09:47:02.9702026Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9702385Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9702742Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9703095Z return func(*args, **kwargs) 2025-12-04T09:47:02.9703440Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 613, in forward 2025-12-04T09:47:02.9703841Z hidden_states = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:47:02.9704027Z 2025-12-04T09:47:02.9704128Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9704468Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9704771Z res = mod(**inputs) 2025-12-04T09:47:02.9705092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9705448Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9705969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9706322Z layer_outputs = layer_module( 2025-12-04T09:47:02.9706661Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9707012Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9707380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9707780Z return func(*args, **kwargs) 2025-12-04T09:47:02.9708117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9708488Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9708859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9709250Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9709640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:02.9710002Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:02.9710131Z 2025-12-04T09:47:02.9710230Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9710573Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9710882Z res = mod(**inputs) 2025-12-04T09:47:02.9711260Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9711635Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9712017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9712390Z layer_outputs = layer_module( 2025-12-04T09:47:02.9712724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9713085Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9713607Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9713980Z return func(*args, **kwargs) 2025-12-04T09:47:02.9714323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9714743Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9715131Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9715539Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9715932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:02.9716296Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:02.9716424Z 2025-12-04T09:47:02.9716527Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9716868Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9717186Z res = mod(**inputs) 2025-12-04T09:47:02.9717525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:02.9717895Z encoder_outputs = self.encoder( 2025-12-04T09:47:02.9718247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9718607Z layer_outputs = layer_module( 2025-12-04T09:47:02.9718951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9719304Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9719676Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9720045Z return func(*args, **kwargs) 2025-12-04T09:47:02.9720390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9720765Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9721129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9721528Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9721917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:02.9722266Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:02.9722395Z 2025-12-04T09:47:02.9722490Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9722823Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9723121Z res = mod(**inputs) 2025-12-04T09:47:02.9723454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9723817Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9724160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9724507Z layer_outputs = layer_module( 2025-12-04T09:47:02.9724853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9725204Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9725576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9725936Z return func(*args, **kwargs) 2025-12-04T09:47:02.9726283Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9726697Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9727057Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9727409Z return func(*args, **kwargs) 2025-12-04T09:47:02.9727748Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9728129Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9728496Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9728858Z return func(*args, **kwargs) 2025-12-04T09:47:02.9729198Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:02.9729550Z key_states = self.k(current_states) 2025-12-04T09:47:02.9729682Z 2025-12-04T09:47:02.9729781Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9730121Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9730430Z res = mod(**inputs) 2025-12-04T09:47:02.9730761Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9731126Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9731479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9731826Z layer_outputs = layer_module( 2025-12-04T09:47:02.9732159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9732506Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9732866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9733217Z return func(*args, **kwargs) 2025-12-04T09:47:02.9733555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9733921Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9734289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9734645Z return func(*args, **kwargs) 2025-12-04T09:47:02.9734985Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9735352Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9735713Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9736071Z return func(*args, **kwargs) 2025-12-04T09:47:02.9736408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:02.9736483Z value_states = self.v(current_states) 2025-12-04T09:47:02.9736487Z 2025-12-04T09:47:02.9736568Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9736642Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9736739Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9736951Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9737012Z res = mod(**inputs) 2025-12-04T09:47:02.9737256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9737336Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9737560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9737633Z layer_outputs = layer_module( 2025-12-04T09:47:02.9737847Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9737935Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9738172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9738248Z return func(*args, **kwargs) 2025-12-04T09:47:02.9738473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9738556Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9738785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9738856Z return func(*args, **kwargs) 2025-12-04T09:47:02.9739085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9739162Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9739399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9739461Z return func(*args, **kwargs) 2025-12-04T09:47:02.9739690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:02.9739763Z attn_output = self.o(attn_output) 2025-12-04T09:47:02.9739768Z 2025-12-04T09:47:02.9739865Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9740061Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9740121Z res = mod(**inputs) 2025-12-04T09:47:02.9740356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9740433Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9740662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9740739Z layer_outputs = layer_module( 2025-12-04T09:47:02.9740950Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9741021Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9741261Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9741325Z return func(*args, **kwargs) 2025-12-04T09:47:02.9741551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9741646Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9741869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9741988Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9742212Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:02.9742287Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:02.9742291Z 2025-12-04T09:47:02.9742397Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9742598Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9742669Z res = mod(**inputs) 2025-12-04T09:47:02.9742900Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9742970Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9743196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9743262Z layer_outputs = layer_module( 2025-12-04T09:47:02.9743497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9743575Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9743805Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9743891Z return func(*args, **kwargs) 2025-12-04T09:47:02.9744124Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9744210Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9744445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9744557Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9744792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:02.9744871Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:02.9744874Z 2025-12-04T09:47:02.9744969Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9745168Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9745227Z res = mod(**inputs) 2025-12-04T09:47:02.9745457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9745531Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9745761Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9745835Z layer_outputs = layer_module( 2025-12-04T09:47:02.9746048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9746122Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9746366Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9746431Z return func(*args, **kwargs) 2025-12-04T09:47:02.9746657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9746751Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9746980Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9747095Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9747323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:02.9747399Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:02.9747403Z 2025-12-04T09:47:02.9747506Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9747696Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9747763Z res = mod(**inputs) 2025-12-04T09:47:02.9747991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9748060Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9748319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9748387Z layer_outputs = layer_module( 2025-12-04T09:47:02.9748618Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9748699Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9748931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9749004Z return func(*args, **kwargs) 2025-12-04T09:47:02.9749245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9749321Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9749557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9749640Z return func(*args, **kwargs) 2025-12-04T09:47:02.9749872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9749949Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9750177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9750247Z return func(*args, **kwargs) 2025-12-04T09:47:02.9750470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:02.9750542Z query_states = self.q(hidden_states) 2025-12-04T09:47:02.9750546Z 2025-12-04T09:47:02.9750648Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9750832Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9750897Z res = mod(**inputs) 2025-12-04T09:47:02.9751120Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9751187Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9751420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9751487Z layer_outputs = layer_module( 2025-12-04T09:47:02.9751694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9751774Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9752004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9752073Z return func(*args, **kwargs) 2025-12-04T09:47:02.9752294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9752369Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9752608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9752670Z return func(*args, **kwargs) 2025-12-04T09:47:02.9752899Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9752976Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9753265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9753346Z return func(*args, **kwargs) 2025-12-04T09:47:02.9753571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:02.9753641Z key_states = self.k(current_states) 2025-12-04T09:47:02.9753645Z 2025-12-04T09:47:02.9753748Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9753954Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9754023Z res = mod(**inputs) 2025-12-04T09:47:02.9754268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9754339Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9754572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9754639Z layer_outputs = layer_module( 2025-12-04T09:47:02.9754864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9754945Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9755171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9755259Z return func(*args, **kwargs) 2025-12-04T09:47:02.9755483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9755559Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9755798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9755862Z return func(*args, **kwargs) 2025-12-04T09:47:02.9756089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9756167Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9756398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9756469Z return func(*args, **kwargs) 2025-12-04T09:47:02.9756691Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:02.9756768Z value_states = self.v(current_states) 2025-12-04T09:47:02.9756771Z 2025-12-04T09:47:02.9756853Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9756928Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9757034Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9757225Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9757287Z res = mod(**inputs) 2025-12-04T09:47:02.9757523Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9757593Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9757817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9757891Z layer_outputs = layer_module( 2025-12-04T09:47:02.9758104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9758185Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9758415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9758477Z return func(*args, **kwargs) 2025-12-04T09:47:02.9758705Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9758779Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9759006Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9759079Z return func(*args, **kwargs) 2025-12-04T09:47:02.9759301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9759386Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9759628Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9759691Z return func(*args, **kwargs) 2025-12-04T09:47:02.9759933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:02.9760006Z attn_output = self.o(attn_output) 2025-12-04T09:47:02.9760009Z 2025-12-04T09:47:02.9760111Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9760292Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9760366Z res = mod(**inputs) 2025-12-04T09:47:02.9760597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9760665Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9760892Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9760981Z layer_outputs = layer_module( 2025-12-04T09:47:02.9761192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9761273Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9761502Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9761564Z return func(*args, **kwargs) 2025-12-04T09:47:02.9761795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9761873Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9762100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9762171Z return func(*args, **kwargs) 2025-12-04T09:47:02.9762397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9762483Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9762710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9762772Z return func(*args, **kwargs) 2025-12-04T09:47:02.9763003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:02.9763075Z query_states = self.q(hidden_states) 2025-12-04T09:47:02.9763079Z 2025-12-04T09:47:02.9763183Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9763367Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9763428Z res = mod(**inputs) 2025-12-04T09:47:02.9763657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9763728Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9763951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9764024Z layer_outputs = layer_module( 2025-12-04T09:47:02.9764234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9764310Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9764539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9764601Z return func(*args, **kwargs) 2025-12-04T09:47:02.9764830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9764905Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9765156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9765219Z return func(*args, **kwargs) 2025-12-04T09:47:02.9765453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9765537Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9765762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9765824Z return func(*args, **kwargs) 2025-12-04T09:47:02.9766072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:02.9766144Z key_states = self.k(current_states) 2025-12-04T09:47:02.9766148Z 2025-12-04T09:47:02.9766248Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9766449Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9766510Z res = mod(**inputs) 2025-12-04T09:47:02.9766743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9766811Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9767035Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9767108Z layer_outputs = layer_module( 2025-12-04T09:47:02.9767316Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9767397Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9767627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9767689Z return func(*args, **kwargs) 2025-12-04T09:47:02.9767926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9768001Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9768238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9768299Z return func(*args, **kwargs) 2025-12-04T09:47:02.9768523Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9768605Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9768837Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9768898Z return func(*args, **kwargs) 2025-12-04T09:47:02.9769128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:02.9769203Z value_states = self.v(current_states) 2025-12-04T09:47:02.9769206Z 2025-12-04T09:47:02.9769287Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9769360Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9769455Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9769649Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9769709Z res = mod(**inputs) 2025-12-04T09:47:02.9769936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9770013Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9770239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9770315Z layer_outputs = layer_module( 2025-12-04T09:47:02.9770524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9770598Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9770868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9770954Z return func(*args, **kwargs) 2025-12-04T09:47:02.9771180Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9771266Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9771491Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9771577Z return func(*args, **kwargs) 2025-12-04T09:47:02.9771807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9771882Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9772135Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9772199Z return func(*args, **kwargs) 2025-12-04T09:47:02.9772431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:02.9772502Z attn_output = self.o(attn_output) 2025-12-04T09:47:02.9772506Z 2025-12-04T09:47:02.9772600Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9772793Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9772855Z res = mod(**inputs) 2025-12-04T09:47:02.9773081Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9773155Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9773380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9773457Z layer_outputs = layer_module( 2025-12-04T09:47:02.9773669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9773743Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9773980Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9774045Z return func(*args, **kwargs) 2025-12-04T09:47:02.9774266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9774362Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9774584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9774703Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9774929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:02.9775004Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:02.9775008Z 2025-12-04T09:47:02.9775112Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9775297Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9775365Z res = mod(**inputs) 2025-12-04T09:47:02.9775591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9775660Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9775891Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9775958Z layer_outputs = layer_module( 2025-12-04T09:47:02.9776167Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9776264Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9776495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9776581Z return func(*args, **kwargs) 2025-12-04T09:47:02.9776807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9776894Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9777128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9777254Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9777482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:02.9777558Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:02.9777578Z 2025-12-04T09:47:02.9777677Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9777875Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9777935Z res = mod(**inputs) 2025-12-04T09:47:02.9778164Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9778239Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9778463Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9778536Z layer_outputs = layer_module( 2025-12-04T09:47:02.9778747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9778819Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9779055Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9779123Z return func(*args, **kwargs) 2025-12-04T09:47:02.9779346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9779444Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9779669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9779783Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9780006Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:02.9780081Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:02.9780085Z 2025-12-04T09:47:02.9780185Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9780373Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9780440Z res = mod(**inputs) 2025-12-04T09:47:02.9780668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9780737Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9780972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9781039Z layer_outputs = layer_module( 2025-12-04T09:47:02.9781250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9781333Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9781561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9781631Z return func(*args, **kwargs) 2025-12-04T09:47:02.9781854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9781951Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9782189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9782273Z return func(*args, **kwargs) 2025-12-04T09:47:02.9782507Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9782584Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9782809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9782896Z return func(*args, **kwargs) 2025-12-04T09:47:02.9783122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:02.9783196Z query_states = self.q(hidden_states) 2025-12-04T09:47:02.9783215Z 2025-12-04T09:47:02.9783322Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9783513Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9783583Z res = mod(**inputs) 2025-12-04T09:47:02.9783813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9783883Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9784119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9784187Z layer_outputs = layer_module( 2025-12-04T09:47:02.9784398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9784480Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9784708Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9784782Z return func(*args, **kwargs) 2025-12-04T09:47:02.9785007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9785085Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9785321Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9785384Z return func(*args, **kwargs) 2025-12-04T09:47:02.9785614Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9785693Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9785919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9785987Z return func(*args, **kwargs) 2025-12-04T09:47:02.9786216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:02.9786286Z key_states = self.k(current_states) 2025-12-04T09:47:02.9786290Z 2025-12-04T09:47:02.9786394Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9786578Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9786645Z res = mod(**inputs) 2025-12-04T09:47:02.9786871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9786943Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9787176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9787241Z layer_outputs = layer_module( 2025-12-04T09:47:02.9787453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9787552Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9787783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9787868Z return func(*args, **kwargs) 2025-12-04T09:47:02.9788092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9788166Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9788404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9788482Z return func(*args, **kwargs) 2025-12-04T09:47:02.9788717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9788794Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9789047Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9789119Z return func(*args, **kwargs) 2025-12-04T09:47:02.9789343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:02.9789414Z value_states = self.v(current_states) 2025-12-04T09:47:02.9789417Z 2025-12-04T09:47:02.9789499Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9789571Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9789673Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9789858Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9789918Z res = mod(**inputs) 2025-12-04T09:47:02.9790148Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9790219Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9790446Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9790519Z layer_outputs = layer_module( 2025-12-04T09:47:02.9790730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9790811Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9791038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9791102Z return func(*args, **kwargs) 2025-12-04T09:47:02.9791330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9791405Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9791630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9791703Z return func(*args, **kwargs) 2025-12-04T09:47:02.9791923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9792007Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9792237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9792299Z return func(*args, **kwargs) 2025-12-04T09:47:02.9792527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:02.9792602Z attn_output = self.o(attn_output) 2025-12-04T09:47:02.9792605Z 2025-12-04T09:47:02.9792704Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9792888Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9792950Z res = mod(**inputs) 2025-12-04T09:47:02.9793274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9793351Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9793590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9793666Z layer_outputs = layer_module( 2025-12-04T09:47:02.9793871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9793953Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9794201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9794263Z return func(*args, **kwargs) 2025-12-04T09:47:02.9794491Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9794586Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9794817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9794889Z return func(*args, **kwargs) 2025-12-04T09:47:02.9795112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 613, in forward 2025-12-04T09:47:02.9795246Z hidden_states = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:47:02.9795249Z 2025-12-04T09:47:02.9795346Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9795531Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9795601Z res = mod(**inputs) 2025-12-04T09:47:02.9795827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9795904Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9796127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9796194Z layer_outputs = layer_module( 2025-12-04T09:47:02.9796410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9796483Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9796720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9796793Z return func(*args, **kwargs) 2025-12-04T09:47:02.9797013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9797096Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9797322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9797387Z return func(*args, **kwargs) 2025-12-04T09:47:02.9797621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9797700Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9797935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9797998Z return func(*args, **kwargs) 2025-12-04T09:47:02.9798218Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:02.9798299Z query_states = self.q(hidden_states) 2025-12-04T09:47:02.9798303Z 2025-12-04T09:47:02.9798399Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9798581Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9798651Z res = mod(**inputs) 2025-12-04T09:47:02.9798891Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9798969Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9799210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9799277Z layer_outputs = layer_module( 2025-12-04T09:47:02.9799497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9799571Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9799816Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9799885Z return func(*args, **kwargs) 2025-12-04T09:47:02.9800109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9800205Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9800434Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9800501Z return func(*args, **kwargs) 2025-12-04T09:47:02.9800737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9800816Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9801050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9801114Z return func(*args, **kwargs) 2025-12-04T09:47:02.9801334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:02.9801413Z key_states = self.k(current_states) 2025-12-04T09:47:02.9801417Z 2025-12-04T09:47:02.9801515Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9801698Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9801768Z res = mod(**inputs) 2025-12-04T09:47:02.9801995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9802071Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9802297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9802363Z layer_outputs = layer_module( 2025-12-04T09:47:02.9802584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9802656Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9802882Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9802954Z return func(*args, **kwargs) 2025-12-04T09:47:02.9803178Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9803260Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9803486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9803549Z return func(*args, **kwargs) 2025-12-04T09:47:02.9803779Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9803856Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9804091Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9804156Z return func(*args, **kwargs) 2025-12-04T09:47:02.9804375Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:02.9804474Z value_states = self.v(current_states) 2025-12-04T09:47:02.9804477Z 2025-12-04T09:47:02.9804550Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9804637Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9804741Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9804923Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9804987Z res = mod(**inputs) 2025-12-04T09:47:02.9805214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9805296Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9805527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9805592Z layer_outputs = layer_module( 2025-12-04T09:47:02.9805980Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9806066Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9806298Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9806369Z return func(*args, **kwargs) 2025-12-04T09:47:02.9806592Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9806666Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9806904Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9806968Z return func(*args, **kwargs) 2025-12-04T09:47:02.9807193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9807279Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9807516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9807591Z return func(*args, **kwargs) 2025-12-04T09:47:02.9807823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:02.9807895Z attn_output = self.o(attn_output) 2025-12-04T09:47:02.9807898Z 2025-12-04T09:47:02.9808003Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9808184Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9808249Z res = mod(**inputs) 2025-12-04T09:47:02.9808469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9808534Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9808764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9808827Z layer_outputs = layer_module( 2025-12-04T09:47:02.9809035Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9809112Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9809335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9809403Z return func(*args, **kwargs) 2025-12-04T09:47:02.9809627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9809713Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9809938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9810045Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9810310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:02.9810386Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:02.9810412Z 2025-12-04T09:47:02.9810508Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9810701Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9810759Z res = mod(**inputs) 2025-12-04T09:47:02.9810986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9811093Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9811321Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9811397Z layer_outputs = layer_module( 2025-12-04T09:47:02.9811633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9811709Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9811947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9812011Z return func(*args, **kwargs) 2025-12-04T09:47:02.9812232Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9812327Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9812549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9812664Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9812888Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:02.9812966Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:02.9812971Z 2025-12-04T09:47:02.9813075Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9813264Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9813329Z res = mod(**inputs) 2025-12-04T09:47:02.9813553Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9813621Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9813851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9813921Z layer_outputs = layer_module( 2025-12-04T09:47:02.9814132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9814215Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9814444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9814517Z return func(*args, **kwargs) 2025-12-04T09:47:02.9814740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9814825Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9815052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9815164Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9815390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:02.9815465Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:02.9815469Z 2025-12-04T09:47:02.9815563Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9815773Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9815834Z res = mod(**inputs) 2025-12-04T09:47:02.9816072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9816150Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9816373Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9816446Z layer_outputs = layer_module( 2025-12-04T09:47:02.9816672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9816744Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9816974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9817054Z return func(*args, **kwargs) 2025-12-04T09:47:02.9817276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9817359Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9817586Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9817655Z return func(*args, **kwargs) 2025-12-04T09:47:02.9817875Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9817951Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9818182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9818242Z return func(*args, **kwargs) 2025-12-04T09:47:02.9818467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:02.9818542Z query_states = self.q(hidden_states) 2025-12-04T09:47:02.9818546Z 2025-12-04T09:47:02.9818640Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9818826Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9818883Z res = mod(**inputs) 2025-12-04T09:47:02.9819104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9819177Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9819399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9819467Z layer_outputs = layer_module( 2025-12-04T09:47:02.9819672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9819743Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9819977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9820037Z return func(*args, **kwargs) 2025-12-04T09:47:02.9820259Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9820339Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9820567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9820637Z return func(*args, **kwargs) 2025-12-04T09:47:02.9820859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9820934Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9821166Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9821230Z return func(*args, **kwargs) 2025-12-04T09:47:02.9821787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:02.9821862Z key_states = self.k(current_states) 2025-12-04T09:47:02.9821918Z 2025-12-04T09:47:02.9822018Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9822210Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9822271Z res = mod(**inputs) 2025-12-04T09:47:02.9822495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9822590Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9822822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9822915Z layer_outputs = layer_module( 2025-12-04T09:47:02.9823124Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9823198Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9823435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9823501Z return func(*args, **kwargs) 2025-12-04T09:47:02.9823725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9823808Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9824036Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9824107Z return func(*args, **kwargs) 2025-12-04T09:47:02.9824326Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9824403Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9824637Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9824697Z return func(*args, **kwargs) 2025-12-04T09:47:02.9824927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:02.9825000Z value_states = self.v(current_states) 2025-12-04T09:47:02.9825003Z 2025-12-04T09:47:02.9825077Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9825161Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9825257Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9825444Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9825514Z res = mod(**inputs) 2025-12-04T09:47:02.9825737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9825817Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9826039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9826108Z layer_outputs = layer_module( 2025-12-04T09:47:02.9826326Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9826398Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9826628Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9826699Z return func(*args, **kwargs) 2025-12-04T09:47:02.9826920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9827001Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9827250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9827314Z return func(*args, **kwargs) 2025-12-04T09:47:02.9827558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9827635Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9827864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9827936Z return func(*args, **kwargs) 2025-12-04T09:47:02.9828173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:02.9828254Z attn_output = self.o(attn_output) 2025-12-04T09:47:02.9828257Z 2025-12-04T09:47:02.9828351Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9828556Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9828627Z res = mod(**inputs) 2025-12-04T09:47:02.9828849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9828929Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9829151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9829216Z layer_outputs = layer_module( 2025-12-04T09:47:02.9829436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9829511Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9829739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9829809Z return func(*args, **kwargs) 2025-12-04T09:47:02.9830035Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9830118Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9830348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9830412Z return func(*args, **kwargs) 2025-12-04T09:47:02.9830647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9830724Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9830961Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9831025Z return func(*args, **kwargs) 2025-12-04T09:47:02.9831247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:02.9831329Z query_states = self.q(hidden_states) 2025-12-04T09:47:02.9831332Z 2025-12-04T09:47:02.9831430Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9831615Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9831683Z res = mod(**inputs) 2025-12-04T09:47:02.9831906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9831983Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9832208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9832275Z layer_outputs = layer_module( 2025-12-04T09:47:02.9832493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9832563Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9832792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9832877Z return func(*args, **kwargs) 2025-12-04T09:47:02.9833123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9833267Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9833505Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9833569Z return func(*args, **kwargs) 2025-12-04T09:47:02.9833817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9833903Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9834145Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9834227Z return func(*args, **kwargs) 2025-12-04T09:47:02.9834447Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:02.9834525Z key_states = self.k(current_states) 2025-12-04T09:47:02.9834528Z 2025-12-04T09:47:02.9834625Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9834809Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9834876Z res = mod(**inputs) 2025-12-04T09:47:02.9835095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9835175Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9835396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9835462Z layer_outputs = layer_module( 2025-12-04T09:47:02.9835678Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9835751Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9835978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9836051Z return func(*args, **kwargs) 2025-12-04T09:47:02.9836269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9836352Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9836577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9836645Z return func(*args, **kwargs) 2025-12-04T09:47:02.9836878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9836957Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9837190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9837254Z return func(*args, **kwargs) 2025-12-04T09:47:02.9837472Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:02.9837552Z value_states = self.v(current_states) 2025-12-04T09:47:02.9837555Z 2025-12-04T09:47:02.9837628Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9837700Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9837806Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9837988Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9838056Z res = mod(**inputs) 2025-12-04T09:47:02.9838276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9838346Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9838592Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9838659Z layer_outputs = layer_module( 2025-12-04T09:47:02.9838882Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9838962Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9839190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9839282Z return func(*args, **kwargs) 2025-12-04T09:47:02.9839507Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9839581Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9839818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9839903Z return func(*args, **kwargs) 2025-12-04T09:47:02.9840128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9840213Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9840441Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9840510Z return func(*args, **kwargs) 2025-12-04T09:47:02.9840729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:02.9840802Z attn_output = self.o(attn_output) 2025-12-04T09:47:02.9840805Z 2025-12-04T09:47:02.9840912Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9841095Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9841164Z res = mod(**inputs) 2025-12-04T09:47:02.9841392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9841459Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9841689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9841755Z layer_outputs = layer_module( 2025-12-04T09:47:02.9841964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9842044Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9842269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9842340Z return func(*args, **kwargs) 2025-12-04T09:47:02.9842562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9842640Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9842881Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9842944Z return func(*args, **kwargs) 2025-12-04T09:47:02.9843167Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 652, in forward 2025-12-04T09:47:02.9843295Z layer_output = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:47:02.9843298Z 2025-12-04T09:47:02.9843394Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9843588Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9843648Z res = mod(**inputs) 2025-12-04T09:47:02.9843872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9843950Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9844189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9844266Z layer_outputs = layer_module( 2025-12-04T09:47:02.9844484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9844555Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9844785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9844863Z return func(*args, **kwargs) 2025-12-04T09:47:02.9845085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9845176Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9845399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9845531Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9845756Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:02.9845829Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:02.9845832Z 2025-12-04T09:47:02.9845934Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9846119Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9846188Z res = mod(**inputs) 2025-12-04T09:47:02.9846414Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9846479Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9846710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9846777Z layer_outputs = layer_module( 2025-12-04T09:47:02.9846988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9847068Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9847295Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9847364Z return func(*args, **kwargs) 2025-12-04T09:47:02.9847586Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9847674Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9847906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9848012Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9848235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:02.9848319Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:02.9848322Z 2025-12-04T09:47:02.9848417Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9848616Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9848676Z res = mod(**inputs) 2025-12-04T09:47:02.9848902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9848980Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9849204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9849277Z layer_outputs = layer_module( 2025-12-04T09:47:02.9849489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9849582Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9849821Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9849899Z return func(*args, **kwargs) 2025-12-04T09:47:02.9850124Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9850214Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9850436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9850568Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9850797Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:02.9850869Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:02.9850888Z 2025-12-04T09:47:02.9850994Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9851180Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9851248Z res = mod(**inputs) 2025-12-04T09:47:02.9851477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9851548Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9851782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9851850Z layer_outputs = layer_module( 2025-12-04T09:47:02.9852063Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9852145Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9852374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9852447Z return func(*args, **kwargs) 2025-12-04T09:47:02.9852673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9852752Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9852988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9853054Z return func(*args, **kwargs) 2025-12-04T09:47:02.9853276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9853361Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9853590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9853660Z return func(*args, **kwargs) 2025-12-04T09:47:02.9853884Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:02.9853956Z query_states = self.q(hidden_states) 2025-12-04T09:47:02.9853959Z 2025-12-04T09:47:02.9854066Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9854250Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9854317Z res = mod(**inputs) 2025-12-04T09:47:02.9854542Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9854611Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9854844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9854910Z layer_outputs = layer_module( 2025-12-04T09:47:02.9855119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9855224Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9855454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9855539Z return func(*args, **kwargs) 2025-12-04T09:47:02.9855764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9855839Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9856078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9856158Z return func(*args, **kwargs) 2025-12-04T09:47:02.9856377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9856462Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9856709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9856779Z return func(*args, **kwargs) 2025-12-04T09:47:02.9857004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:02.9857076Z key_states = self.k(current_states) 2025-12-04T09:47:02.9857079Z 2025-12-04T09:47:02.9857183Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9857367Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9857438Z res = mod(**inputs) 2025-12-04T09:47:02.9857663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9857731Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9857962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9858034Z layer_outputs = layer_module( 2025-12-04T09:47:02.9858244Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9858323Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9858553Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9858623Z return func(*args, **kwargs) 2025-12-04T09:47:02.9858845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9858922Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9859160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9859222Z return func(*args, **kwargs) 2025-12-04T09:47:02.9859447Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9859532Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9859762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9859830Z return func(*args, **kwargs) 2025-12-04T09:47:02.9860053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:02.9860124Z value_states = self.v(current_states) 2025-12-04T09:47:02.9860129Z 2025-12-04T09:47:02.9860209Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9860281Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9860387Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9860572Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9860632Z res = mod(**inputs) 2025-12-04T09:47:02.9860879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9860952Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9861198Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9861273Z layer_outputs = layer_module( 2025-12-04T09:47:02.9861485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9861564Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9861822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9861885Z return func(*args, **kwargs) 2025-12-04T09:47:02.9862115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9862206Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9862443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9862513Z return func(*args, **kwargs) 2025-12-04T09:47:02.9862736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9862819Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9863046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9863111Z return func(*args, **kwargs) 2025-12-04T09:47:02.9863339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:02.9863410Z attn_output = self.o(attn_output) 2025-12-04T09:47:02.9863413Z 2025-12-04T09:47:02.9863519Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9863707Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9863767Z res = mod(**inputs) 2025-12-04T09:47:02.9864000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9864069Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9864294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9864368Z layer_outputs = layer_module( 2025-12-04T09:47:02.9864577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9864656Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9864886Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9864950Z return func(*args, **kwargs) 2025-12-04T09:47:02.9865181Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9865256Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9865483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9865553Z return func(*args, **kwargs) 2025-12-04T09:47:02.9865780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9865868Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9866097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9866159Z return func(*args, **kwargs) 2025-12-04T09:47:02.9866386Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:02.9866477Z query_states = self.q(hidden_states) 2025-12-04T09:47:02.9866481Z 2025-12-04T09:47:02.9866584Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9866782Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9866842Z res = mod(**inputs) 2025-12-04T09:47:02.9867074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9867141Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9867379Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9867454Z layer_outputs = layer_module( 2025-12-04T09:47:02.9867662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9867755Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9867984Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9868046Z return func(*args, **kwargs) 2025-12-04T09:47:02.9868275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9868347Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9868574Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9868645Z return func(*args, **kwargs) 2025-12-04T09:47:02.9868866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9868949Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9869175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9869242Z return func(*args, **kwargs) 2025-12-04T09:47:02.9869469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:02.9869540Z key_states = self.k(current_states) 2025-12-04T09:47:02.9869543Z 2025-12-04T09:47:02.9869644Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9869825Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9869884Z res = mod(**inputs) 2025-12-04T09:47:02.9870115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9870182Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9870408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9870483Z layer_outputs = layer_module( 2025-12-04T09:47:02.9870694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9870774Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9871003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9871066Z return func(*args, **kwargs) 2025-12-04T09:47:02.9871293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9871370Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9871598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9871669Z return func(*args, **kwargs) 2025-12-04T09:47:02.9871887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9871985Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9872215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9872294Z return func(*args, **kwargs) 2025-12-04T09:47:02.9872523Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:02.9872597Z value_states = self.v(current_states) 2025-12-04T09:47:02.9872600Z 2025-12-04T09:47:02.9872680Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9872768Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9872865Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9873060Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9873119Z res = mod(**inputs) 2025-12-04T09:47:02.9873497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9873579Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9873807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9873880Z layer_outputs = layer_module( 2025-12-04T09:47:02.9874094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9874165Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9874405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9874469Z return func(*args, **kwargs) 2025-12-04T09:47:02.9874691Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9874777Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9875009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9875079Z return func(*args, **kwargs) 2025-12-04T09:47:02.9875311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9875390Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9875638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9875701Z return func(*args, **kwargs) 2025-12-04T09:47:02.9875922Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:02.9876004Z attn_output = self.o(attn_output) 2025-12-04T09:47:02.9876007Z 2025-12-04T09:47:02.9876104Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9876304Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9876366Z res = mod(**inputs) 2025-12-04T09:47:02.9876594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9876671Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9876895Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9876968Z layer_outputs = layer_module( 2025-12-04T09:47:02.9877182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9877255Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9877495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9877559Z return func(*args, **kwargs) 2025-12-04T09:47:02.9877801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9877897Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9878137Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9878252Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9878476Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:02.9878566Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:02.9878569Z 2025-12-04T09:47:02.9878672Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9878854Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9878924Z res = mod(**inputs) 2025-12-04T09:47:02.9879166Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9879236Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9879471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9879542Z layer_outputs = layer_module( 2025-12-04T09:47:02.9879751Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9879831Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9880060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9880131Z return func(*args, **kwargs) 2025-12-04T09:47:02.9880352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9880441Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9880676Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9880786Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9881011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:02.9881093Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:02.9881096Z 2025-12-04T09:47:02.9881191Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9881384Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9881445Z res = mod(**inputs) 2025-12-04T09:47:02.9881673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9881748Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9881975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9882048Z layer_outputs = layer_module( 2025-12-04T09:47:02.9882260Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9882333Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9882569Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9882633Z return func(*args, **kwargs) 2025-12-04T09:47:02.9882860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9882950Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9883173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9883290Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9883527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:02.9883601Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:02.9883618Z 2025-12-04T09:47:02.9883723Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9883915Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9883980Z res = mod(**inputs) 2025-12-04T09:47:02.9884207Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9884289Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9884522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9884607Z layer_outputs = layer_module( 2025-12-04T09:47:02.9884818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9884898Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9885129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9885199Z return func(*args, **kwargs) 2025-12-04T09:47:02.9885419Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9885504Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9885747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 344, in forward 2025-12-04T09:47:02.9885868Z hidden_states = hidden_states + self.dropout(forwarded_states) 2025-12-04T09:47:02.9885871Z 2025-12-04T09:47:02.9885974Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9886158Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9886218Z res = mod(**inputs) 2025-12-04T09:47:02.9886450Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9886519Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9886745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9886818Z layer_outputs = layer_module( 2025-12-04T09:47:02.9887026Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9887104Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9887335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9887402Z return func(*args, **kwargs) 2025-12-04T09:47:02.9887633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9887709Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9887940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9888010Z return func(*args, **kwargs) 2025-12-04T09:47:02.9888233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9888318Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9888545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9888606Z return func(*args, **kwargs) 2025-12-04T09:47:02.9888836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:02.9888908Z query_states = self.q(hidden_states) 2025-12-04T09:47:02.9888930Z 2025-12-04T09:47:02.9889033Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9889238Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9889301Z res = mod(**inputs) 2025-12-04T09:47:02.9889535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9889602Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9889824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9889913Z layer_outputs = layer_module( 2025-12-04T09:47:02.9890123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9890217Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9890444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9890507Z return func(*args, **kwargs) 2025-12-04T09:47:02.9890740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9890816Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9891047Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9891117Z return func(*args, **kwargs) 2025-12-04T09:47:02.9891341Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9891426Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9891651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9891716Z return func(*args, **kwargs) 2025-12-04T09:47:02.9891950Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:02.9892021Z key_states = self.k(current_states) 2025-12-04T09:47:02.9892025Z 2025-12-04T09:47:02.9892128Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9892314Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9892375Z res = mod(**inputs) 2025-12-04T09:47:02.9892607Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9892678Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9892905Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9892979Z layer_outputs = layer_module( 2025-12-04T09:47:02.9893199Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9893280Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9893509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9893576Z return func(*args, **kwargs) 2025-12-04T09:47:02.9893810Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9893885Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9894116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9894187Z return func(*args, **kwargs) 2025-12-04T09:47:02.9894411Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9894494Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9894741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9894807Z return func(*args, **kwargs) 2025-12-04T09:47:02.9895054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:02.9895128Z value_states = self.v(current_states) 2025-12-04T09:47:02.9895131Z 2025-12-04T09:47:02.9895212Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9895285Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9895403Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9895596Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9895654Z res = mod(**inputs) 2025-12-04T09:47:02.9895877Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9895971Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9896196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9896271Z layer_outputs = layer_module( 2025-12-04T09:47:02.9896480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9896553Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9896788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9896853Z return func(*args, **kwargs) 2025-12-04T09:47:02.9897074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:02.9897156Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:02.9897387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9897457Z return func(*args, **kwargs) 2025-12-04T09:47:02.9897681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:02.9897757Z attention_output = self.SelfAttention( 2025-12-04T09:47:02.9897992Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9898055Z return func(*args, **kwargs) 2025-12-04T09:47:02.9898278Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:02.9898356Z attn_output = self.o(attn_output) 2025-12-04T09:47:02.9898359Z 2025-12-04T09:47:02.9898454Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9898644Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9898707Z res = mod(**inputs) 2025-12-04T09:47:02.9898940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9899017Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9899244Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9899316Z layer_outputs = layer_module( 2025-12-04T09:47:02.9899527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9899600Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9899835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9899898Z return func(*args, **kwargs) 2025-12-04T09:47:02.9900122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9900224Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9900464Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9900534Z return func(*args, **kwargs) 2025-12-04T09:47:02.9900760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9900839Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9901089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9901152Z return func(*args, **kwargs) 2025-12-04T09:47:02.9901372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:02.9901466Z query_states = self.q(hidden_states) 2025-12-04T09:47:02.9901470Z 2025-12-04T09:47:02.9901566Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9901760Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9901822Z res = mod(**inputs) 2025-12-04T09:47:02.9902046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9902121Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9902345Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9902420Z layer_outputs = layer_module( 2025-12-04T09:47:02.9902629Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9902700Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9902935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9902999Z return func(*args, **kwargs) 2025-12-04T09:47:02.9903225Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9903309Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9903540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9903610Z return func(*args, **kwargs) 2025-12-04T09:47:02.9903834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9903912Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9904150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9904215Z return func(*args, **kwargs) 2025-12-04T09:47:02.9904437Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:02.9904514Z key_states = self.k(current_states) 2025-12-04T09:47:02.9904518Z 2025-12-04T09:47:02.9904614Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9904802Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9904862Z res = mod(**inputs) 2025-12-04T09:47:02.9905086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9905163Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9905389Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9905462Z layer_outputs = layer_module( 2025-12-04T09:47:02.9905843Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9905965Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9906204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9906289Z return func(*args, **kwargs) 2025-12-04T09:47:02.9906523Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9906608Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9906835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9906926Z return func(*args, **kwargs) 2025-12-04T09:47:02.9907152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9907229Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9907501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9907565Z return func(*args, **kwargs) 2025-12-04T09:47:02.9907790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:02.9907871Z value_states = self.v(current_states) 2025-12-04T09:47:02.9907874Z 2025-12-04T09:47:02.9907946Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9908025Z cudagraph partition due to non gpu ops 2025-12-04T09:47:02.9908121Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9908305Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9908373Z res = mod(**inputs) 2025-12-04T09:47:02.9908599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9908676Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9908902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9908968Z layer_outputs = layer_module( 2025-12-04T09:47:02.9909187Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9909259Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9909489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9909561Z return func(*args, **kwargs) 2025-12-04T09:47:02.9909792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:02.9909874Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:02.9910101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9910167Z return func(*args, **kwargs) 2025-12-04T09:47:02.9910395Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:02.9910476Z attention_output = self.EncDecAttention( 2025-12-04T09:47:02.9910703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9910772Z return func(*args, **kwargs) 2025-12-04T09:47:02.9910995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:02.9911075Z attn_output = self.o(attn_output) 2025-12-04T09:47:02.9911078Z 2025-12-04T09:47:02.9911173Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9911356Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9911425Z res = mod(**inputs) 2025-12-04T09:47:02.9911702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9911779Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9912019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9912087Z layer_outputs = layer_module( 2025-12-04T09:47:02.9912306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9912393Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9912618Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9912687Z return func(*args, **kwargs) 2025-12-04T09:47:02.9912903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9913015Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9913341Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9913463Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9913702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:02.9913778Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:02.9913782Z 2025-12-04T09:47:02.9913891Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9914090Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9914151Z res = mod(**inputs) 2025-12-04T09:47:02.9914385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9914454Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9914681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9914757Z layer_outputs = layer_module( 2025-12-04T09:47:02.9914969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9915049Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9915280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9915348Z return func(*args, **kwargs) 2025-12-04T09:47:02.9915631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9915722Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9915958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9916082Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9916315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:02.9916400Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:02.9916403Z 2025-12-04T09:47:02.9916504Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9916690Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9916762Z res = mod(**inputs) 2025-12-04T09:47:02.9916995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:02.9917070Z decoder_outputs = self.decoder( 2025-12-04T09:47:02.9917301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:02.9917369Z layer_outputs = layer_module( 2025-12-04T09:47:02.9917610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:02.9917685Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:02.9917949Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:02.9918023Z return func(*args, **kwargs) 2025-12-04T09:47:02.9918250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:02.9918358Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:02.9918591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:02.9918703Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:02.9918962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:02.9919037Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:02.9919041Z 2025-12-04T09:47:02.9919147Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9919340Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9919403Z res = mod(**inputs) 2025-12-04T09:47:02.9919644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1793, in forward 2025-12-04T09:47:02.9919728Z lm_logits = self.lm_head(sequence_output) 2025-12-04T09:47:02.9919731Z 2025-12-04T09:47:02.9919827Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:02.9920022Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:02.9920083Z res = mod(**inputs) 2025-12-04T09:47:02.9920330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1800, in forward 2025-12-04T09:47:02.9920472Z loss = loss_fct(lm_logits.view(-1, lm_logits.size(-1)), labels.view(-1)) 2025-12-04T09:47:02.9920475Z 2025-12-04T09:47:12.2824746Z Compilation time (from dynamo_timed): 18.434725098 2025-12-04T09:47:12.3056802Z pass 2025-12-04T09:47:12.3061333Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:47:12.3063683Z TIMING: _recursive_pre_grad_passes:0.0463 _recursive_joint_graph_passes:0.56771 _recursive_post_grad_passes:0.05483 async_compile.wait:0.70624 code_gen:8.89125 inductor_compile:10.8733 backend_compile:15.65651 gc:0.00032 entire_frame_compile:18.43473 total_wall_time:18.43473 2025-12-04T09:47:12.3064796Z STATS: call_* op count: 810 | FakeTensorMode.__torch_dispatch__:21989 | FakeTensor.__torch_dispatch__:3647 | ProxyTorchDispatchMode.__torch_dispatch__:4640 2025-12-04T09:47:12.3066818Z Dynamo produced 1 graphs covering 810 ops with 0 graph breaks (0 unique) 2025-12-04T09:47:14.7049981Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:47:14.7050967Z import pynvml # type: ignore[import] 2025-12-04T09:47:17.7974122Z 2025-12-04T09:47:18.5998348Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:47:18.6000538Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:47:18.6012828Z cpu eval T5Small 2025-12-04T09:47:19.6136433Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:47:19.9469716Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:47:20.2915689Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:47:30.0648419Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0649058Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0649489Z res = mod(**inputs) 2025-12-04T09:47:30.0649907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.0650292Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.0654071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0654590Z layer_outputs = layer_module( 2025-12-04T09:47:30.0654946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0655316Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0655757Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0656118Z return func(*args, **kwargs) 2025-12-04T09:47:30.0656467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0656834Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0657205Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0657556Z return func(*args, **kwargs) 2025-12-04T09:47:30.0657896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0658265Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0658634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0658982Z return func(*args, **kwargs) 2025-12-04T09:47:30.0659318Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 549, in forward 2025-12-04T09:47:30.0659689Z position_bias = position_bias + causal_mask 2025-12-04T09:47:30.0659948Z 2025-12-04T09:47:30.0660055Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0660410Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0660726Z res = mod(**inputs) 2025-12-04T09:47:30.0661063Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0661417Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0661770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0662128Z layer_outputs = layer_module( 2025-12-04T09:47:30.0662460Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0662812Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0663190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0663559Z return func(*args, **kwargs) 2025-12-04T09:47:30.0663902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0664270Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0664647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0665011Z return func(*args, **kwargs) 2025-12-04T09:47:30.0665352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0665727Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0666131Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0666491Z return func(*args, **kwargs) 2025-12-04T09:47:30.0666900Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:30.0667268Z query_states = self.q(hidden_states) 2025-12-04T09:47:30.0667400Z 2025-12-04T09:47:30.0667509Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0667855Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0668195Z res = mod(**inputs) 2025-12-04T09:47:30.0668534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0668883Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0669232Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0669604Z layer_outputs = layer_module( 2025-12-04T09:47:30.0669932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0670272Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0670632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0670987Z return func(*args, **kwargs) 2025-12-04T09:47:30.0671315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0671684Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0672046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0672398Z return func(*args, **kwargs) 2025-12-04T09:47:30.0672727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0673087Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0673588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0673974Z return func(*args, **kwargs) 2025-12-04T09:47:30.0674372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:30.0674733Z key_states = self.k(current_states) 2025-12-04T09:47:30.0674858Z 2025-12-04T09:47:30.0674968Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0675301Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0675609Z res = mod(**inputs) 2025-12-04T09:47:30.0675936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0676294Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0676638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0676989Z layer_outputs = layer_module( 2025-12-04T09:47:30.0677317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0677651Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0678012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0678371Z return func(*args, **kwargs) 2025-12-04T09:47:30.0678704Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0679051Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0679435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0679797Z return func(*args, **kwargs) 2025-12-04T09:47:30.0680152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0680503Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0680866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0681220Z return func(*args, **kwargs) 2025-12-04T09:47:30.0682188Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:30.0682547Z value_states = self.v(current_states) 2025-12-04T09:47:30.0682681Z 2025-12-04T09:47:30.0682758Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.0682960Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.0683208Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0683554Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0683862Z res = mod(**inputs) 2025-12-04T09:47:30.0684186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0684544Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0684891Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0685244Z layer_outputs = layer_module( 2025-12-04T09:47:30.0685565Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0685911Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0686269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0686616Z return func(*args, **kwargs) 2025-12-04T09:47:30.0686955Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0687312Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0687677Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0688034Z return func(*args, **kwargs) 2025-12-04T09:47:30.0688369Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0688732Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0689091Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0689448Z return func(*args, **kwargs) 2025-12-04T09:47:30.0689791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:30.0690149Z attn_output = self.o(attn_output) 2025-12-04T09:47:30.0690270Z 2025-12-04T09:47:30.0690369Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0690715Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0691024Z res = mod(**inputs) 2025-12-04T09:47:30.0691352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0691701Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0692054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0692408Z layer_outputs = layer_module( 2025-12-04T09:47:30.0692733Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0693076Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0693457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0693811Z return func(*args, **kwargs) 2025-12-04T09:47:30.0694159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.0694534Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.0694903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.0695311Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.0695699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:30.0696059Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:30.0696207Z 2025-12-04T09:47:30.0696312Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0696647Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0696958Z res = mod(**inputs) 2025-12-04T09:47:30.0697292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0697653Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0697998Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0698358Z layer_outputs = layer_module( 2025-12-04T09:47:30.0698694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0699038Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0699396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0699757Z return func(*args, **kwargs) 2025-12-04T09:47:30.0700095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.0700462Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.0700830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.0701229Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.0701615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:30.0701977Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:30.0702111Z 2025-12-04T09:47:30.0702213Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0702565Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0702869Z res = mod(**inputs) 2025-12-04T09:47:30.0703199Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0703557Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0703916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0704276Z layer_outputs = layer_module( 2025-12-04T09:47:30.0704634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0704986Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0705343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0705913Z return func(*args, **kwargs) 2025-12-04T09:47:30.0706280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.0706708Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.0707077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.0707596Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.0707992Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:30.0708352Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:30.0708508Z 2025-12-04T09:47:30.0708610Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0708956Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0709270Z res = mod(**inputs) 2025-12-04T09:47:30.0709595Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0709988Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0710346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0710708Z layer_outputs = layer_module( 2025-12-04T09:47:30.0711039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0711399Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0711776Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0712145Z return func(*args, **kwargs) 2025-12-04T09:47:30.0712493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0712870Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0713320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0713697Z return func(*args, **kwargs) 2025-12-04T09:47:30.0714066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0714527Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0714900Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0715254Z return func(*args, **kwargs) 2025-12-04T09:47:30.0715597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:30.0715966Z query_states = self.q(hidden_states) 2025-12-04T09:47:30.0716097Z 2025-12-04T09:47:30.0716197Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0716542Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0716860Z res = mod(**inputs) 2025-12-04T09:47:30.0717199Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0717560Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0717933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0718297Z layer_outputs = layer_module( 2025-12-04T09:47:30.0718630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0718989Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0719362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0719724Z return func(*args, **kwargs) 2025-12-04T09:47:30.0720065Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0720480Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0720853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0721260Z return func(*args, **kwargs) 2025-12-04T09:47:30.0721616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0722001Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0722360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0722725Z return func(*args, **kwargs) 2025-12-04T09:47:30.0723062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:30.0723415Z key_states = self.k(current_states) 2025-12-04T09:47:30.0723559Z 2025-12-04T09:47:30.0723666Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0724002Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0724315Z res = mod(**inputs) 2025-12-04T09:47:30.0724643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0724994Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0725346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0725703Z layer_outputs = layer_module( 2025-12-04T09:47:30.0726040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0726379Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0726740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0727102Z return func(*args, **kwargs) 2025-12-04T09:47:30.0727436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0727793Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0728159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0728510Z return func(*args, **kwargs) 2025-12-04T09:47:30.0728840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0729201Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0729563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0729913Z return func(*args, **kwargs) 2025-12-04T09:47:30.0730242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:30.0730594Z value_states = self.v(current_states) 2025-12-04T09:47:30.0730720Z 2025-12-04T09:47:30.0730803Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.0731004Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.0731229Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0731568Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0731872Z res = mod(**inputs) 2025-12-04T09:47:30.0732195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0732548Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0732897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0733240Z layer_outputs = layer_module( 2025-12-04T09:47:30.0733591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0733940Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0734320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0734669Z return func(*args, **kwargs) 2025-12-04T09:47:30.0735011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0735388Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0735739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0736090Z return func(*args, **kwargs) 2025-12-04T09:47:30.0736423Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0736803Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0737164Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0737524Z return func(*args, **kwargs) 2025-12-04T09:47:30.0737858Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:30.0738206Z attn_output = self.o(attn_output) 2025-12-04T09:47:30.0738335Z 2025-12-04T09:47:30.0738434Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0738778Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0739080Z res = mod(**inputs) 2025-12-04T09:47:30.0739402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0739758Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0740108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0740458Z layer_outputs = layer_module( 2025-12-04T09:47:30.0740781Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0741125Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0741486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0741833Z return func(*args, **kwargs) 2025-12-04T09:47:30.0742171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.0742541Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.0742904Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.0743289Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.0743675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:30.0744035Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:30.0744159Z 2025-12-04T09:47:30.0744263Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0744593Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0744896Z res = mod(**inputs) 2025-12-04T09:47:30.0745220Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0745565Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0745909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0746258Z layer_outputs = layer_module( 2025-12-04T09:47:30.0746607Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0746948Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0747325Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0747675Z return func(*args, **kwargs) 2025-12-04T09:47:30.0747999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.0748388Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.0748753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.0749146Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.0749527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:30.0749916Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:30.0750043Z 2025-12-04T09:47:30.0750149Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0750480Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0750791Z res = mod(**inputs) 2025-12-04T09:47:30.0751121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0751475Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0751819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0752172Z layer_outputs = layer_module( 2025-12-04T09:47:30.0752503Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0752848Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0753204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0753629Z return func(*args, **kwargs) 2025-12-04T09:47:30.0753975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.0754345Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.0754717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.0755119Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.0755509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:30.0755862Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:30.0756001Z 2025-12-04T09:47:30.0756099Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0756444Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0756742Z res = mod(**inputs) 2025-12-04T09:47:30.0757075Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0757434Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0757786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0758134Z layer_outputs = layer_module( 2025-12-04T09:47:30.0758468Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0758812Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0759172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0759521Z return func(*args, **kwargs) 2025-12-04T09:47:30.0759883Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0760253Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0760636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0761002Z return func(*args, **kwargs) 2025-12-04T09:47:30.0761355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0761744Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0762110Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0762509Z return func(*args, **kwargs) 2025-12-04T09:47:30.0762896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:30.0763325Z query_states = self.q(hidden_states) 2025-12-04T09:47:30.0763476Z 2025-12-04T09:47:30.0763591Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0763987Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0764338Z res = mod(**inputs) 2025-12-04T09:47:30.0764697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0765103Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0765523Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0765927Z layer_outputs = layer_module( 2025-12-04T09:47:30.0766306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0766684Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0767099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0767477Z return func(*args, **kwargs) 2025-12-04T09:47:30.0767849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0768243Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0768644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0769008Z return func(*args, **kwargs) 2025-12-04T09:47:30.0769345Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0769704Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0770062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0770416Z return func(*args, **kwargs) 2025-12-04T09:47:30.0770768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:30.0771123Z key_states = self.k(current_states) 2025-12-04T09:47:30.0771247Z 2025-12-04T09:47:30.0771345Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0771682Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0771985Z res = mod(**inputs) 2025-12-04T09:47:30.0772306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0772664Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0773013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0773367Z layer_outputs = layer_module( 2025-12-04T09:47:30.0773707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0774055Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0774431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0774786Z return func(*args, **kwargs) 2025-12-04T09:47:30.0775118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0775494Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0775856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0776204Z return func(*args, **kwargs) 2025-12-04T09:47:30.0776539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0776918Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0777281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0777636Z return func(*args, **kwargs) 2025-12-04T09:47:30.0777970Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:30.0778323Z value_states = self.v(current_states) 2025-12-04T09:47:30.0778449Z 2025-12-04T09:47:30.0778522Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.0778725Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.0778947Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0779285Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0779582Z res = mod(**inputs) 2025-12-04T09:47:30.0780196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0780552Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0780892Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0781251Z layer_outputs = layer_module( 2025-12-04T09:47:30.0781580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0781929Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0782282Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0782636Z return func(*args, **kwargs) 2025-12-04T09:47:30.0782973Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0783336Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0783694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0784050Z return func(*args, **kwargs) 2025-12-04T09:47:30.0784385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0784737Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0785103Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0785457Z return func(*args, **kwargs) 2025-12-04T09:47:30.0785790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:30.0786136Z attn_output = self.o(attn_output) 2025-12-04T09:47:30.0786264Z 2025-12-04T09:47:30.0786360Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0786718Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0787019Z res = mod(**inputs) 2025-12-04T09:47:30.0787363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0787718Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0788064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0788411Z layer_outputs = layer_module( 2025-12-04T09:47:30.0788737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0789102Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0789454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0789836Z return func(*args, **kwargs) 2025-12-04T09:47:30.0790176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.0790548Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.0790908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.0791301Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.0791691Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:30.0792051Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:30.0792177Z 2025-12-04T09:47:30.0792274Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0792615Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0792921Z res = mod(**inputs) 2025-12-04T09:47:30.0793312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0793699Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0794051Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0794408Z layer_outputs = layer_module( 2025-12-04T09:47:30.0794731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0795084Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0795437Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0795779Z return func(*args, **kwargs) 2025-12-04T09:47:30.0796102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.0796466Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.0796833Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.0797217Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.0797608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:30.0797976Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:30.0798108Z 2025-12-04T09:47:30.0798216Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0798559Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0798878Z res = mod(**inputs) 2025-12-04T09:47:30.0799229Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0799601Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0800003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0800396Z layer_outputs = layer_module( 2025-12-04T09:47:30.0800783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0801164Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0801566Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0801961Z return func(*args, **kwargs) 2025-12-04T09:47:30.0802344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.0802748Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.0803150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.0803591Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.0804003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:30.0804391Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:30.0804537Z 2025-12-04T09:47:30.0804637Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0804979Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0805283Z res = mod(**inputs) 2025-12-04T09:47:30.0805615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0806125Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0806482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0806855Z layer_outputs = layer_module( 2025-12-04T09:47:30.0807189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0807532Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0807886Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0808242Z return func(*args, **kwargs) 2025-12-04T09:47:30.0808583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0808943Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0809302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0809657Z return func(*args, **kwargs) 2025-12-04T09:47:30.0809989Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0810341Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0810705Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0811055Z return func(*args, **kwargs) 2025-12-04T09:47:30.0811387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:30.0811742Z query_states = self.q(hidden_states) 2025-12-04T09:47:30.0811873Z 2025-12-04T09:47:30.0811970Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0812309Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0812607Z res = mod(**inputs) 2025-12-04T09:47:30.0812935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0813291Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0813684Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0814029Z layer_outputs = layer_module( 2025-12-04T09:47:30.0814386Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0814732Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0815082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0815436Z return func(*args, **kwargs) 2025-12-04T09:47:30.0815819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0816172Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0816534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0816917Z return func(*args, **kwargs) 2025-12-04T09:47:30.0817261Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0817625Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0817983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0818337Z return func(*args, **kwargs) 2025-12-04T09:47:30.0818669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:30.0819022Z key_states = self.k(current_states) 2025-12-04T09:47:30.0819153Z 2025-12-04T09:47:30.0819249Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0819586Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0819889Z res = mod(**inputs) 2025-12-04T09:47:30.0820211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0820566Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0820914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0821262Z layer_outputs = layer_module( 2025-12-04T09:47:30.0821590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0821940Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0822301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0822650Z return func(*args, **kwargs) 2025-12-04T09:47:30.0822983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0823344Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0823700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0824053Z return func(*args, **kwargs) 2025-12-04T09:47:30.0824386Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0824749Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0825106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0825465Z return func(*args, **kwargs) 2025-12-04T09:47:30.0825804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:30.0826160Z value_states = self.v(current_states) 2025-12-04T09:47:30.0826286Z 2025-12-04T09:47:30.0826360Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.0826564Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.0826810Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0827144Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0827471Z res = mod(**inputs) 2025-12-04T09:47:30.0827800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0828152Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0828488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0828856Z layer_outputs = layer_module( 2025-12-04T09:47:30.0829182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0829518Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0829977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0830343Z return func(*args, **kwargs) 2025-12-04T09:47:30.0830683Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0831035Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0831403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0831761Z return func(*args, **kwargs) 2025-12-04T09:47:30.0832094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0832460Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0832829Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0833183Z return func(*args, **kwargs) 2025-12-04T09:47:30.0833595Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:30.0833972Z attn_output = self.o(attn_output) 2025-12-04T09:47:30.0834100Z 2025-12-04T09:47:30.0834209Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0834565Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0834889Z res = mod(**inputs) 2025-12-04T09:47:30.0835235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0835610Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0835968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0836336Z layer_outputs = layer_module( 2025-12-04T09:47:30.0836672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0837025Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0837387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0837750Z return func(*args, **kwargs) 2025-12-04T09:47:30.0838101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0838464Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0838845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0839218Z return func(*args, **kwargs) 2025-12-04T09:47:30.0839562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 613, in forward 2025-12-04T09:47:30.0839978Z hidden_states = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:47:30.0840167Z 2025-12-04T09:47:30.0840292Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0844300Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0844656Z res = mod(**inputs) 2025-12-04T09:47:30.0844994Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0845359Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0845715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0846105Z layer_outputs = layer_module( 2025-12-04T09:47:30.0846435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0846790Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0847187Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0847543Z return func(*args, **kwargs) 2025-12-04T09:47:30.0847895Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.0848280Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.0848648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.0849030Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.0849424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:30.0849778Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:30.0849905Z 2025-12-04T09:47:30.0850009Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0850343Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0850648Z res = mod(**inputs) 2025-12-04T09:47:30.0850975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0851322Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0851669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0852018Z layer_outputs = layer_module( 2025-12-04T09:47:30.0852350Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0852688Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0853048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0853399Z return func(*args, **kwargs) 2025-12-04T09:47:30.0853730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.0854098Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.0854465Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.0854854Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.0855233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:30.0855590Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:30.0855719Z 2025-12-04T09:47:30.0855824Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0856159Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0856457Z res = mod(**inputs) 2025-12-04T09:47:30.0856783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0857166Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0857508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0857880Z layer_outputs = layer_module( 2025-12-04T09:47:30.0858202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0858534Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0858875Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0859245Z return func(*args, **kwargs) 2025-12-04T09:47:30.0859577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.0859953Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.0860337Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.0860729Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.0861110Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:30.0861460Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:30.0861592Z 2025-12-04T09:47:30.0861690Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0862027Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0862329Z res = mod(**inputs) 2025-12-04T09:47:30.0862655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0863005Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0863355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0863703Z layer_outputs = layer_module( 2025-12-04T09:47:30.0864036Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0864379Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0864743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0865090Z return func(*args, **kwargs) 2025-12-04T09:47:30.0865426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0865785Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0866139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0866495Z return func(*args, **kwargs) 2025-12-04T09:47:30.0866832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0867194Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0867555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0867911Z return func(*args, **kwargs) 2025-12-04T09:47:30.0868247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:30.0868598Z query_states = self.q(hidden_states) 2025-12-04T09:47:30.0868734Z 2025-12-04T09:47:30.0868832Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0869170Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0869473Z res = mod(**inputs) 2025-12-04T09:47:30.0869794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0870183Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0870553Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0870906Z layer_outputs = layer_module( 2025-12-04T09:47:30.0871230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0871576Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0871936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0872305Z return func(*args, **kwargs) 2025-12-04T09:47:30.0872646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0873002Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0873487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0873852Z return func(*args, **kwargs) 2025-12-04T09:47:30.0874207Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0874600Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0874952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0875308Z return func(*args, **kwargs) 2025-12-04T09:47:30.0875692Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:30.0876061Z key_states = self.k(current_states) 2025-12-04T09:47:30.0876189Z 2025-12-04T09:47:30.0876290Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0876639Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0876956Z res = mod(**inputs) 2025-12-04T09:47:30.0877282Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0877647Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0878000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0878357Z layer_outputs = layer_module( 2025-12-04T09:47:30.0878683Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0879037Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0879402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0879763Z return func(*args, **kwargs) 2025-12-04T09:47:30.0880104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0880470Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0880842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0881199Z return func(*args, **kwargs) 2025-12-04T09:47:30.0881545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0881919Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0882296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0882652Z return func(*args, **kwargs) 2025-12-04T09:47:30.0883001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:30.0883370Z value_states = self.v(current_states) 2025-12-04T09:47:30.0883500Z 2025-12-04T09:47:30.0883603Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.0883812Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.0884041Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0884412Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0884723Z res = mod(**inputs) 2025-12-04T09:47:30.0885054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0885441Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0885792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0886152Z layer_outputs = layer_module( 2025-12-04T09:47:30.0886490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0886869Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0887233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0887602Z return func(*args, **kwargs) 2025-12-04T09:47:30.0887953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0888302Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0888663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0889017Z return func(*args, **kwargs) 2025-12-04T09:47:30.0889350Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0889702Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0890071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0890426Z return func(*args, **kwargs) 2025-12-04T09:47:30.0890762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:30.0891108Z attn_output = self.o(attn_output) 2025-12-04T09:47:30.0891239Z 2025-12-04T09:47:30.0891333Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0891669Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0891968Z res = mod(**inputs) 2025-12-04T09:47:30.0892296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0892651Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0892994Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0893338Z layer_outputs = layer_module( 2025-12-04T09:47:30.0893667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0894014Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0894366Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0894719Z return func(*args, **kwargs) 2025-12-04T09:47:30.0895054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.0895429Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.0895789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.0896181Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.0896591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:30.0896958Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:30.0897085Z 2025-12-04T09:47:30.0897208Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0897540Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0897845Z res = mod(**inputs) 2025-12-04T09:47:30.0898165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0898535Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0898877Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0899229Z layer_outputs = layer_module( 2025-12-04T09:47:30.0899557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0899916Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0900279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0900634Z return func(*args, **kwargs) 2025-12-04T09:47:30.0900972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.0901340Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.0901706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.0902102Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.0902482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:30.0902842Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:30.0902977Z 2025-12-04T09:47:30.0903075Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0903414Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0903715Z res = mod(**inputs) 2025-12-04T09:47:30.0904040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0904397Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0904740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0905086Z layer_outputs = layer_module( 2025-12-04T09:47:30.0905412Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0905904Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0906264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0906625Z return func(*args, **kwargs) 2025-12-04T09:47:30.0906967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.0907342Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.0907702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.0908101Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.0908494Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:30.0908844Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:30.0908976Z 2025-12-04T09:47:30.0909074Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0909411Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0909721Z res = mod(**inputs) 2025-12-04T09:47:30.0910093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0910481Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0910835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0911194Z layer_outputs = layer_module( 2025-12-04T09:47:30.0911518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0911894Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0912257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0912604Z return func(*args, **kwargs) 2025-12-04T09:47:30.0912983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0913420Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0913797Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0914153Z return func(*args, **kwargs) 2025-12-04T09:47:30.0914506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0914876Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0915243Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0915603Z return func(*args, **kwargs) 2025-12-04T09:47:30.0915941Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:30.0916305Z query_states = self.q(hidden_states) 2025-12-04T09:47:30.0916435Z 2025-12-04T09:47:30.0916536Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0916884Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0917201Z res = mod(**inputs) 2025-12-04T09:47:30.0917531Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0917896Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0918256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0918621Z layer_outputs = layer_module( 2025-12-04T09:47:30.0918952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0919310Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0919679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0920045Z return func(*args, **kwargs) 2025-12-04T09:47:30.0920384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0920755Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0921130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0921486Z return func(*args, **kwargs) 2025-12-04T09:47:30.0921834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0922206Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0922584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0922940Z return func(*args, **kwargs) 2025-12-04T09:47:30.0923314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:30.0923687Z key_states = self.k(current_states) 2025-12-04T09:47:30.0923812Z 2025-12-04T09:47:30.0923927Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0924282Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0924598Z res = mod(**inputs) 2025-12-04T09:47:30.0924937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0925305Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0925657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0926014Z layer_outputs = layer_module( 2025-12-04T09:47:30.0926344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0926711Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0927087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0927459Z return func(*args, **kwargs) 2025-12-04T09:47:30.0927809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0928167Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0928534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0928890Z return func(*args, **kwargs) 2025-12-04T09:47:30.0929219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0929580Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0929946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0930291Z return func(*args, **kwargs) 2025-12-04T09:47:30.0930630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:30.0930988Z value_states = self.v(current_states) 2025-12-04T09:47:30.0931113Z 2025-12-04T09:47:30.0931196Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.0931388Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.0931610Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0931952Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0932249Z res = mod(**inputs) 2025-12-04T09:47:30.0932580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.0932936Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.0933283Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0933626Z layer_outputs = layer_module( 2025-12-04T09:47:30.0933956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0934298Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0934648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0935002Z return func(*args, **kwargs) 2025-12-04T09:47:30.0935338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0935696Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0936050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0936402Z return func(*args, **kwargs) 2025-12-04T09:47:30.0936753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0937133Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0937492Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0937838Z return func(*args, **kwargs) 2025-12-04T09:47:30.0938170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:30.0938539Z query_states = self.q(hidden_states) 2025-12-04T09:47:30.0938669Z 2025-12-04T09:47:30.0938765Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0939105Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0939431Z res = mod(**inputs) 2025-12-04T09:47:30.0939751Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.0940105Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.0940456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0940801Z layer_outputs = layer_module( 2025-12-04T09:47:30.0941132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0941479Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0941841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0942190Z return func(*args, **kwargs) 2025-12-04T09:47:30.0942526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0942889Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0943247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0943602Z return func(*args, **kwargs) 2025-12-04T09:47:30.0943939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0944298Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0944655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0945006Z return func(*args, **kwargs) 2025-12-04T09:47:30.0945343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:30.0945697Z key_states = self.k(current_states) 2025-12-04T09:47:30.0945822Z 2025-12-04T09:47:30.0945918Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0946259Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0946570Z res = mod(**inputs) 2025-12-04T09:47:30.0946893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.0947250Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.0947595Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0947944Z layer_outputs = layer_module( 2025-12-04T09:47:30.0948276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0948619Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0948986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0949335Z return func(*args, **kwargs) 2025-12-04T09:47:30.0949701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0950061Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0950441Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0950782Z return func(*args, **kwargs) 2025-12-04T09:47:30.0951115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0951492Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0951854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0952202Z return func(*args, **kwargs) 2025-12-04T09:47:30.0952535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:30.0952911Z value_states = self.v(current_states) 2025-12-04T09:47:30.0953035Z 2025-12-04T09:47:30.0953109Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.0953386Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.0953616Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0953956Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0954258Z res = mod(**inputs) 2025-12-04T09:47:30.0954589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.0954948Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.0955295Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0955648Z layer_outputs = layer_module( 2025-12-04T09:47:30.0955986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0956333Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0956688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0957039Z return func(*args, **kwargs) 2025-12-04T09:47:30.0957376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0957722Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0958089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0958440Z return func(*args, **kwargs) 2025-12-04T09:47:30.0958777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0959136Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0959503Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0959857Z return func(*args, **kwargs) 2025-12-04T09:47:30.0960187Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:30.0960541Z attn_output = self.o(attn_output) 2025-12-04T09:47:30.0960670Z 2025-12-04T09:47:30.0960768Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0961105Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0961413Z res = mod(**inputs) 2025-12-04T09:47:30.0961737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.0962089Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.0962465Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0962813Z layer_outputs = layer_module( 2025-12-04T09:47:30.0963160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0963510Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0963861Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0964214Z return func(*args, **kwargs) 2025-12-04T09:47:30.0964575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.0964924Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.0965269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0965640Z return func(*args, **kwargs) 2025-12-04T09:47:30.0965978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.0966330Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.0966702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0967063Z return func(*args, **kwargs) 2025-12-04T09:47:30.0967400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:30.0967750Z query_states = self.q(hidden_states) 2025-12-04T09:47:30.0967880Z 2025-12-04T09:47:30.0967977Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0968317Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0968623Z res = mod(**inputs) 2025-12-04T09:47:30.0968944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0969298Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0969650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0969990Z layer_outputs = layer_module( 2025-12-04T09:47:30.0970325Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0970667Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0971029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0971375Z return func(*args, **kwargs) 2025-12-04T09:47:30.0971711Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0972071Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0972425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0972775Z return func(*args, **kwargs) 2025-12-04T09:47:30.0973108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.0973467Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.0973821Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0974178Z return func(*args, **kwargs) 2025-12-04T09:47:30.0974513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:30.0974860Z attn_output = self.o(attn_output) 2025-12-04T09:47:30.0974990Z 2025-12-04T09:47:30.0975086Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0975452Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0975758Z res = mod(**inputs) 2025-12-04T09:47:30.0976095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0976455Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0976803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0977154Z layer_outputs = layer_module( 2025-12-04T09:47:30.0977475Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0977837Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0978197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0978561Z return func(*args, **kwargs) 2025-12-04T09:47:30.0978899Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.0979252Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.0979615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0979960Z return func(*args, **kwargs) 2025-12-04T09:47:30.0980301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 613, in forward 2025-12-04T09:47:30.0980707Z hidden_states = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:47:30.0980884Z 2025-12-04T09:47:30.0980980Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0981319Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0981625Z res = mod(**inputs) 2025-12-04T09:47:30.0981951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0982303Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0982650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0983006Z layer_outputs = layer_module( 2025-12-04T09:47:30.0983344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0983683Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0984045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0984395Z return func(*args, **kwargs) 2025-12-04T09:47:30.0984731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.0985103Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.0985469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.0985861Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.0986244Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:30.0986600Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:30.0986723Z 2025-12-04T09:47:30.0986826Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0987156Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0987463Z res = mod(**inputs) 2025-12-04T09:47:30.0987792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0988147Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0988506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0988861Z layer_outputs = layer_module( 2025-12-04T09:47:30.0989212Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0989567Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0989929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0990282Z return func(*args, **kwargs) 2025-12-04T09:47:30.0990651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.0991011Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.0991376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.0991797Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.0992186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:30.0992536Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:30.0992674Z 2025-12-04T09:47:30.0992772Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0993110Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0993489Z res = mod(**inputs) 2025-12-04T09:47:30.0993836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:47:30.0994208Z encoder_outputs = self.encoder( 2025-12-04T09:47:30.0994577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.0994929Z layer_outputs = layer_module( 2025-12-04T09:47:30.0995264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.0995611Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.0995977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.0996324Z return func(*args, **kwargs) 2025-12-04T09:47:30.0996662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.0997028Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.0997390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.0997783Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.0998168Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:30.0998523Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:30.0998650Z 2025-12-04T09:47:30.0998748Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.0999086Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.0999389Z res = mod(**inputs) 2025-12-04T09:47:30.0999708Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1000063Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1000408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1000756Z layer_outputs = layer_module( 2025-12-04T09:47:30.1001077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1001422Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1001806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1002155Z return func(*args, **kwargs) 2025-12-04T09:47:30.1002504Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1002868Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1003227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1003596Z return func(*args, **kwargs) 2025-12-04T09:47:30.1003936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.1004300Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.1004665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1005033Z return func(*args, **kwargs) 2025-12-04T09:47:30.1005374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:30.1005834Z key_states = self.k(current_states) 2025-12-04T09:47:30.1005969Z 2025-12-04T09:47:30.1006069Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1006412Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1006719Z res = mod(**inputs) 2025-12-04T09:47:30.1007054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1007410Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1007762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1008120Z layer_outputs = layer_module( 2025-12-04T09:47:30.1008449Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1008798Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1009166Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1009517Z return func(*args, **kwargs) 2025-12-04T09:47:30.1009851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1010210Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1010577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1010932Z return func(*args, **kwargs) 2025-12-04T09:47:30.1011264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.1011629Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.1011993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1012342Z return func(*args, **kwargs) 2025-12-04T09:47:30.1012682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:30.1013041Z value_states = self.v(current_states) 2025-12-04T09:47:30.1013170Z 2025-12-04T09:47:30.1013253Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.1013449Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.1013673Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1014009Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1014307Z res = mod(**inputs) 2025-12-04T09:47:30.1014642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1015042Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1015387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1015759Z layer_outputs = layer_module( 2025-12-04T09:47:30.1016082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1016419Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1016761Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1017147Z return func(*args, **kwargs) 2025-12-04T09:47:30.1017486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1017862Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1018251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1018603Z return func(*args, **kwargs) 2025-12-04T09:47:30.1018942Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.1019301Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.1019661Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1020021Z return func(*args, **kwargs) 2025-12-04T09:47:30.1020361Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:30.1020715Z attn_output = self.o(attn_output) 2025-12-04T09:47:30.1020845Z 2025-12-04T09:47:30.1020944Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1021291Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1021603Z res = mod(**inputs) 2025-12-04T09:47:30.1021930Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1022289Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1022641Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1022984Z layer_outputs = layer_module( 2025-12-04T09:47:30.1023320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1023670Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1024031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1024379Z return func(*args, **kwargs) 2025-12-04T09:47:30.1024717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.1025093Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.1025455Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.1025861Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.1026252Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:30.1026608Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:30.1026734Z 2025-12-04T09:47:30.1026832Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1027173Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1027481Z res = mod(**inputs) 2025-12-04T09:47:30.1027807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1028180Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1028529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1028897Z layer_outputs = layer_module( 2025-12-04T09:47:30.1029215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1029556Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1029909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1030281Z return func(*args, **kwargs) 2025-12-04T09:47:30.1030614Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.1030988Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.1031374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.1031755Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.1032144Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:30.1032500Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:30.1032629Z 2025-12-04T09:47:30.1032732Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1033060Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1033429Z res = mod(**inputs) 2025-12-04T09:47:30.1033765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1034129Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1034529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1034888Z layer_outputs = layer_module( 2025-12-04T09:47:30.1035223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1035565Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1035929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1036284Z return func(*args, **kwargs) 2025-12-04T09:47:30.1036628Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.1036997Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.1037371Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.1037764Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.1038145Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:30.1038508Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:30.1038641Z 2025-12-04T09:47:30.1038739Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1039085Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1039385Z res = mod(**inputs) 2025-12-04T09:47:30.1039716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1040071Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1040423Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1040768Z layer_outputs = layer_module( 2025-12-04T09:47:30.1041123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1041475Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1041867Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1042228Z return func(*args, **kwargs) 2025-12-04T09:47:30.1042568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.1042928Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.1043310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1043667Z return func(*args, **kwargs) 2025-12-04T09:47:30.1044010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.1044384Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.1044746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1045100Z return func(*args, **kwargs) 2025-12-04T09:47:30.1045443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:30.1045793Z query_states = self.q(hidden_states) 2025-12-04T09:47:30.1045926Z 2025-12-04T09:47:30.1046023Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1046362Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1046670Z res = mod(**inputs) 2025-12-04T09:47:30.1046997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1047350Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1047703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1048050Z layer_outputs = layer_module( 2025-12-04T09:47:30.1048379Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1063800Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1064214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1064589Z return func(*args, **kwargs) 2025-12-04T09:47:30.1064952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.1065326Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.1065693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1066047Z return func(*args, **kwargs) 2025-12-04T09:47:30.1066391Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.1066753Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.1067121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1067466Z return func(*args, **kwargs) 2025-12-04T09:47:30.1067806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:30.1068166Z key_states = self.k(current_states) 2025-12-04T09:47:30.1068294Z 2025-12-04T09:47:30.1068399Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1068752Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1069063Z res = mod(**inputs) 2025-12-04T09:47:30.1069401Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1069823Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1070210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1070574Z layer_outputs = layer_module( 2025-12-04T09:47:30.1070904Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1071261Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1071627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1072011Z return func(*args, **kwargs) 2025-12-04T09:47:30.1072346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.1072731Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.1073098Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1073585Z return func(*args, **kwargs) 2025-12-04T09:47:30.1073929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.1074311Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.1074671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1075014Z return func(*args, **kwargs) 2025-12-04T09:47:30.1075352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:30.1075708Z value_states = self.v(current_states) 2025-12-04T09:47:30.1075835Z 2025-12-04T09:47:30.1075922Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.1076119Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.1076348Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1076690Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1076989Z res = mod(**inputs) 2025-12-04T09:47:30.1077324Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1077676Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1078023Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1078366Z layer_outputs = layer_module( 2025-12-04T09:47:30.1078698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1079041Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1079388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1079739Z return func(*args, **kwargs) 2025-12-04T09:47:30.1080072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.1080426Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.1080774Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1081123Z return func(*args, **kwargs) 2025-12-04T09:47:30.1081457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.1081807Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.1082167Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1082515Z return func(*args, **kwargs) 2025-12-04T09:47:30.1082869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:30.1083217Z attn_output = self.o(attn_output) 2025-12-04T09:47:30.1083352Z 2025-12-04T09:47:30.1083472Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1083814Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1084122Z res = mod(**inputs) 2025-12-04T09:47:30.1084445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1084821Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1085172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1085514Z layer_outputs = layer_module( 2025-12-04T09:47:30.1085846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1086215Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1086584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1086943Z return func(*args, **kwargs) 2025-12-04T09:47:30.1087289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1087659Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1088030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1088389Z return func(*args, **kwargs) 2025-12-04T09:47:30.1088730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.1089101Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.1089470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1089829Z return func(*args, **kwargs) 2025-12-04T09:47:30.1090175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:30.1090535Z query_states = self.q(hidden_states) 2025-12-04T09:47:30.1090663Z 2025-12-04T09:47:30.1090766Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1091113Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1091431Z res = mod(**inputs) 2025-12-04T09:47:30.1091763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1092128Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1092483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1092846Z layer_outputs = layer_module( 2025-12-04T09:47:30.1093184Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1093543Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1093916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1094267Z return func(*args, **kwargs) 2025-12-04T09:47:30.1094611Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1094981Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1095353Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1095706Z return func(*args, **kwargs) 2025-12-04T09:47:30.1096070Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.1096434Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.1096817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1097163Z return func(*args, **kwargs) 2025-12-04T09:47:30.1097499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:30.1097853Z key_states = self.k(current_states) 2025-12-04T09:47:30.1097996Z 2025-12-04T09:47:30.1098093Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1098434Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1098739Z res = mod(**inputs) 2025-12-04T09:47:30.1099068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1099431Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1099781Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1100133Z layer_outputs = layer_module( 2025-12-04T09:47:30.1100452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1100797Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1101154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1101504Z return func(*args, **kwargs) 2025-12-04T09:47:30.1101835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1102192Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1102551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1102905Z return func(*args, **kwargs) 2025-12-04T09:47:30.1103232Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.1103599Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.1103960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1104304Z return func(*args, **kwargs) 2025-12-04T09:47:30.1104639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:30.1104999Z value_states = self.v(current_states) 2025-12-04T09:47:30.1105126Z 2025-12-04T09:47:30.1105208Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.1105404Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.1105631Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1106145Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1106452Z res = mod(**inputs) 2025-12-04T09:47:30.1106791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1107151Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1107505Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1107854Z layer_outputs = layer_module( 2025-12-04T09:47:30.1108194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1108544Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1108898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1109259Z return func(*args, **kwargs) 2025-12-04T09:47:30.1109645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1110010Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1110391Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1110754Z return func(*args, **kwargs) 2025-12-04T09:47:30.1111093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.1111484Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.1111847Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1112199Z return func(*args, **kwargs) 2025-12-04T09:47:30.1112541Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:30.1112916Z attn_output = self.o(attn_output) 2025-12-04T09:47:30.1113050Z 2025-12-04T09:47:30.1113148Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1113559Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1113873Z res = mod(**inputs) 2025-12-04T09:47:30.1114197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1114553Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1114905Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1115249Z layer_outputs = layer_module( 2025-12-04T09:47:30.1115581Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1115929Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1116291Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1116637Z return func(*args, **kwargs) 2025-12-04T09:47:30.1116980Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.1117354Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.1117714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.1118113Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.1118507Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:30.1118867Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:30.1118997Z 2025-12-04T09:47:30.1119094Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1119432Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1119737Z res = mod(**inputs) 2025-12-04T09:47:30.1120064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1120416Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1120765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1121120Z layer_outputs = layer_module( 2025-12-04T09:47:30.1121443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1121791Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1122152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1122511Z return func(*args, **kwargs) 2025-12-04T09:47:30.1122862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.1123237Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.1123618Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.1124008Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.1124400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:30.1124776Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:30.1124903Z 2025-12-04T09:47:30.1125009Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1125339Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1125661Z res = mod(**inputs) 2025-12-04T09:47:30.1125989Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1126347Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1126690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1127043Z layer_outputs = layer_module( 2025-12-04T09:47:30.1127373Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1127711Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1128069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1128423Z return func(*args, **kwargs) 2025-12-04T09:47:30.1128762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.1129127Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.1129490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.1129885Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.1130263Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:30.1130617Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:30.1130749Z 2025-12-04T09:47:30.1130849Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1131194Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1131493Z res = mod(**inputs) 2025-12-04T09:47:30.1131836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1132193Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1132539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1132885Z layer_outputs = layer_module( 2025-12-04T09:47:30.1133219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1133563Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1133921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1134271Z return func(*args, **kwargs) 2025-12-04T09:47:30.1134612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.1134970Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.1135330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1135696Z return func(*args, **kwargs) 2025-12-04T09:47:30.1136035Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.1136410Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.1136768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1137120Z return func(*args, **kwargs) 2025-12-04T09:47:30.1137456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:30.1137825Z query_states = self.q(hidden_states) 2025-12-04T09:47:30.1137949Z 2025-12-04T09:47:30.1138046Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1138387Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1138712Z res = mod(**inputs) 2025-12-04T09:47:30.1139039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1139401Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1139751Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1140103Z layer_outputs = layer_module( 2025-12-04T09:47:30.1140427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1140774Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1141137Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1141492Z return func(*args, **kwargs) 2025-12-04T09:47:30.1141827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.1142189Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.1142558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1142907Z return func(*args, **kwargs) 2025-12-04T09:47:30.1143249Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.1143609Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.1143971Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1144321Z return func(*args, **kwargs) 2025-12-04T09:47:30.1144657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:30.1145013Z key_states = self.k(current_states) 2025-12-04T09:47:30.1145138Z 2025-12-04T09:47:30.1145234Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1145576Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1145882Z res = mod(**inputs) 2025-12-04T09:47:30.1146215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1146563Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1146911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1147274Z layer_outputs = layer_module( 2025-12-04T09:47:30.1147602Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1147949Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1148310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1148668Z return func(*args, **kwargs) 2025-12-04T09:47:30.1149012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.1149389Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.1149744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1150087Z return func(*args, **kwargs) 2025-12-04T09:47:30.1150409Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.1150790Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.1151150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1151505Z return func(*args, **kwargs) 2025-12-04T09:47:30.1151849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:30.1152194Z value_states = self.v(current_states) 2025-12-04T09:47:30.1152317Z 2025-12-04T09:47:30.1152399Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.1152593Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.1152813Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1153149Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1153530Z res = mod(**inputs) 2025-12-04T09:47:30.1153862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1154224Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1154574Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1154927Z layer_outputs = layer_module( 2025-12-04T09:47:30.1155258Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1155607Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1155964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1156324Z return func(*args, **kwargs) 2025-12-04T09:47:30.1156665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.1157021Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.1157380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1157733Z return func(*args, **kwargs) 2025-12-04T09:47:30.1158068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.1158426Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.1158785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1159137Z return func(*args, **kwargs) 2025-12-04T09:47:30.1159478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:30.1159827Z attn_output = self.o(attn_output) 2025-12-04T09:47:30.1159959Z 2025-12-04T09:47:30.1160057Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1160401Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1160709Z res = mod(**inputs) 2025-12-04T09:47:30.1161034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1161389Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1161761Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1162109Z layer_outputs = layer_module( 2025-12-04T09:47:30.1162460Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1162810Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1163172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1163516Z return func(*args, **kwargs) 2025-12-04T09:47:30.1163870Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.1164227Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.1164579Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1164951Z return func(*args, **kwargs) 2025-12-04T09:47:30.1165288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 613, in forward 2025-12-04T09:47:30.1165694Z hidden_states = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:47:30.1165872Z 2025-12-04T09:47:30.1165970Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1166314Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1166619Z res = mod(**inputs) 2025-12-04T09:47:30.1166946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1167299Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1167648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1168009Z layer_outputs = layer_module( 2025-12-04T09:47:30.1168334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1168681Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1169043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1169397Z return func(*args, **kwargs) 2025-12-04T09:47:30.1169729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1170090Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1170452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1170799Z return func(*args, **kwargs) 2025-12-04T09:47:30.1171137Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.1171500Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.1171865Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1172211Z return func(*args, **kwargs) 2025-12-04T09:47:30.1172547Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:30.1172902Z query_states = self.q(hidden_states) 2025-12-04T09:47:30.1173026Z 2025-12-04T09:47:30.1173129Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1173466Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1173770Z res = mod(**inputs) 2025-12-04T09:47:30.1174094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1174439Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1174807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1175162Z layer_outputs = layer_module( 2025-12-04T09:47:30.1175509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1175847Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1176204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1176557Z return func(*args, **kwargs) 2025-12-04T09:47:30.1176910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1177269Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1177638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1178007Z return func(*args, **kwargs) 2025-12-04T09:47:30.1178333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.1178694Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.1179055Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1179401Z return func(*args, **kwargs) 2025-12-04T09:47:30.1179735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:30.1180090Z key_states = self.k(current_states) 2025-12-04T09:47:30.1180214Z 2025-12-04T09:47:30.1180316Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1180649Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1180951Z res = mod(**inputs) 2025-12-04T09:47:30.1181281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1181628Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1181975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1182326Z layer_outputs = layer_module( 2025-12-04T09:47:30.1182650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1182985Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1183342Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1183693Z return func(*args, **kwargs) 2025-12-04T09:47:30.1184028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1184381Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1184740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1185090Z return func(*args, **kwargs) 2025-12-04T09:47:30.1185416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.1185775Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.1186134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1186482Z return func(*args, **kwargs) 2025-12-04T09:47:30.1186807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:30.1187162Z value_states = self.v(current_states) 2025-12-04T09:47:30.1187285Z 2025-12-04T09:47:30.1187366Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.1187557Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.1187795Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1188144Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1188476Z res = mod(**inputs) 2025-12-04T09:47:30.1188790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1189139Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1189479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1189845Z layer_outputs = layer_module( 2025-12-04T09:47:30.1190180Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1190524Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1190903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1191249Z return func(*args, **kwargs) 2025-12-04T09:47:30.1191582Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1191944Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1192304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1192650Z return func(*args, **kwargs) 2025-12-04T09:47:30.1192990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.1193446Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.1193830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1194225Z return func(*args, **kwargs) 2025-12-04T09:47:30.1194571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:30.1194942Z attn_output = self.o(attn_output) 2025-12-04T09:47:30.1195063Z 2025-12-04T09:47:30.1195162Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1195504Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1195815Z res = mod(**inputs) 2025-12-04T09:47:30.1196154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1196532Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1196901Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1197275Z layer_outputs = layer_module( 2025-12-04T09:47:30.1197616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1197982Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1198359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1198727Z return func(*args, **kwargs) 2025-12-04T09:47:30.1199077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.1199467Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.1199848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.1199961Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.1200195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:30.1200280Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:30.1200283Z 2025-12-04T09:47:30.1200402Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1200605Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1200686Z res = mod(**inputs) 2025-12-04T09:47:30.1200927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1201005Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1201238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1201325Z layer_outputs = layer_module( 2025-12-04T09:47:30.1201551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1201627Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1201893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1201959Z return func(*args, **kwargs) 2025-12-04T09:47:30.1202194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.1202290Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.1202524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.1202641Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.1202881Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:30.1202957Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:30.1202961Z 2025-12-04T09:47:30.1203068Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1203272Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1203338Z res = mod(**inputs) 2025-12-04T09:47:30.1203590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1203663Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1203906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1203971Z layer_outputs = layer_module( 2025-12-04T09:47:30.1204183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1204265Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1204491Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1204555Z return func(*args, **kwargs) 2025-12-04T09:47:30.1204787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.1204871Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.1205099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.1205207Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.1205428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:30.1205508Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:30.1205512Z 2025-12-04T09:47:30.1205606Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1205927Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1205994Z res = mod(**inputs) 2025-12-04T09:47:30.1206222Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1206339Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1207157Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1207232Z layer_outputs = layer_module( 2025-12-04T09:47:30.1207450Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1207523Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1207758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1207852Z return func(*args, **kwargs) 2025-12-04T09:47:30.1208067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.1208174Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.1208403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1208473Z return func(*args, **kwargs) 2025-12-04T09:47:30.1208693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.1208770Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.1209004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1209065Z return func(*args, **kwargs) 2025-12-04T09:47:30.1209287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:30.1209365Z query_states = self.q(hidden_states) 2025-12-04T09:47:30.1209369Z 2025-12-04T09:47:30.1209462Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1209653Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1209714Z res = mod(**inputs) 2025-12-04T09:47:30.1209936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1210013Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1210235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1210299Z layer_outputs = layer_module( 2025-12-04T09:47:30.1210516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1210589Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1210821Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1210884Z return func(*args, **kwargs) 2025-12-04T09:47:30.1211107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.1211188Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.1211416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1211485Z return func(*args, **kwargs) 2025-12-04T09:47:30.1211704Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.1211779Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.1212012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1212073Z return func(*args, **kwargs) 2025-12-04T09:47:30.1212292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:30.1212370Z key_states = self.k(current_states) 2025-12-04T09:47:30.1212373Z 2025-12-04T09:47:30.1212483Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1212675Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1212750Z res = mod(**inputs) 2025-12-04T09:47:30.1212976Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1213051Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1213273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1213354Z layer_outputs = layer_module( 2025-12-04T09:47:30.1213568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1213639Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1213889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1213953Z return func(*args, **kwargs) 2025-12-04T09:47:30.1214174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.1214255Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.1214480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1214549Z return func(*args, **kwargs) 2025-12-04T09:47:30.1214775Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.1214848Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.1215079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1215142Z return func(*args, **kwargs) 2025-12-04T09:47:30.1215362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:30.1215439Z value_states = self.v(current_states) 2025-12-04T09:47:30.1215443Z 2025-12-04T09:47:30.1215517Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.1215595Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.1215688Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1215870Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1215936Z res = mod(**inputs) 2025-12-04T09:47:30.1216158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1216224Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1216451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1216516Z layer_outputs = layer_module( 2025-12-04T09:47:30.1216731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1216804Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1217027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1217096Z return func(*args, **kwargs) 2025-12-04T09:47:30.1217317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.1217390Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.1217627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1217690Z return func(*args, **kwargs) 2025-12-04T09:47:30.1217921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.1218011Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.1218253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1218324Z return func(*args, **kwargs) 2025-12-04T09:47:30.1218550Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:30.1218629Z attn_output = self.o(attn_output) 2025-12-04T09:47:30.1218647Z 2025-12-04T09:47:30.1218743Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1218926Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1218993Z res = mod(**inputs) 2025-12-04T09:47:30.1219216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1219324Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1219551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1219616Z layer_outputs = layer_module( 2025-12-04T09:47:30.1219828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1219901Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1220126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1220197Z return func(*args, **kwargs) 2025-12-04T09:47:30.1220415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1220488Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1220717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1220779Z return func(*args, **kwargs) 2025-12-04T09:47:30.1221002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.1221080Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.1221300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1221369Z return func(*args, **kwargs) 2025-12-04T09:47:30.1221591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:30.1221670Z query_states = self.q(hidden_states) 2025-12-04T09:47:30.1221673Z 2025-12-04T09:47:30.1221767Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1221947Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1222015Z res = mod(**inputs) 2025-12-04T09:47:30.1222234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1222299Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1222528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1222593Z layer_outputs = layer_module( 2025-12-04T09:47:30.1222802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1222875Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1223100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1223170Z return func(*args, **kwargs) 2025-12-04T09:47:30.1223385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1223484Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1223711Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1223788Z return func(*args, **kwargs) 2025-12-04T09:47:30.1224019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.1224093Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.1224317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1224402Z return func(*args, **kwargs) 2025-12-04T09:47:30.1224627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:30.1224704Z key_states = self.k(current_states) 2025-12-04T09:47:30.1224732Z 2025-12-04T09:47:30.1224827Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1225011Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1225077Z res = mod(**inputs) 2025-12-04T09:47:30.1225302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1225369Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1225597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1225663Z layer_outputs = layer_module( 2025-12-04T09:47:30.1225881Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1225950Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1226176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1226245Z return func(*args, **kwargs) 2025-12-04T09:47:30.1226466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1226545Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1226773Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1226835Z return func(*args, **kwargs) 2025-12-04T09:47:30.1227060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.1227139Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.1227364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1227433Z return func(*args, **kwargs) 2025-12-04T09:47:30.1227656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:30.1227732Z value_states = self.v(current_states) 2025-12-04T09:47:30.1227735Z 2025-12-04T09:47:30.1227807Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.1227881Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.1227980Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1228164Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1228223Z res = mod(**inputs) 2025-12-04T09:47:30.1228456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1228521Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1228749Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1228815Z layer_outputs = layer_module( 2025-12-04T09:47:30.1229038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1229118Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1229359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1229422Z return func(*args, **kwargs) 2025-12-04T09:47:30.1229650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1229740Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1229969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1230030Z return func(*args, **kwargs) 2025-12-04T09:47:30.1230246Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.1230347Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.1230571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1230639Z return func(*args, **kwargs) 2025-12-04T09:47:30.1230859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:30.1230930Z attn_output = self.o(attn_output) 2025-12-04T09:47:30.1230933Z 2025-12-04T09:47:30.1231033Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1231217Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1231277Z res = mod(**inputs) 2025-12-04T09:47:30.1231504Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1231572Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1231804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1231869Z layer_outputs = layer_module( 2025-12-04T09:47:30.1232079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1232158Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1232382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1232448Z return func(*args, **kwargs) 2025-12-04T09:47:30.1232676Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1232750Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1232983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1233045Z return func(*args, **kwargs) 2025-12-04T09:47:30.1233376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 652, in forward 2025-12-04T09:47:30.1233520Z layer_output = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:47:30.1233524Z 2025-12-04T09:47:30.1233623Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1233820Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1233882Z res = mod(**inputs) 2025-12-04T09:47:30.1234118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1234196Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1234427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1234498Z layer_outputs = layer_module( 2025-12-04T09:47:30.1234755Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1234831Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1235091Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1235154Z return func(*args, **kwargs) 2025-12-04T09:47:30.1235374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.1235485Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.1235709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.1235827Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.1236057Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:30.1236149Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:30.1236153Z 2025-12-04T09:47:30.1236256Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1236445Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1236506Z res = mod(**inputs) 2025-12-04T09:47:30.1236745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1236813Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1237049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1237117Z layer_outputs = layer_module( 2025-12-04T09:47:30.1237331Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1237412Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1237648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1237711Z return func(*args, **kwargs) 2025-12-04T09:47:30.1237949Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.1238035Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.1238274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.1238385Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.1238614Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:30.1238696Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:30.1238701Z 2025-12-04T09:47:30.1238797Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1238994Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1239053Z res = mod(**inputs) 2025-12-04T09:47:30.1239285Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1239361Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1239590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1239659Z layer_outputs = layer_module( 2025-12-04T09:47:30.1239880Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1239953Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1240192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1240257Z return func(*args, **kwargs) 2025-12-04T09:47:30.1240501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.1240608Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.1240840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.1240953Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.1241182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:30.1241273Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:30.1241276Z 2025-12-04T09:47:30.1241377Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1241565Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1241640Z res = mod(**inputs) 2025-12-04T09:47:30.1241883Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1241950Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1242193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1242260Z layer_outputs = layer_module( 2025-12-04T09:47:30.1242473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1242553Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1242785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1242848Z return func(*args, **kwargs) 2025-12-04T09:47:30.1243083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.1243161Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.1243400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1243464Z return func(*args, **kwargs) 2025-12-04T09:47:30.1243693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.1243778Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.1244010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1244082Z return func(*args, **kwargs) 2025-12-04T09:47:30.1244311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:30.1244382Z query_states = self.q(hidden_states) 2025-12-04T09:47:30.1244386Z 2025-12-04T09:47:30.1244489Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1244678Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1244738Z res = mod(**inputs) 2025-12-04T09:47:30.1244978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1245047Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1245286Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1245355Z layer_outputs = layer_module( 2025-12-04T09:47:30.1245568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1245648Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1245880Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1245945Z return func(*args, **kwargs) 2025-12-04T09:47:30.1246201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.1246292Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.1246537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1246600Z return func(*args, **kwargs) 2025-12-04T09:47:30.1246830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.1246941Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.1247179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1247252Z return func(*args, **kwargs) 2025-12-04T09:47:30.1247503Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:30.1247577Z key_states = self.k(current_states) 2025-12-04T09:47:30.1247580Z 2025-12-04T09:47:30.1247684Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1247875Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1247935Z res = mod(**inputs) 2025-12-04T09:47:30.1248203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1248270Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1248500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1248565Z layer_outputs = layer_module( 2025-12-04T09:47:30.1248774Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1248857Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1249088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1249149Z return func(*args, **kwargs) 2025-12-04T09:47:30.1249378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.1249451Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.1249687Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1249750Z return func(*args, **kwargs) 2025-12-04T09:47:30.1249974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.1250057Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.1250283Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1250355Z return func(*args, **kwargs) 2025-12-04T09:47:30.1250577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:30.1250649Z value_states = self.v(current_states) 2025-12-04T09:47:30.1250653Z 2025-12-04T09:47:30.1250733Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.1250806Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.1250898Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1251092Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1251150Z res = mod(**inputs) 2025-12-04T09:47:30.1251382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1251449Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1251690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1251765Z layer_outputs = layer_module( 2025-12-04T09:47:30.1251991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1252064Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1252298Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1252362Z return func(*args, **kwargs) 2025-12-04T09:47:30.1252607Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.1252680Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.1252907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1252996Z return func(*args, **kwargs) 2025-12-04T09:47:30.1253222Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.1253302Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.1253527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1253589Z return func(*args, **kwargs) 2025-12-04T09:47:30.1253814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:30.1253887Z attn_output = self.o(attn_output) 2025-12-04T09:47:30.1253890Z 2025-12-04T09:47:30.1253984Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1254171Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1254230Z res = mod(**inputs) 2025-12-04T09:47:30.1254463Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1254530Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1254756Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1254828Z layer_outputs = layer_module( 2025-12-04T09:47:30.1255039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1255112Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1255346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1255409Z return func(*args, **kwargs) 2025-12-04T09:47:30.1255642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1255718Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1255944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1256014Z return func(*args, **kwargs) 2025-12-04T09:47:30.1256237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.1256319Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.1256549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1256614Z return func(*args, **kwargs) 2025-12-04T09:47:30.1256847Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:30.1256918Z query_states = self.q(hidden_states) 2025-12-04T09:47:30.1256921Z 2025-12-04T09:47:30.1257015Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1257223Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1257283Z res = mod(**inputs) 2025-12-04T09:47:30.1257521Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1257590Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1257814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1257886Z layer_outputs = layer_module( 2025-12-04T09:47:30.1258113Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1258186Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1258423Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1258501Z return func(*args, **kwargs) 2025-12-04T09:47:30.1258732Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1258807Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1259035Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1259106Z return func(*args, **kwargs) 2025-12-04T09:47:30.1259325Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.1259409Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.1259636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1259698Z return func(*args, **kwargs) 2025-12-04T09:47:30.1259929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:30.1260001Z key_states = self.k(current_states) 2025-12-04T09:47:30.1260006Z 2025-12-04T09:47:30.1260101Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1260291Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1260351Z res = mod(**inputs) 2025-12-04T09:47:30.1260585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1260652Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1260879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1260953Z layer_outputs = layer_module( 2025-12-04T09:47:30.1261163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1261236Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1261473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1261537Z return func(*args, **kwargs) 2025-12-04T09:47:30.1261765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1261839Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1262067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1262139Z return func(*args, **kwargs) 2025-12-04T09:47:30.1262360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.1262443Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.1262670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1262733Z return func(*args, **kwargs) 2025-12-04T09:47:30.1262975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:30.1263048Z value_states = self.v(current_states) 2025-12-04T09:47:30.1263067Z 2025-12-04T09:47:30.1263139Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.1263218Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.1263312Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1263502Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1263578Z res = mod(**inputs) 2025-12-04T09:47:30.1263804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1263876Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1264102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1264183Z layer_outputs = layer_module( 2025-12-04T09:47:30.1264400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1264473Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1264709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1264771Z return func(*args, **kwargs) 2025-12-04T09:47:30.1264993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1265074Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1265299Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1265362Z return func(*args, **kwargs) 2025-12-04T09:47:30.1265592Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.1265669Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.1265904Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1265967Z return func(*args, **kwargs) 2025-12-04T09:47:30.1266188Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:30.1266267Z attn_output = self.o(attn_output) 2025-12-04T09:47:30.1266272Z 2025-12-04T09:47:30.1266366Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1266556Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1266614Z res = mod(**inputs) 2025-12-04T09:47:30.1266839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1266915Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1267138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1267205Z layer_outputs = layer_module( 2025-12-04T09:47:30.1267421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1267494Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1267726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1267789Z return func(*args, **kwargs) 2025-12-04T09:47:30.1268012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.1268102Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.1268343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.1268458Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.1268698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:30.1268772Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:30.1268776Z 2025-12-04T09:47:30.1268875Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1269058Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1269132Z res = mod(**inputs) 2025-12-04T09:47:30.1269363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1269431Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1269661Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1269745Z layer_outputs = layer_module( 2025-12-04T09:47:30.1269949Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1270029Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1270250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1270312Z return func(*args, **kwargs) 2025-12-04T09:47:30.1270542Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.1270626Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.1270849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.1270953Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.1271170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:30.1271251Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:30.1271254Z 2025-12-04T09:47:30.1271347Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1271532Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1271590Z res = mod(**inputs) 2025-12-04T09:47:30.1271808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1271884Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1272101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1272165Z layer_outputs = layer_module( 2025-12-04T09:47:30.1272379Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1272452Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1272679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1272742Z return func(*args, **kwargs) 2025-12-04T09:47:30.1272956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.1273046Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.1273333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.1273456Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.1273683Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:30.1273758Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:30.1273762Z 2025-12-04T09:47:30.1273883Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1274070Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1274148Z res = mod(**inputs) 2025-12-04T09:47:30.1274386Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1274453Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1274685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1274770Z layer_outputs = layer_module( 2025-12-04T09:47:30.1274983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1275064Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1275309Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1275374Z return func(*args, **kwargs) 2025-12-04T09:47:30.1275603Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.1275686Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.1275913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 344, in forward 2025-12-04T09:47:30.1276035Z hidden_states = hidden_states + self.dropout(forwarded_states) 2025-12-04T09:47:30.1276040Z 2025-12-04T09:47:30.1276134Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1276324Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1276383Z res = mod(**inputs) 2025-12-04T09:47:30.1276614Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1276683Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1276905Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1276978Z layer_outputs = layer_module( 2025-12-04T09:47:30.1277186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1277259Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1277492Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1277557Z return func(*args, **kwargs) 2025-12-04T09:47:30.1277786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.1277861Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.1278090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1278161Z return func(*args, **kwargs) 2025-12-04T09:47:30.1278381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.1278458Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.1278694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1278757Z return func(*args, **kwargs) 2025-12-04T09:47:30.1278987Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:30.1279058Z query_states = self.q(hidden_states) 2025-12-04T09:47:30.1279062Z 2025-12-04T09:47:30.1279154Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1279350Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1279424Z res = mod(**inputs) 2025-12-04T09:47:30.1279660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1279742Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1279967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1280040Z layer_outputs = layer_module( 2025-12-04T09:47:30.1280247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1280334Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1280569Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1280633Z return func(*args, **kwargs) 2025-12-04T09:47:30.1280877Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.1280951Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.1281177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1281246Z return func(*args, **kwargs) 2025-12-04T09:47:30.1281468Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.1281543Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.1281782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1281845Z return func(*args, **kwargs) 2025-12-04T09:47:30.1282073Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:30.1282144Z key_states = self.k(current_states) 2025-12-04T09:47:30.1282147Z 2025-12-04T09:47:30.1282241Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1282429Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1282489Z res = mod(**inputs) 2025-12-04T09:47:30.1282720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1282787Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1283009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1283082Z layer_outputs = layer_module( 2025-12-04T09:47:30.1283291Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1283361Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1283597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1283660Z return func(*args, **kwargs) 2025-12-04T09:47:30.1283892Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.1283965Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.1284193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1284261Z return func(*args, **kwargs) 2025-12-04T09:47:30.1284487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.1284567Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.1284795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1284858Z return func(*args, **kwargs) 2025-12-04T09:47:30.1285101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:30.1285174Z value_states = self.v(current_states) 2025-12-04T09:47:30.1285177Z 2025-12-04T09:47:30.1285265Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.1285346Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.1285437Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1285626Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1285701Z res = mod(**inputs) 2025-12-04T09:47:30.1285926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1285998Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1286222Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1286303Z layer_outputs = layer_module( 2025-12-04T09:47:30.1286518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1286592Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1286828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1286891Z return func(*args, **kwargs) 2025-12-04T09:47:30.1287114Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:47:30.1287195Z self_attention_outputs = self.layer[0]( 2025-12-04T09:47:30.1287423Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1287486Z return func(*args, **kwargs) 2025-12-04T09:47:30.1287714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:47:30.1287791Z attention_output = self.SelfAttention( 2025-12-04T09:47:30.1288025Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1288088Z return func(*args, **kwargs) 2025-12-04T09:47:30.1288309Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:30.1288390Z attn_output = self.o(attn_output) 2025-12-04T09:47:30.1288394Z 2025-12-04T09:47:30.1288487Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1288677Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1288738Z res = mod(**inputs) 2025-12-04T09:47:30.1288963Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1289039Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1289263Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1289328Z layer_outputs = layer_module( 2025-12-04T09:47:30.1289553Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1289625Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1289855Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1289919Z return func(*args, **kwargs) 2025-12-04T09:47:30.1290139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1290219Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1290443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1290531Z return func(*args, **kwargs) 2025-12-04T09:47:30.1290764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.1290858Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.1291094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1291158Z return func(*args, **kwargs) 2025-12-04T09:47:30.1291381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:47:30.1291477Z query_states = self.q(hidden_states) 2025-12-04T09:47:30.1291481Z 2025-12-04T09:47:30.1291574Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1291765Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1291840Z res = mod(**inputs) 2025-12-04T09:47:30.1292065Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1292142Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1292367Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1292432Z layer_outputs = layer_module( 2025-12-04T09:47:30.1292657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1292731Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1292961Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1293022Z return func(*args, **kwargs) 2025-12-04T09:47:30.1293241Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1293325Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1293552Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1293616Z return func(*args, **kwargs) 2025-12-04T09:47:30.1293844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.1293921Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.1294155Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1294218Z return func(*args, **kwargs) 2025-12-04T09:47:30.1294439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:47:30.1294516Z key_states = self.k(current_states) 2025-12-04T09:47:30.1294521Z 2025-12-04T09:47:30.1294614Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1294805Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1294864Z res = mod(**inputs) 2025-12-04T09:47:30.1295085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1295159Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1295380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1295447Z layer_outputs = layer_module( 2025-12-04T09:47:30.1295663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1295733Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1295964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1296044Z return func(*args, **kwargs) 2025-12-04T09:47:30.1296268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1296361Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1296589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1296651Z return func(*args, **kwargs) 2025-12-04T09:47:30.1296878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.1296972Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.1297207Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1297269Z return func(*args, **kwargs) 2025-12-04T09:47:30.1297510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:47:30.1297590Z value_states = self.v(current_states) 2025-12-04T09:47:30.1297593Z 2025-12-04T09:47:30.1297667Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.1297746Z cudagraph partition due to non gpu ops 2025-12-04T09:47:30.1297840Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1298023Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1298089Z res = mod(**inputs) 2025-12-04T09:47:30.1298315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1298381Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1298612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1298678Z layer_outputs = layer_module( 2025-12-04T09:47:30.1298897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1298966Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1299198Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1299266Z return func(*args, **kwargs) 2025-12-04T09:47:30.1299484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:47:30.1299557Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:47:30.1299789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1299851Z return func(*args, **kwargs) 2025-12-04T09:47:30.1300077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:47:30.1300155Z attention_output = self.EncDecAttention( 2025-12-04T09:47:30.1300382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1300453Z return func(*args, **kwargs) 2025-12-04T09:47:30.1300680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:47:30.1300759Z attn_output = self.o(attn_output) 2025-12-04T09:47:30.1300762Z 2025-12-04T09:47:30.1300854Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1301038Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1301105Z res = mod(**inputs) 2025-12-04T09:47:30.1301328Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1301399Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1301648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1301714Z layer_outputs = layer_module( 2025-12-04T09:47:30.1301946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1302017Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1302245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1302333Z return func(*args, **kwargs) 2025-12-04T09:47:30.1302557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.1302641Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.1302873Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.1303001Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.1303231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:47:30.1303304Z hidden_states = self.wi(hidden_states) 2025-12-04T09:47:30.1303307Z 2025-12-04T09:47:30.1303401Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1303591Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1303649Z res = mod(**inputs) 2025-12-04T09:47:30.1303875Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1303941Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1304158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1304230Z layer_outputs = layer_module( 2025-12-04T09:47:30.1304437Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1304509Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1304740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1304802Z return func(*args, **kwargs) 2025-12-04T09:47:30.1305027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.1305111Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.1305330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.1305445Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.1305859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:47:30.1305954Z hidden_states = self.act(hidden_states) 2025-12-04T09:47:30.1305958Z 2025-12-04T09:47:30.1306052Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1306239Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1306307Z res = mod(**inputs) 2025-12-04T09:47:30.1306543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:47:30.1306610Z decoder_outputs = self.decoder( 2025-12-04T09:47:30.1306847Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:47:30.1306913Z layer_outputs = layer_module( 2025-12-04T09:47:30.1307133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:30.1307208Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:30.1307471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:30.1307544Z return func(*args, **kwargs) 2025-12-04T09:47:30.1307785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:47:30.1307870Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:47:30.1308095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:47:30.1308248Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:47:30.1308489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:47:30.1308561Z hidden_states = self.wo(hidden_states) 2025-12-04T09:47:30.1308595Z 2025-12-04T09:47:30.1308690Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1308882Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1308942Z res = mod(**inputs) 2025-12-04T09:47:30.1309175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1793, in forward 2025-12-04T09:47:30.1309252Z lm_logits = self.lm_head(sequence_output) 2025-12-04T09:47:30.1309255Z 2025-12-04T09:47:30.1309347Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:30.1309535Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:30.1309596Z res = mod(**inputs) 2025-12-04T09:47:30.1309822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1800, in forward 2025-12-04T09:47:30.1309959Z loss = loss_fct(lm_logits.view(-1, lm_logits.size(-1)), labels.view(-1)) 2025-12-04T09:47:30.1309964Z 2025-12-04T09:47:38.5966969Z Compilation time (from dynamo_timed): 17.547599677 2025-12-04T09:47:38.6101568Z pass 2025-12-04T09:47:38.6102227Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:47:38.6103580Z TIMING: _recursive_pre_grad_passes:0.046 _recursive_joint_graph_passes:0.55104 _recursive_post_grad_passes:0.05416 async_compile.wait:0.00339 code_gen:8.06693 inductor_compile:10.06207 backend_compile:14.78444 gc:0.00028 entire_frame_compile:17.5476 total_wall_time:17.5476 2025-12-04T09:47:38.6104949Z STATS: call_* op count: 810 | FakeTensorMode.__torch_dispatch__:21989 | FakeTensor.__torch_dispatch__:3647 | ProxyTorchDispatchMode.__torch_dispatch__:4640 2025-12-04T09:47:38.6105847Z Dynamo produced 1 graphs covering 810 ops with 0 graph breaks (0 unique) 2025-12-04T09:47:40.8372921Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:47:40.8373695Z import pynvml # type: ignore[import] 2025-12-04T09:47:43.9494790Z 2025-12-04T09:47:45.8805655Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:47:45.8806234Z loading model: 0it [00:01, ?it/s] 2025-12-04T09:47:45.8823118Z cpu eval TrOCRForCausalLM 2025-12-04T09:47:46.0142487Z WARNING:common:fp64 golden ref were not generated for TrOCRForCausalLM. Setting accuracy check to cosine 2025-12-04T09:47:46.0416041Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:47:46.2664001Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:47:46.4910806Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:47:54.9745449Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9745900Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9746586Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9746801Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9747019Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9747362Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9747580Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9747784Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9748037Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9748238Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9748504Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9748698Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9748890Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9749076Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9749266Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9749516Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9749705Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9749904Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9750099Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9750284Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9750484Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9750681Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9750908Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:54.9751286Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:54.9751617Z res = mod(**inputs) 2025-12-04T09:47:54.9752007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:47:54.9752395Z outputs = self.model.decoder( 2025-12-04T09:47:54.9752772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:47:54.9753156Z layer_outputs = decoder_layer( 2025-12-04T09:47:54.9753633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:54.9754014Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:54.9754423Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:54.9754821Z return func(*args, **kwargs) 2025-12-04T09:47:54.9755208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:47:54.9755659Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:47:54.9756058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:47:54.9756404Z return self.act(input) 2025-12-04T09:47:54.9756522Z 2025-12-04T09:47:54.9756603Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9756816Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9757021Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9757210Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9757407Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9757605Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9757792Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9757989Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9758188Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9758385Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9758576Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9758804Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:54.9759159Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:54.9759474Z res = mod(**inputs) 2025-12-04T09:47:54.9759897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:47:54.9760291Z outputs = self.model.decoder( 2025-12-04T09:47:54.9760716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:47:54.9761100Z layer_outputs = decoder_layer( 2025-12-04T09:47:54.9761450Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:54.9761806Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:54.9762217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:54.9762580Z return func(*args, **kwargs) 2025-12-04T09:47:54.9762939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:47:54.9763378Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:47:54.9763769Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:47:54.9764111Z return self.act(input) 2025-12-04T09:47:54.9764220Z 2025-12-04T09:47:54.9764301Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9764509Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9764708Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9764907Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9765096Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9765294Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9765495Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9765687Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9765887Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9766084Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9766283Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9766513Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:54.9766868Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:54.9767184Z res = mod(**inputs) 2025-12-04T09:47:54.9767541Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:47:54.9767919Z outputs = self.model.decoder( 2025-12-04T09:47:54.9768295Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:47:54.9768674Z layer_outputs = decoder_layer( 2025-12-04T09:47:54.9769016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:54.9769375Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:54.9769750Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:54.9770117Z return func(*args, **kwargs) 2025-12-04T09:47:54.9770489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:47:54.9770912Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:47:54.9771286Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:47:54.9771624Z return self.act(input) 2025-12-04T09:47:54.9771740Z 2025-12-04T09:47:54.9771818Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9772022Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9772214Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9772465Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9772654Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9772850Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9773051Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9773266Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9773464Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9773660Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9773867Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9774100Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:54.9774456Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:54.9774771Z res = mod(**inputs) 2025-12-04T09:47:54.9775122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:47:54.9775559Z outputs = self.model.decoder( 2025-12-04T09:47:54.9775941Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:47:54.9776329Z layer_outputs = decoder_layer( 2025-12-04T09:47:54.9776675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:54.9777032Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:54.9777405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:54.9777778Z return func(*args, **kwargs) 2025-12-04T09:47:54.9778135Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:47:54.9778548Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:47:54.9778913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:47:54.9779244Z return self.act(input) 2025-12-04T09:47:54.9779355Z 2025-12-04T09:47:54.9779430Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9779630Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9779820Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9780016Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9780208Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9780395Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9780589Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9780783Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9780969Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9781159Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9781350Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9781571Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:54.9781905Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:54.9782213Z res = mod(**inputs) 2025-12-04T09:47:54.9782560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:47:54.9782925Z outputs = self.model.decoder( 2025-12-04T09:47:54.9783288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:47:54.9783658Z layer_outputs = decoder_layer( 2025-12-04T09:47:54.9783998Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:54.9784342Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:54.9784710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:54.9785070Z return func(*args, **kwargs) 2025-12-04T09:47:54.9785415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:47:54.9785827Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:47:54.9786218Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:47:54.9786554Z return self.act(input) 2025-12-04T09:47:54.9786659Z 2025-12-04T09:47:54.9786734Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9786951Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9787150Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9787336Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9787531Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9787741Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9787992Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9788186Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9788380Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9788570Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9788752Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9789020Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:54.9789410Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:54.9789705Z res = mod(**inputs) 2025-12-04T09:47:54.9790052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:47:54.9790418Z outputs = self.model.decoder( 2025-12-04T09:47:54.9790778Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:47:54.9791133Z layer_outputs = decoder_layer( 2025-12-04T09:47:54.9791468Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:54.9791808Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:54.9792156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:54.9792508Z return func(*args, **kwargs) 2025-12-04T09:47:54.9792859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:47:54.9793352Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:47:54.9793731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:47:54.9794065Z return self.act(input) 2025-12-04T09:47:54.9794171Z 2025-12-04T09:47:54.9794252Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9794452Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9794650Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9794843Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9795035Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9795221Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9795413Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9795607Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9795793Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9795985Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9796176Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9796390Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:54.9796731Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:54.9797037Z res = mod(**inputs) 2025-12-04T09:47:54.9797379Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:47:54.9797751Z outputs = self.model.decoder( 2025-12-04T09:47:54.9798114Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:47:54.9798482Z layer_outputs = decoder_layer( 2025-12-04T09:47:54.9798807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:54.9799222Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:54.9799594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:54.9799971Z return func(*args, **kwargs) 2025-12-04T09:47:54.9800323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:47:54.9800741Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:47:54.9801132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:47:54.9801451Z return self.act(input) 2025-12-04T09:47:54.9801565Z 2025-12-04T09:47:54.9801639Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9801837Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9802048Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9802234Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9802428Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9802625Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9802810Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9803003Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9803197Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9803384Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9803577Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9803800Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:54.9804147Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:54.9804444Z res = mod(**inputs) 2025-12-04T09:47:54.9804788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:47:54.9805162Z outputs = self.model.decoder( 2025-12-04T09:47:54.9805519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:47:54.9806223Z layer_outputs = decoder_layer( 2025-12-04T09:47:54.9806560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:54.9806904Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:54.9807257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:54.9807611Z return func(*args, **kwargs) 2025-12-04T09:47:54.9807968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:47:54.9808362Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:47:54.9808730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:47:54.9809059Z return self.act(input) 2025-12-04T09:47:54.9809163Z 2025-12-04T09:47:54.9809243Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9809430Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9809622Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9809815Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9810000Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9810192Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9810384Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9810567Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9810762Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9810953Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9811142Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9811352Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:54.9811688Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:54.9812046Z res = mod(**inputs) 2025-12-04T09:47:54.9812382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:47:54.9812774Z outputs = self.model.decoder( 2025-12-04T09:47:54.9813142Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:47:54.9813514Z layer_outputs = decoder_layer( 2025-12-04T09:47:54.9813846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:54.9814229Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:54.9814607Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:54.9814963Z return func(*args, **kwargs) 2025-12-04T09:47:54.9815336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:47:54.9815734Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:47:54.9816097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:47:54.9816409Z return self.act(input) 2025-12-04T09:47:54.9816520Z 2025-12-04T09:47:54.9816594Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9816787Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9816969Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9817162Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9817350Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9817539Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9817722Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9817914Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9818103Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9818286Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9818474Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9818692Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:54.9819019Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:54.9819317Z res = mod(**inputs) 2025-12-04T09:47:54.9819655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:47:54.9820013Z outputs = self.model.decoder( 2025-12-04T09:47:54.9820359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:47:54.9820715Z layer_outputs = decoder_layer( 2025-12-04T09:47:54.9821038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:54.9821370Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:54.9821727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:54.9822076Z return func(*args, **kwargs) 2025-12-04T09:47:54.9822427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:47:54.9822816Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:47:54.9823177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:47:54.9823498Z return self.act(input) 2025-12-04T09:47:54.9823598Z 2025-12-04T09:47:54.9823669Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9823863Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9824052Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9824242Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9824426Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9824630Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9824825Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9825010Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9825223Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9825417Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9825601Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9825823Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:54.9826164Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:54.9826486Z res = mod(**inputs) 2025-12-04T09:47:54.9826827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:47:54.9827196Z outputs = self.model.decoder( 2025-12-04T09:47:54.9827558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:47:54.9827932Z layer_outputs = decoder_layer( 2025-12-04T09:47:54.9828263Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:54.9828610Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:54.9828970Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:54.9829321Z return func(*args, **kwargs) 2025-12-04T09:47:54.9829678Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:47:54.9830086Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:47:54.9830448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:47:54.9830776Z return self.act(input) 2025-12-04T09:47:54.9830884Z 2025-12-04T09:47:54.9830956Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9831151Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9831375Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9831568Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9831760Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9831948Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9832132Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9832323Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9832511Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9832696Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9832887Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9833114Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:54.9833550Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:54.9833874Z res = mod(**inputs) 2025-12-04T09:47:54.9834229Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:47:54.9834612Z outputs = self.model.decoder( 2025-12-04T09:47:54.9834981Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:47:54.9835354Z layer_outputs = decoder_layer( 2025-12-04T09:47:54.9835696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:54.9836043Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:54.9836422Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:54.9836793Z return func(*args, **kwargs) 2025-12-04T09:47:54.9837161Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:47:54.9837622Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:47:54.9838044Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:47:54.9838422Z return self.act(input) 2025-12-04T09:47:54.9838533Z 2025-12-04T09:47:54.9838611Z cudagraph partition due to non gpu ops 2025-12-04T09:47:54.9838852Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:54.9839244Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:54.9839602Z res = mod(**inputs) 2025-12-04T09:47:54.9839970Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 844, in forward 2025-12-04T09:47:54.9840398Z logits = self.output_projection(outputs[0]) 2025-12-04T09:47:54.9840557Z 2025-12-04T09:47:54.9840667Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:54.9841078Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:54.9841407Z res = mod(**inputs) 2025-12-04T09:47:54.9841781Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 849, in forward 2025-12-04T09:47:54.9842285Z loss = loss_fct(logits.view(-1, self.config.vocab_size), labels.view(-1)) 2025-12-04T09:47:54.9842491Z 2025-12-04T09:48:04.3543572Z Compilation time (from dynamo_timed): 17.222492828 2025-12-04T09:48:04.3564039Z pass 2025-12-04T09:48:04.3564376Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:48:04.3565190Z TIMING: _recursive_pre_grad_passes:0.03208 _recursive_joint_graph_passes:0.67246 _recursive_post_grad_passes:0.05442 async_compile.wait:0.78732 code_gen:9.37584 inductor_compile:11.00411 backend_compile:14.89604 gc:0.00117 entire_frame_compile:17.22249 total_wall_time:17.22249 2025-12-04T09:48:04.3566100Z STATS: call_* op count: 443 | FakeTensorMode.__torch_dispatch__:20105 | FakeTensor.__torch_dispatch__:3533 | ProxyTorchDispatchMode.__torch_dispatch__:3349 2025-12-04T09:48:04.3566590Z Dynamo produced 1 graphs covering 443 ops with 0 graph breaks (0 unique) 2025-12-04T09:48:06.6460201Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:48:06.6461147Z import pynvml # type: ignore[import] 2025-12-04T09:48:09.7628527Z 2025-12-04T09:48:15.0136346Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:48:15.0140375Z loading model: 0it [00:05, ?it/s] 2025-12-04T09:48:15.0156205Z cpu eval XGLMForCausalLM 2025-12-04T09:48:15.3927658Z WARNING:common:fp64 golden ref were not generated for XGLMForCausalLM. Setting accuracy check to cosine 2025-12-04T09:48:15.4994509Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:48:15.9995251Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:48:16.4965719Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:48:33.0090726Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0091044Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0091273Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0091677Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0092000Z res = mod(**inputs) 2025-12-04T09:48:33.0092369Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0092748Z outputs = self.model( 2025-12-04T09:48:33.0093449Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0093833Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0094230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0094598Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0094978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0095410Z return func(*args, **kwargs) 2025-12-04T09:48:33.0095836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0096222Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0096613Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0097015Z return func(*args, **kwargs) 2025-12-04T09:48:33.0097371Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0097791Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0097956Z 2025-12-04T09:48:33.0098065Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0098413Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0098722Z res = mod(**inputs) 2025-12-04T09:48:33.0099062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0099421Z outputs = self.model( 2025-12-04T09:48:33.0099766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0100134Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0100474Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0100833Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0101210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0101577Z return func(*args, **kwargs) 2025-12-04T09:48:33.0101983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0102403Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0102796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0103158Z return func(*args, **kwargs) 2025-12-04T09:48:33.0103511Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0103892Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0104026Z 2025-12-04T09:48:33.0104137Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0104478Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0104792Z res = mod(**inputs) 2025-12-04T09:48:33.0105134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0105502Z outputs = self.model( 2025-12-04T09:48:33.0106156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0106547Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0106897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0107261Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0107730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0108129Z return func(*args, **kwargs) 2025-12-04T09:48:33.0108551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0108994Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0109381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0109782Z return func(*args, **kwargs) 2025-12-04T09:48:33.0110143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0110578Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0110761Z 2025-12-04T09:48:33.0110871Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0111107Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0111457Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0111768Z res = mod(**inputs) 2025-12-04T09:48:33.0112118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0112497Z outputs = self.model( 2025-12-04T09:48:33.0112954Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0113335Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0113675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0114026Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0114384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0114744Z return func(*args, **kwargs) 2025-12-04T09:48:33.0115094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0115484Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0115857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0116258Z return func(*args, **kwargs) 2025-12-04T09:48:33.0116618Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0117001Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0117139Z 2025-12-04T09:48:33.0117239Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0117590Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0117915Z res = mod(**inputs) 2025-12-04T09:48:33.0118247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0118611Z outputs = self.model( 2025-12-04T09:48:33.0119130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0119492Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0119819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0120168Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0120533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0120892Z return func(*args, **kwargs) 2025-12-04T09:48:33.0121237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0121653Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0122047Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0122424Z return func(*args, **kwargs) 2025-12-04T09:48:33.0122776Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0123162Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0123304Z 2025-12-04T09:48:33.0123429Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0123769Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0124082Z res = mod(**inputs) 2025-12-04T09:48:33.0124429Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0124806Z outputs = self.model( 2025-12-04T09:48:33.0125156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0125525Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0125864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0126213Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0126598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0126962Z return func(*args, **kwargs) 2025-12-04T09:48:33.0127319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0127706Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0128092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0128457Z return func(*args, **kwargs) 2025-12-04T09:48:33.0128803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0129232Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0129414Z 2025-12-04T09:48:33.0129491Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0129704Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0129916Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0130253Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0130555Z res = mod(**inputs) 2025-12-04T09:48:33.0130884Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0131242Z outputs = self.model( 2025-12-04T09:48:33.0131579Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0131936Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0132264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0132607Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0132967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0133312Z return func(*args, **kwargs) 2025-12-04T09:48:33.0133660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0134065Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0134443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0134776Z return self.act(input) 2025-12-04T09:48:33.0134908Z 2025-12-04T09:48:33.0134985Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0135190Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0135401Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0135634Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0135997Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0136312Z res = mod(**inputs) 2025-12-04T09:48:33.0136654Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0137041Z outputs = self.model( 2025-12-04T09:48:33.0137387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0137754Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0138117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0138465Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0138828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0139176Z return func(*args, **kwargs) 2025-12-04T09:48:33.0139528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0139918Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0140302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0140651Z return func(*args, **kwargs) 2025-12-04T09:48:33.0140999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0141399Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0141553Z 2025-12-04T09:48:33.0141652Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0141992Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0142301Z res = mod(**inputs) 2025-12-04T09:48:33.0142636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0142984Z outputs = self.model( 2025-12-04T09:48:33.0143323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0143683Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0144011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0144363Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0144727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0145078Z return func(*args, **kwargs) 2025-12-04T09:48:33.0145421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0145803Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0146180Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0146531Z return func(*args, **kwargs) 2025-12-04T09:48:33.0146872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0147239Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0147366Z 2025-12-04T09:48:33.0147470Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0147821Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0148126Z res = mod(**inputs) 2025-12-04T09:48:33.0148477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0148847Z outputs = self.model( 2025-12-04T09:48:33.0149180Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0149543Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0149876Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0150229Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0150590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0150980Z return func(*args, **kwargs) 2025-12-04T09:48:33.0151330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0151710Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0152101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0152463Z return func(*args, **kwargs) 2025-12-04T09:48:33.0152815Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0153324Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0153516Z 2025-12-04T09:48:33.0153592Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0153821Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0154168Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0154534Z res = mod(**inputs) 2025-12-04T09:48:33.0154895Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0155347Z outputs = self.model( 2025-12-04T09:48:33.0155690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0156063Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0156415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0156770Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0157143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0157503Z return func(*args, **kwargs) 2025-12-04T09:48:33.0157859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0158251Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0158643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0159013Z return func(*args, **kwargs) 2025-12-04T09:48:33.0159367Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0159743Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0159890Z 2025-12-04T09:48:33.0159991Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0160335Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0160645Z res = mod(**inputs) 2025-12-04T09:48:33.0160993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0161363Z outputs = self.model( 2025-12-04T09:48:33.0161731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0162099Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0162454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0162805Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0163162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0163546Z return func(*args, **kwargs) 2025-12-04T09:48:33.0163903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0164301Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0164685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0165069Z return func(*args, **kwargs) 2025-12-04T09:48:33.0165424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0165827Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0165975Z 2025-12-04T09:48:33.0166079Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0166425Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0166742Z res = mod(**inputs) 2025-12-04T09:48:33.0167090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0167453Z outputs = self.model( 2025-12-04T09:48:33.0167792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0168156Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0168479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0168825Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0169191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0169539Z return func(*args, **kwargs) 2025-12-04T09:48:33.0169892Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0170280Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0170662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0171007Z return func(*args, **kwargs) 2025-12-04T09:48:33.0171354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0171768Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0171937Z 2025-12-04T09:48:33.0172017Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0172211Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0172430Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0172765Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0173071Z res = mod(**inputs) 2025-12-04T09:48:33.0173410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0173770Z outputs = self.model( 2025-12-04T09:48:33.0174104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0174459Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0174808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0175153Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0175535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0175893Z return func(*args, **kwargs) 2025-12-04T09:48:33.0176242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0176662Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0177028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0177352Z return self.act(input) 2025-12-04T09:48:33.0177455Z 2025-12-04T09:48:33.0177539Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0177748Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0177943Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0178164Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0178510Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0178812Z res = mod(**inputs) 2025-12-04T09:48:33.0179154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0179517Z outputs = self.model( 2025-12-04T09:48:33.0179850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0180214Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0180544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0180890Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0181249Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0181609Z return func(*args, **kwargs) 2025-12-04T09:48:33.0181964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0182344Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0182727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0183081Z return func(*args, **kwargs) 2025-12-04T09:48:33.0183433Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0183824Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0183986Z 2025-12-04T09:48:33.0184085Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0184426Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0184730Z res = mod(**inputs) 2025-12-04T09:48:33.0185064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0185427Z outputs = self.model( 2025-12-04T09:48:33.0185769Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0186123Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0186455Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0186805Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0187169Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0187514Z return func(*args, **kwargs) 2025-12-04T09:48:33.0187879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0188266Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0188652Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0188999Z return func(*args, **kwargs) 2025-12-04T09:48:33.0189345Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0189731Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0189858Z 2025-12-04T09:48:33.0189956Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0190292Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0190597Z res = mod(**inputs) 2025-12-04T09:48:33.0190959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0191311Z outputs = self.model( 2025-12-04T09:48:33.0191651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0192017Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0192342Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0192687Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0193154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0193538Z return func(*args, **kwargs) 2025-12-04T09:48:33.0193901Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0194321Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0194715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0195105Z return func(*args, **kwargs) 2025-12-04T09:48:33.0195455Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0195875Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0196053Z 2025-12-04T09:48:33.0196135Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0196356Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0196694Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0197001Z res = mod(**inputs) 2025-12-04T09:48:33.0197340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0197701Z outputs = self.model( 2025-12-04T09:48:33.0198043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0198404Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0198730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0199077Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0199438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0199800Z return func(*args, **kwargs) 2025-12-04T09:48:33.0200143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0200527Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0200908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0201279Z return func(*args, **kwargs) 2025-12-04T09:48:33.0201631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0202024Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0202159Z 2025-12-04T09:48:33.0202262Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0202594Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0202902Z res = mod(**inputs) 2025-12-04T09:48:33.0203265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0203613Z outputs = self.model( 2025-12-04T09:48:33.0203936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0204318Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0204656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0204995Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0205359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0205849Z return func(*args, **kwargs) 2025-12-04T09:48:33.0206214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0206599Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0206977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0207332Z return func(*args, **kwargs) 2025-12-04T09:48:33.0207671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0208065Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0208216Z 2025-12-04T09:48:33.0208313Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0208649Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0208948Z res = mod(**inputs) 2025-12-04T09:48:33.0209290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0209649Z outputs = self.model( 2025-12-04T09:48:33.0209988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0210342Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0210680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0211038Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0211391Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0211748Z return func(*args, **kwargs) 2025-12-04T09:48:33.0212100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0212486Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0212854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0213207Z return func(*args, **kwargs) 2025-12-04T09:48:33.0213549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0213955Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0214130Z 2025-12-04T09:48:33.0214205Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0214448Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0214677Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0215032Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0215339Z res = mod(**inputs) 2025-12-04T09:48:33.0215675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0216029Z outputs = self.model( 2025-12-04T09:48:33.0216397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0216758Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0217092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0217457Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0217825Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0218182Z return func(*args, **kwargs) 2025-12-04T09:48:33.0218523Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0218936Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0219316Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0219652Z return self.act(input) 2025-12-04T09:48:33.0219756Z 2025-12-04T09:48:33.0219832Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0220037Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0220232Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0220442Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0220781Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0221091Z res = mod(**inputs) 2025-12-04T09:48:33.0221431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0221782Z outputs = self.model( 2025-12-04T09:48:33.0222121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0222481Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0222803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0223152Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0223514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0223871Z return func(*args, **kwargs) 2025-12-04T09:48:33.0224216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0224601Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0224983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0225335Z return func(*args, **kwargs) 2025-12-04T09:48:33.0225679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0226078Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0226234Z 2025-12-04T09:48:33.0226339Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0226672Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0226980Z res = mod(**inputs) 2025-12-04T09:48:33.0227336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0227696Z outputs = self.model( 2025-12-04T09:48:33.0228053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0228411Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0228737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0229083Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0229454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0229806Z return func(*args, **kwargs) 2025-12-04T09:48:33.0230150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0230545Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0230924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0231279Z return func(*args, **kwargs) 2025-12-04T09:48:33.0231632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0231992Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0232130Z 2025-12-04T09:48:33.0232227Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0232566Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0233654Z res = mod(**inputs) 2025-12-04T09:48:33.0234032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0234387Z outputs = self.model( 2025-12-04T09:48:33.0234726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0235114Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0235451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0235792Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0236164Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0236519Z return func(*args, **kwargs) 2025-12-04T09:48:33.0236868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0237254Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0237624Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0237977Z return func(*args, **kwargs) 2025-12-04T09:48:33.0238326Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0238742Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0238921Z 2025-12-04T09:48:33.0238996Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0239217Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0239554Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0239851Z res = mod(**inputs) 2025-12-04T09:48:33.0240192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0240553Z outputs = self.model( 2025-12-04T09:48:33.0240898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0241261Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0241642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0241991Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0242368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0242721Z return func(*args, **kwargs) 2025-12-04T09:48:33.0243072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0243482Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0243854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0244210Z return func(*args, **kwargs) 2025-12-04T09:48:33.0244556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0244948Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0245082Z 2025-12-04T09:48:33.0245178Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0245516Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0245819Z res = mod(**inputs) 2025-12-04T09:48:33.0246146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0246505Z outputs = self.model( 2025-12-04T09:48:33.0246846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0247208Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0247529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0247876Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0248234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0248578Z return func(*args, **kwargs) 2025-12-04T09:48:33.0248925Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0249312Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0249687Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0250031Z return func(*args, **kwargs) 2025-12-04T09:48:33.0250378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0250758Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0250902Z 2025-12-04T09:48:33.0251006Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0251332Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0251638Z res = mod(**inputs) 2025-12-04T09:48:33.0251973Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0252322Z outputs = self.model( 2025-12-04T09:48:33.0252660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0253018Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0253346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0253680Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0254040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0254395Z return func(*args, **kwargs) 2025-12-04T09:48:33.0254749Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0255165Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0255536Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0255881Z return func(*args, **kwargs) 2025-12-04T09:48:33.0256216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0256632Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0256797Z 2025-12-04T09:48:33.0256879Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0257077Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0257290Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0257648Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0257954Z res = mod(**inputs) 2025-12-04T09:48:33.0258286Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0258646Z outputs = self.model( 2025-12-04T09:48:33.0258990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0259352Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0259680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0260023Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0260384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0260733Z return func(*args, **kwargs) 2025-12-04T09:48:33.0261083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0261487Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0261862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0262180Z return self.act(input) 2025-12-04T09:48:33.0262292Z 2025-12-04T09:48:33.0262366Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0262567Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0262754Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0262973Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0263309Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0263615Z res = mod(**inputs) 2025-12-04T09:48:33.0263950Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0264310Z outputs = self.model( 2025-12-04T09:48:33.0264651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0265007Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0265339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0265687Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0266048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0266400Z return func(*args, **kwargs) 2025-12-04T09:48:33.0266747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0267133Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0267519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0267873Z return func(*args, **kwargs) 2025-12-04T09:48:33.0268234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0268636Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0268788Z 2025-12-04T09:48:33.0268885Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0269223Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0269541Z res = mod(**inputs) 2025-12-04T09:48:33.0269878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0270232Z outputs = self.model( 2025-12-04T09:48:33.0270586Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0270946Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0271270Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0271610Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0271972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0272331Z return func(*args, **kwargs) 2025-12-04T09:48:33.0272681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0273169Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0273562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0273923Z return func(*args, **kwargs) 2025-12-04T09:48:33.0274285Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0274657Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0274783Z 2025-12-04T09:48:33.0274891Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0275223Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0275530Z res = mod(**inputs) 2025-12-04T09:48:33.0275869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0276231Z outputs = self.model( 2025-12-04T09:48:33.0276562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0276925Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0277257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0277590Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0277953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0278306Z return func(*args, **kwargs) 2025-12-04T09:48:33.0278648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0279022Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0279401Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0279754Z return func(*args, **kwargs) 2025-12-04T09:48:33.0280091Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0280510Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0280710Z 2025-12-04T09:48:33.0280787Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0281010Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0281356Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0281663Z res = mod(**inputs) 2025-12-04T09:48:33.0282004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0282355Z outputs = self.model( 2025-12-04T09:48:33.0282712Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0283074Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0283404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0283758Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0284128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0284487Z return func(*args, **kwargs) 2025-12-04T09:48:33.0284840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0285225Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0285608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0285968Z return func(*args, **kwargs) 2025-12-04T09:48:33.0286314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0286694Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0286838Z 2025-12-04T09:48:33.0286936Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0287283Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0287588Z res = mod(**inputs) 2025-12-04T09:48:33.0287932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0288297Z outputs = self.model( 2025-12-04T09:48:33.0288635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0289000Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0289343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0289691Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0290048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0290412Z return func(*args, **kwargs) 2025-12-04T09:48:33.0290771Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0291164Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0291543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0291908Z return func(*args, **kwargs) 2025-12-04T09:48:33.0292262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0292653Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0292806Z 2025-12-04T09:48:33.0292905Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0293251Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0293566Z res = mod(**inputs) 2025-12-04T09:48:33.0293929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0294280Z outputs = self.model( 2025-12-04T09:48:33.0294626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0294981Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0295312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0295653Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0296036Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0296385Z return func(*args, **kwargs) 2025-12-04T09:48:33.0296735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0297143Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0297521Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0297871Z return func(*args, **kwargs) 2025-12-04T09:48:33.0298219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0298632Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0298799Z 2025-12-04T09:48:33.0298874Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0299078Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0299304Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0299641Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0299938Z res = mod(**inputs) 2025-12-04T09:48:33.0300279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0300641Z outputs = self.model( 2025-12-04T09:48:33.0300973Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0301336Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0301665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0302009Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0302365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0302720Z return func(*args, **kwargs) 2025-12-04T09:48:33.0303062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0303460Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0303835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0304159Z return self.act(input) 2025-12-04T09:48:33.0304261Z 2025-12-04T09:48:33.0304343Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0304536Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0304730Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0304949Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0305279Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0305591Z res = mod(**inputs) 2025-12-04T09:48:33.0306086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0306443Z outputs = self.model( 2025-12-04T09:48:33.0306775Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0307188Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0307524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0307891Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0308255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0308608Z return func(*args, **kwargs) 2025-12-04T09:48:33.0308959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0309361Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0309746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0310122Z return func(*args, **kwargs) 2025-12-04T09:48:33.0310471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0310861Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0311023Z 2025-12-04T09:48:33.0311124Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0311463Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0311761Z res = mod(**inputs) 2025-12-04T09:48:33.0312100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0312459Z outputs = self.model( 2025-12-04T09:48:33.0312794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0313203Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0313545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0313893Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0314248Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0314604Z return func(*args, **kwargs) 2025-12-04T09:48:33.0314956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0315345Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0315720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0316080Z return func(*args, **kwargs) 2025-12-04T09:48:33.0316431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0316803Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0316930Z 2025-12-04T09:48:33.0317027Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0317365Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0317671Z res = mod(**inputs) 2025-12-04T09:48:33.0317998Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0318355Z outputs = self.model( 2025-12-04T09:48:33.0318696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0319058Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0319384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0319732Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0320130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0320476Z return func(*args, **kwargs) 2025-12-04T09:48:33.0320839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0321227Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0321605Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0321951Z return func(*args, **kwargs) 2025-12-04T09:48:33.0322312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0322735Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0322913Z 2025-12-04T09:48:33.0322996Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0323230Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0323566Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0323874Z res = mod(**inputs) 2025-12-04T09:48:33.0324207Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0324566Z outputs = self.model( 2025-12-04T09:48:33.0324908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0325267Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0325593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0325934Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0326291Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0326641Z return func(*args, **kwargs) 2025-12-04T09:48:33.0326991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0327373Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0327751Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0328101Z return func(*args, **kwargs) 2025-12-04T09:48:33.0328452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0328839Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0328971Z 2025-12-04T09:48:33.0329074Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0329404Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0329710Z res = mod(**inputs) 2025-12-04T09:48:33.0330051Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0330413Z outputs = self.model( 2025-12-04T09:48:33.0330756Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0331116Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0331446Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0331784Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0332145Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0332498Z return func(*args, **kwargs) 2025-12-04T09:48:33.0332832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0333217Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0333618Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0333974Z return func(*args, **kwargs) 2025-12-04T09:48:33.0334330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0334718Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0334858Z 2025-12-04T09:48:33.0334961Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0335311Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0335611Z res = mod(**inputs) 2025-12-04T09:48:33.0335946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0336319Z outputs = self.model( 2025-12-04T09:48:33.0336649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0337012Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0337346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0337688Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0338040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0338394Z return func(*args, **kwargs) 2025-12-04T09:48:33.0338739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0339118Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0339498Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0339854Z return func(*args, **kwargs) 2025-12-04T09:48:33.0340203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0340612Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0340786Z 2025-12-04T09:48:33.0340862Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0341063Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0341276Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0341613Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0341918Z res = mod(**inputs) 2025-12-04T09:48:33.0342257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0342606Z outputs = self.model( 2025-12-04T09:48:33.0342951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0343317Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0343644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0343991Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0344349Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0344706Z return func(*args, **kwargs) 2025-12-04T09:48:33.0345050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0345454Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0345827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0346157Z return self.act(input) 2025-12-04T09:48:33.0346262Z 2025-12-04T09:48:33.0346352Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0346551Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0346749Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0346975Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0347317Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0347628Z res = mod(**inputs) 2025-12-04T09:48:33.0347962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0348336Z outputs = self.model( 2025-12-04T09:48:33.0348671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0349031Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0349374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0349718Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0350075Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0350426Z return func(*args, **kwargs) 2025-12-04T09:48:33.0350762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0351142Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0351514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0351856Z return func(*args, **kwargs) 2025-12-04T09:48:33.0352201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0352596Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0352748Z 2025-12-04T09:48:33.0352916Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0353260Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0353572Z res = mod(**inputs) 2025-12-04T09:48:33.0353915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0354265Z outputs = self.model( 2025-12-04T09:48:33.0354607Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0354973Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0355308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0355646Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0356013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0356367Z return func(*args, **kwargs) 2025-12-04T09:48:33.0356716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0357097Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0357476Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0357836Z return func(*args, **kwargs) 2025-12-04T09:48:33.0358175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0358547Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0358679Z 2025-12-04T09:48:33.0358777Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0359114Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0359429Z res = mod(**inputs) 2025-12-04T09:48:33.0359764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0360134Z outputs = self.model( 2025-12-04T09:48:33.0360471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0360837Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0361175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0361537Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0361892Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0362249Z return func(*args, **kwargs) 2025-12-04T09:48:33.0362615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0363002Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0363377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0363732Z return func(*args, **kwargs) 2025-12-04T09:48:33.0364079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0364493Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0364678Z 2025-12-04T09:48:33.0364751Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0364971Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0365305Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0365609Z res = mod(**inputs) 2025-12-04T09:48:33.0365944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0366299Z outputs = self.model( 2025-12-04T09:48:33.0366632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0366992Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0367327Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0367668Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0368025Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0368382Z return func(*args, **kwargs) 2025-12-04T09:48:33.0368730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0369119Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0369493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0369846Z return func(*args, **kwargs) 2025-12-04T09:48:33.0370193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0370560Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0370699Z 2025-12-04T09:48:33.0370793Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0371132Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0371438Z res = mod(**inputs) 2025-12-04T09:48:33.0371767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0372130Z outputs = self.model( 2025-12-04T09:48:33.0372486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0372834Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0373175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0373520Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0373880Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0374224Z return func(*args, **kwargs) 2025-12-04T09:48:33.0374589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0374976Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0375348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0375720Z return func(*args, **kwargs) 2025-12-04T09:48:33.0376068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0376455Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0376595Z 2025-12-04T09:48:33.0376691Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0377027Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0377331Z res = mod(**inputs) 2025-12-04T09:48:33.0377670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0378024Z outputs = self.model( 2025-12-04T09:48:33.0378365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0378729Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0379057Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0379402Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0379760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0380115Z return func(*args, **kwargs) 2025-12-04T09:48:33.0380450Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0380834Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0381208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0381554Z return func(*args, **kwargs) 2025-12-04T09:48:33.0381897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0382313Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0382482Z 2025-12-04T09:48:33.0382564Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0382755Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0382976Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0383310Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0383614Z res = mod(**inputs) 2025-12-04T09:48:33.0383945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0384306Z outputs = self.model( 2025-12-04T09:48:33.0384646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0385004Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0385375Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0385719Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0386097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0386443Z return func(*args, **kwargs) 2025-12-04T09:48:33.0386793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0387208Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0387594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0387919Z return self.act(input) 2025-12-04T09:48:33.0388028Z 2025-12-04T09:48:33.0388102Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0388300Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0388504Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0388722Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0389059Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0389354Z res = mod(**inputs) 2025-12-04T09:48:33.0389685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0390036Z outputs = self.model( 2025-12-04T09:48:33.0390373Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0390721Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0391047Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0391385Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0391735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0392085Z return func(*args, **kwargs) 2025-12-04T09:48:33.0392429Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0392803Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0393239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0393590Z return func(*args, **kwargs) 2025-12-04T09:48:33.0393938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0394325Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0394484Z 2025-12-04T09:48:33.0394580Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0394919Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0395220Z res = mod(**inputs) 2025-12-04T09:48:33.0395547Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0395903Z outputs = self.model( 2025-12-04T09:48:33.0396237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0396594Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0396916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0397255Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0397608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0397947Z return func(*args, **kwargs) 2025-12-04T09:48:33.0398308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0398698Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0399089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0399438Z return func(*args, **kwargs) 2025-12-04T09:48:33.0399786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0400155Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0400309Z 2025-12-04T09:48:33.0400413Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0400740Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0401046Z res = mod(**inputs) 2025-12-04T09:48:33.0401382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0401752Z outputs = self.model( 2025-12-04T09:48:33.0402092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0402456Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0402790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0403132Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0403494Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0403849Z return func(*args, **kwargs) 2025-12-04T09:48:33.0404191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0404578Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0404961Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0405313Z return func(*args, **kwargs) 2025-12-04T09:48:33.0405657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0406188Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0406367Z 2025-12-04T09:48:33.0406451Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0406671Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0407014Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0407320Z res = mod(**inputs) 2025-12-04T09:48:33.0407660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0408014Z outputs = self.model( 2025-12-04T09:48:33.0408361Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0408725Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0409058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0409398Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0409768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0410120Z return func(*args, **kwargs) 2025-12-04T09:48:33.0410459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0410842Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0411219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0411569Z return func(*args, **kwargs) 2025-12-04T09:48:33.0411944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0412344Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0412477Z 2025-12-04T09:48:33.0412581Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0412917Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0413222Z res = mod(**inputs) 2025-12-04T09:48:33.0413581Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0413931Z outputs = self.model( 2025-12-04T09:48:33.0414254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0414637Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0414974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0415321Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0415679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0416033Z return func(*args, **kwargs) 2025-12-04T09:48:33.0416382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0416762Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0417143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0417495Z return func(*args, **kwargs) 2025-12-04T09:48:33.0417844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0418225Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0418374Z 2025-12-04T09:48:33.0418471Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0418812Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0419114Z res = mod(**inputs) 2025-12-04T09:48:33.0419456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0419816Z outputs = self.model( 2025-12-04T09:48:33.0420157Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0420516Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0420848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0421195Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0421563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0421919Z return func(*args, **kwargs) 2025-12-04T09:48:33.0422270Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0422658Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0423028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0423385Z return func(*args, **kwargs) 2025-12-04T09:48:33.0423735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0424151Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0424319Z 2025-12-04T09:48:33.0424395Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0424596Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0424836Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0425165Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0425482Z res = mod(**inputs) 2025-12-04T09:48:33.0425821Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0426182Z outputs = self.model( 2025-12-04T09:48:33.0426516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0426894Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0427224Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0427558Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0427933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0428283Z return func(*args, **kwargs) 2025-12-04T09:48:33.0428631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0429030Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0429403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0429727Z return self.act(input) 2025-12-04T09:48:33.0429831Z 2025-12-04T09:48:33.0429910Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0430101Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0430292Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0430507Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0430833Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0431137Z res = mod(**inputs) 2025-12-04T09:48:33.0431473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0431825Z outputs = self.model( 2025-12-04T09:48:33.0432162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0432523Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0432922Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0433285Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0433662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0434028Z return func(*args, **kwargs) 2025-12-04T09:48:33.0434385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0434775Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0435202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0435572Z return func(*args, **kwargs) 2025-12-04T09:48:33.0435925Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0436340Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0436508Z 2025-12-04T09:48:33.0436609Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0436955Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0437263Z res = mod(**inputs) 2025-12-04T09:48:33.0437607Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0437993Z outputs = self.model( 2025-12-04T09:48:33.0438334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0438724Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0439069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0439428Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0439799Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0440181Z return func(*args, **kwargs) 2025-12-04T09:48:33.0440539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0440942Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0441339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0441701Z return func(*args, **kwargs) 2025-12-04T09:48:33.0442058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0442430Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0442569Z 2025-12-04T09:48:33.0442668Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0443008Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0443319Z res = mod(**inputs) 2025-12-04T09:48:33.0443657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0444023Z outputs = self.model( 2025-12-04T09:48:33.0444265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0444344Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0444563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0444645Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0444880Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0444945Z return func(*args, **kwargs) 2025-12-04T09:48:33.0445191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0445285Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0445524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0445595Z return func(*args, **kwargs) 2025-12-04T09:48:33.0445840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0445972Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0445976Z 2025-12-04T09:48:33.0446053Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0446153Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0446348Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0446411Z res = mod(**inputs) 2025-12-04T09:48:33.0446655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0446727Z outputs = self.model( 2025-12-04T09:48:33.0446969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0447046Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0447277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0447353Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0447629Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0447693Z return func(*args, **kwargs) 2025-12-04T09:48:33.0447936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0448026Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0448268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0448340Z return func(*args, **kwargs) 2025-12-04T09:48:33.0448571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0448668Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0448673Z 2025-12-04T09:48:33.0448779Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0448963Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0449034Z res = mod(**inputs) 2025-12-04T09:48:33.0449270Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0449339Z outputs = self.model( 2025-12-04T09:48:33.0449585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0449656Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0449886Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0449964Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0450197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0450271Z return func(*args, **kwargs) 2025-12-04T09:48:33.0450508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0450601Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0450844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0450912Z return func(*args, **kwargs) 2025-12-04T09:48:33.0451156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0451249Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0451252Z 2025-12-04T09:48:33.0451351Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0451551Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0451612Z res = mod(**inputs) 2025-12-04T09:48:33.0451851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0451925Z outputs = self.model( 2025-12-04T09:48:33.0452158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0452237Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0452452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0452526Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0452764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0452831Z return func(*args, **kwargs) 2025-12-04T09:48:33.0453086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0453176Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0453415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0453486Z return func(*args, **kwargs) 2025-12-04T09:48:33.0453719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0453853Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0453863Z 2025-12-04T09:48:33.0453937Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0454010Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0454112Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0454312Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0454375Z res = mod(**inputs) 2025-12-04T09:48:33.0454617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0454683Z outputs = self.model( 2025-12-04T09:48:33.0454920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0454998Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0455210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0455295Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0455526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0455590Z return func(*args, **kwargs) 2025-12-04T09:48:33.0455839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0455956Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0456170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0456239Z return self.act(input) 2025-12-04T09:48:33.0456242Z 2025-12-04T09:48:33.0456317Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0456676Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0456747Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0456847Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0457043Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0457104Z res = mod(**inputs) 2025-12-04T09:48:33.0457351Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0457419Z outputs = self.model( 2025-12-04T09:48:33.0457656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0457739Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0457952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0458028Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0458270Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0458340Z return func(*args, **kwargs) 2025-12-04T09:48:33.0458582Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0458674Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0458921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0458993Z return func(*args, **kwargs) 2025-12-04T09:48:33.0459242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0459353Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0459356Z 2025-12-04T09:48:33.0459451Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0459630Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0459711Z res = mod(**inputs) 2025-12-04T09:48:33.0459943Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0460005Z outputs = self.model( 2025-12-04T09:48:33.0460245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0460328Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0460545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0460617Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0460844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0460917Z return func(*args, **kwargs) 2025-12-04T09:48:33.0461151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0461241Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0461477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0461540Z return func(*args, **kwargs) 2025-12-04T09:48:33.0461781Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0461856Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0461859Z 2025-12-04T09:48:33.0461957Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0462154Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0462212Z res = mod(**inputs) 2025-12-04T09:48:33.0462454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0462518Z outputs = self.model( 2025-12-04T09:48:33.0462748Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0462824Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0463031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0463107Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0463345Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0463410Z return func(*args, **kwargs) 2025-12-04T09:48:33.0463653Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0463743Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0463971Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0464046Z return func(*args, **kwargs) 2025-12-04T09:48:33.0464281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0464411Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0464416Z 2025-12-04T09:48:33.0464506Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0464604Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0464810Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0464870Z res = mod(**inputs) 2025-12-04T09:48:33.0465100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0465168Z outputs = self.model( 2025-12-04T09:48:33.0465396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0465486Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0465691Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0465777Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0466016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0466078Z return func(*args, **kwargs) 2025-12-04T09:48:33.0466322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0466420Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0466649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0466722Z return func(*args, **kwargs) 2025-12-04T09:48:33.0466960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0467041Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0467044Z 2025-12-04T09:48:33.0467146Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0467333Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0467400Z res = mod(**inputs) 2025-12-04T09:48:33.0467636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0467699Z outputs = self.model( 2025-12-04T09:48:33.0467938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0468004Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0468217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0468297Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0468526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0468599Z return func(*args, **kwargs) 2025-12-04T09:48:33.0468835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0468923Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0469162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0469226Z return func(*args, **kwargs) 2025-12-04T09:48:33.0469459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0469556Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0469559Z 2025-12-04T09:48:33.0469653Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0469843Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0469903Z res = mod(**inputs) 2025-12-04T09:48:33.0470164Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0470237Z outputs = self.model( 2025-12-04T09:48:33.0470485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0470560Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0470766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0470838Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0471091Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0471154Z return func(*args, **kwargs) 2025-12-04T09:48:33.0471384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0471498Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0471730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0471800Z return func(*args, **kwargs) 2025-12-04T09:48:33.0472029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0472144Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0472147Z 2025-12-04T09:48:33.0472228Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0472301Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0472400Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0472583Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0472640Z res = mod(**inputs) 2025-12-04T09:48:33.0472937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0473012Z outputs = self.model( 2025-12-04T09:48:33.0473247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0473325Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0473536Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0473615Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0473844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0473909Z return func(*args, **kwargs) 2025-12-04T09:48:33.0474154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0474267Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0474477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0474552Z return self.act(input) 2025-12-04T09:48:33.0474556Z 2025-12-04T09:48:33.0474629Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0474707Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0474778Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0474874Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0475069Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0475132Z res = mod(**inputs) 2025-12-04T09:48:33.0475374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0475444Z outputs = self.model( 2025-12-04T09:48:33.0475680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0475770Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0475981Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0476071Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0476306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0476370Z return func(*args, **kwargs) 2025-12-04T09:48:33.0476602Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0476713Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0476941Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0477011Z return func(*args, **kwargs) 2025-12-04T09:48:33.0477274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0477378Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0477382Z 2025-12-04T09:48:33.0477487Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0477670Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0477738Z res = mod(**inputs) 2025-12-04T09:48:33.0477972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0478036Z outputs = self.model( 2025-12-04T09:48:33.0478278Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0478344Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0478551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0478634Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0478864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0478935Z return func(*args, **kwargs) 2025-12-04T09:48:33.0479176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0479266Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0479502Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0479568Z return func(*args, **kwargs) 2025-12-04T09:48:33.0479809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0479884Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0479888Z 2025-12-04T09:48:33.0479984Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0480174Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0480234Z res = mod(**inputs) 2025-12-04T09:48:33.0480469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0480539Z outputs = self.model( 2025-12-04T09:48:33.0480772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0480849Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0481057Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0481128Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0481363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0481443Z return func(*args, **kwargs) 2025-12-04T09:48:33.0481677Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0481788Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0482019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0482093Z return func(*args, **kwargs) 2025-12-04T09:48:33.0482325Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0482463Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0482466Z 2025-12-04T09:48:33.0482547Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0482642Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0483523Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0483584Z res = mod(**inputs) 2025-12-04T09:48:33.0483818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0483888Z outputs = self.model( 2025-12-04T09:48:33.0484120Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0484187Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0484407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0484480Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0484715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0484781Z return func(*args, **kwargs) 2025-12-04T09:48:33.0485016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0485111Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0485339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0485402Z return func(*args, **kwargs) 2025-12-04T09:48:33.0485644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0485724Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0485727Z 2025-12-04T09:48:33.0485830Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0486013Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0486071Z res = mod(**inputs) 2025-12-04T09:48:33.0486311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0486372Z outputs = self.model( 2025-12-04T09:48:33.0486612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0486679Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0486888Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0486968Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0487198Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0487262Z return func(*args, **kwargs) 2025-12-04T09:48:33.0487506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0487597Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0487848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0487913Z return func(*args, **kwargs) 2025-12-04T09:48:33.0488163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0488261Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0488264Z 2025-12-04T09:48:33.0488359Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0488569Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0488629Z res = mod(**inputs) 2025-12-04T09:48:33.0488864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0488934Z outputs = self.model( 2025-12-04T09:48:33.0489193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0489259Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0489482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0489555Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0489793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0489857Z return func(*args, **kwargs) 2025-12-04T09:48:33.0490090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0490188Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0490416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0490480Z return func(*args, **kwargs) 2025-12-04T09:48:33.0490723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0490845Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0490848Z 2025-12-04T09:48:33.0490928Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0491001Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0491097Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0491292Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0491353Z res = mod(**inputs) 2025-12-04T09:48:33.0491597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0491662Z outputs = self.model( 2025-12-04T09:48:33.0491897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0491972Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0492183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0492257Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0492494Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0492558Z return func(*args, **kwargs) 2025-12-04T09:48:33.0492797Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0492910Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0493110Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0493183Z return self.act(input) 2025-12-04T09:48:33.0493186Z 2025-12-04T09:48:33.0493256Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0493340Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0493418Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0493527Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0493718Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0493777Z res = mod(**inputs) 2025-12-04T09:48:33.0494010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0494093Z outputs = self.model( 2025-12-04T09:48:33.0494328Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0494395Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0494612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0494705Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0494932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0494997Z return func(*args, **kwargs) 2025-12-04T09:48:33.0495226Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0495323Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0495546Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0495620Z return func(*args, **kwargs) 2025-12-04T09:48:33.0495846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0495948Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0495952Z 2025-12-04T09:48:33.0496056Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0496240Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0496298Z res = mod(**inputs) 2025-12-04T09:48:33.0496535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0496599Z outputs = self.model( 2025-12-04T09:48:33.0496834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0496903Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0497109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0497190Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0497412Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0497477Z return func(*args, **kwargs) 2025-12-04T09:48:33.0497712Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0497803Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0498034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0498096Z return func(*args, **kwargs) 2025-12-04T09:48:33.0498325Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0498408Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0498411Z 2025-12-04T09:48:33.0498505Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0498694Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0498755Z res = mod(**inputs) 2025-12-04T09:48:33.0515760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0515956Z outputs = self.model( 2025-12-04T09:48:33.0516311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0516390Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0516630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0516749Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0516990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0517071Z return func(*args, **kwargs) 2025-12-04T09:48:33.0517309Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0517454Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0517697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0517764Z return func(*args, **kwargs) 2025-12-04T09:48:33.0518008Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0518141Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0518150Z 2025-12-04T09:48:33.0518232Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0518342Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0518537Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0518608Z res = mod(**inputs) 2025-12-04T09:48:33.0518852Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0518921Z outputs = self.model( 2025-12-04T09:48:33.0519170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0519242Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0519459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0519544Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0519778Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0519851Z return func(*args, **kwargs) 2025-12-04T09:48:33.0520088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0520182Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0520425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0520490Z return func(*args, **kwargs) 2025-12-04T09:48:33.0520734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0520818Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0520822Z 2025-12-04T09:48:33.0520923Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0521124Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0521189Z res = mod(**inputs) 2025-12-04T09:48:33.0521424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0521498Z outputs = self.model( 2025-12-04T09:48:33.0521749Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0521826Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0522076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0522153Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0522392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0522458Z return func(*args, **kwargs) 2025-12-04T09:48:33.0522716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0522807Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0523035Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0523122Z return func(*args, **kwargs) 2025-12-04T09:48:33.0523359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0523450Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0523455Z 2025-12-04T09:48:33.0523558Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0523751Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0523819Z res = mod(**inputs) 2025-12-04T09:48:33.0524058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0524125Z outputs = self.model( 2025-12-04T09:48:33.0524368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0524434Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0524652Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0524734Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0524975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0525048Z return func(*args, **kwargs) 2025-12-04T09:48:33.0525285Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0525376Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0525616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0525680Z return func(*args, **kwargs) 2025-12-04T09:48:33.0525926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0526050Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0526055Z 2025-12-04T09:48:33.0526133Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0526215Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0526315Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0526508Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0526576Z res = mod(**inputs) 2025-12-04T09:48:33.0526818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0526895Z outputs = self.model( 2025-12-04T09:48:33.0527135Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0527202Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0527423Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0527514Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0527743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0527829Z return func(*args, **kwargs) 2025-12-04T09:48:33.0528064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0528189Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0528410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0528477Z return self.act(input) 2025-12-04T09:48:33.0528480Z 2025-12-04T09:48:33.0528561Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0528635Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0528730Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0528827Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0529014Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0529082Z res = mod(**inputs) 2025-12-04T09:48:33.0529321Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0529386Z outputs = self.model( 2025-12-04T09:48:33.0529631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0529700Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0529917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0529992Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0530223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0530300Z return func(*args, **kwargs) 2025-12-04T09:48:33.0530536Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0530630Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0530870Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0530935Z return func(*args, **kwargs) 2025-12-04T09:48:33.0531182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0531293Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0531297Z 2025-12-04T09:48:33.0531391Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0531588Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0531653Z res = mod(**inputs) 2025-12-04T09:48:33.0531898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0531964Z outputs = self.model( 2025-12-04T09:48:33.0532202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0532279Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0532488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0532563Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0532799Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0532866Z return func(*args, **kwargs) 2025-12-04T09:48:33.0533105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0533215Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0533458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0533532Z return func(*args, **kwargs) 2025-12-04T09:48:33.0533764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0533846Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0533849Z 2025-12-04T09:48:33.0533958Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0534141Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0534206Z res = mod(**inputs) 2025-12-04T09:48:33.0534438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0534517Z outputs = self.model( 2025-12-04T09:48:33.0534764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0534831Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0535058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0535131Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0535363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0535435Z return func(*args, **kwargs) 2025-12-04T09:48:33.0535667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0535756Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0535994Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0536059Z return func(*args, **kwargs) 2025-12-04T09:48:33.0536300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0536426Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0536430Z 2025-12-04T09:48:33.0536501Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0536603Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0536790Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0536859Z res = mod(**inputs) 2025-12-04T09:48:33.0537092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0537156Z outputs = self.model( 2025-12-04T09:48:33.0537401Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0537467Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0537676Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0537756Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0537988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0538057Z return func(*args, **kwargs) 2025-12-04T09:48:33.0538288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0538378Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0538613Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0538677Z return func(*args, **kwargs) 2025-12-04T09:48:33.0538923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0539014Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0539018Z 2025-12-04T09:48:33.0539124Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0539320Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0539378Z res = mod(**inputs) 2025-12-04T09:48:33.0539608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0539692Z outputs = self.model( 2025-12-04T09:48:33.0539922Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0539995Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0540222Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0540294Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0540528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0540591Z return func(*args, **kwargs) 2025-12-04T09:48:33.0540821Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0540918Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0541150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0541219Z return func(*args, **kwargs) 2025-12-04T09:48:33.0541451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0541541Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0541544Z 2025-12-04T09:48:33.0541645Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0541831Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0541898Z res = mod(**inputs) 2025-12-04T09:48:33.0542132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0542195Z outputs = self.model( 2025-12-04T09:48:33.0542437Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0542506Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0542714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0542794Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0543027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0543098Z return func(*args, **kwargs) 2025-12-04T09:48:33.0543334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0543423Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0543659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0543721Z return func(*args, **kwargs) 2025-12-04T09:48:33.0543957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0544086Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0544089Z 2025-12-04T09:48:33.0544164Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0544246Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0544365Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0544553Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0544635Z res = mod(**inputs) 2025-12-04T09:48:33.0544870Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0544940Z outputs = self.model( 2025-12-04T09:48:33.0545173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0545255Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0545473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0545543Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0545774Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0545863Z return func(*args, **kwargs) 2025-12-04T09:48:33.0546098Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0546222Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0546429Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0546493Z return self.act(input) 2025-12-04T09:48:33.0546496Z 2025-12-04T09:48:33.0546578Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0546648Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0546718Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0546821Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0547004Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0547074Z res = mod(**inputs) 2025-12-04T09:48:33.0547309Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0547372Z outputs = self.model( 2025-12-04T09:48:33.0547613Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0547679Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0547888Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0547969Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0548202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0548273Z return func(*args, **kwargs) 2025-12-04T09:48:33.0548508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0548602Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0548841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0548907Z return func(*args, **kwargs) 2025-12-04T09:48:33.0549147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0549254Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0549257Z 2025-12-04T09:48:33.0549353Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0549545Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0549607Z res = mod(**inputs) 2025-12-04T09:48:33.0549839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0549910Z outputs = self.model( 2025-12-04T09:48:33.0550158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0550231Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0550453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0550526Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0550764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0550844Z return func(*args, **kwargs) 2025-12-04T09:48:33.0551075Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0551174Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0551418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0551490Z return func(*args, **kwargs) 2025-12-04T09:48:33.0551723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0551796Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0551799Z 2025-12-04T09:48:33.0551899Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0552083Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0552149Z res = mod(**inputs) 2025-12-04T09:48:33.0552379Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0552441Z outputs = self.model( 2025-12-04T09:48:33.0552680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0552745Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0553045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0553130Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0553363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0553435Z return func(*args, **kwargs) 2025-12-04T09:48:33.0553671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0553763Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0554001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0554063Z return func(*args, **kwargs) 2025-12-04T09:48:33.0554308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0554436Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0554440Z 2025-12-04T09:48:33.0554512Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0554618Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0554808Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0554868Z res = mod(**inputs) 2025-12-04T09:48:33.0555113Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0555178Z outputs = self.model( 2025-12-04T09:48:33.0555421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0555486Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0555716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0555796Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0556037Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0556103Z return func(*args, **kwargs) 2025-12-04T09:48:33.0556342Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0556431Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0556682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0556745Z return func(*args, **kwargs) 2025-12-04T09:48:33.0556975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0557080Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0557083Z 2025-12-04T09:48:33.0557179Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0557373Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0557436Z res = mod(**inputs) 2025-12-04T09:48:33.0557670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0557742Z outputs = self.model( 2025-12-04T09:48:33.0557977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0558046Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0558263Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0558338Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0558583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0558649Z return func(*args, **kwargs) 2025-12-04T09:48:33.0558888Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0558987Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0559217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0559283Z return func(*args, **kwargs) 2025-12-04T09:48:33.0559526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0559616Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0559619Z 2025-12-04T09:48:33.0559722Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0559907Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0559971Z res = mod(**inputs) 2025-12-04T09:48:33.0560214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0560280Z outputs = self.model( 2025-12-04T09:48:33.0560522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0560590Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0560800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0560884Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0561116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0561181Z return func(*args, **kwargs) 2025-12-04T09:48:33.0561439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0561530Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0561779Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0561844Z return func(*args, **kwargs) 2025-12-04T09:48:33.0562078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0562207Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0562224Z 2025-12-04T09:48:33.0562298Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0562379Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0562475Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0562665Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0562753Z res = mod(**inputs) 2025-12-04T09:48:33.0562985Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0563049Z outputs = self.model( 2025-12-04T09:48:33.0563293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0563361Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0563575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0563648Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0563873Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0563943Z return func(*args, **kwargs) 2025-12-04T09:48:33.0564176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0564291Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0564498Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0564562Z return self.act(input) 2025-12-04T09:48:33.0564565Z 2025-12-04T09:48:33.0564644Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0564714Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0564785Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0564889Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0565073Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0565133Z res = mod(**inputs) 2025-12-04T09:48:33.0565373Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0565436Z outputs = self.model( 2025-12-04T09:48:33.0565673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0565740Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0565945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0566024Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0566250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0566315Z return func(*args, **kwargs) 2025-12-04T09:48:33.0566551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0566641Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0566872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0566950Z return func(*args, **kwargs) 2025-12-04T09:48:33.0567186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0567313Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0567316Z 2025-12-04T09:48:33.0567413Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0567601Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0567679Z res = mod(**inputs) 2025-12-04T09:48:33.0567912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0567981Z outputs = self.model( 2025-12-04T09:48:33.0568211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0568293Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0568510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0568582Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0568819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0568882Z return func(*args, **kwargs) 2025-12-04T09:48:33.0569118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0569218Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0569448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0569517Z return func(*args, **kwargs) 2025-12-04T09:48:33.0569751Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0569827Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0569830Z 2025-12-04T09:48:33.0569934Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0570121Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0570181Z res = mod(**inputs) 2025-12-04T09:48:33.0570422Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0570487Z outputs = self.model( 2025-12-04T09:48:33.0570728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0570795Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0571007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0571088Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0571318Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0571382Z return func(*args, **kwargs) 2025-12-04T09:48:33.0571623Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0571711Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0571947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0572012Z return func(*args, **kwargs) 2025-12-04T09:48:33.0572246Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0572379Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0572384Z 2025-12-04T09:48:33.0572456Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0572573Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0572759Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0572833Z res = mod(**inputs) 2025-12-04T09:48:33.0573072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0573147Z outputs = self.model( 2025-12-04T09:48:33.0573379Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0573469Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0573684Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0573756Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0574001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0574071Z return func(*args, **kwargs) 2025-12-04T09:48:33.0574306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0574404Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0574631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0574693Z return func(*args, **kwargs) 2025-12-04T09:48:33.0574937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0575015Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0575018Z 2025-12-04T09:48:33.0575118Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0575304Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0575366Z res = mod(**inputs) 2025-12-04T09:48:33.0575605Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0575670Z outputs = self.model( 2025-12-04T09:48:33.0575902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0575976Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0576184Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0576262Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0576488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0576551Z return func(*args, **kwargs) 2025-12-04T09:48:33.0576790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0576878Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0577112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0577175Z return func(*args, **kwargs) 2025-12-04T09:48:33.0577407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0577502Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0577507Z 2025-12-04T09:48:33.0577599Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0577780Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0577846Z res = mod(**inputs) 2025-12-04T09:48:33.0578081Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0578166Z outputs = self.model( 2025-12-04T09:48:33.0578400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0578497Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0578712Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0578784Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0579009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0579096Z return func(*args, **kwargs) 2025-12-04T09:48:33.0579329Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0579425Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0579665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0579728Z return func(*args, **kwargs) 2025-12-04T09:48:33.0579966Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0580086Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0580089Z 2025-12-04T09:48:33.0580167Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0580238Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0580335Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0580525Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0580583Z res = mod(**inputs) 2025-12-04T09:48:33.0580814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0580885Z outputs = self.model( 2025-12-04T09:48:33.0581118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0581193Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0581404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0581476Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0581709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0581773Z return func(*args, **kwargs) 2025-12-04T09:48:33.0582008Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0582131Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0582335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0582406Z return self.act(input) 2025-12-04T09:48:33.0582409Z 2025-12-04T09:48:33.0582482Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0582555Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0582634Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0582727Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0582910Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0582981Z res = mod(**inputs) 2025-12-04T09:48:33.0583216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0583287Z outputs = self.model( 2025-12-04T09:48:33.0583519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0583587Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0583819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0583892Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0584133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0584205Z return func(*args, **kwargs) 2025-12-04T09:48:33.0584435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0584554Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0584782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0584845Z return func(*args, **kwargs) 2025-12-04T09:48:33.0585084Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0585205Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0585209Z 2025-12-04T09:48:33.0585309Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0585495Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0585555Z res = mod(**inputs) 2025-12-04T09:48:33.0585802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0585866Z outputs = self.model( 2025-12-04T09:48:33.0586097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0586171Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0586388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0586470Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0586699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0586762Z return func(*args, **kwargs) 2025-12-04T09:48:33.0587005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0587095Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0587329Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0587394Z return func(*args, **kwargs) 2025-12-04T09:48:33.0587626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0587707Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0587712Z 2025-12-04T09:48:33.0587807Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0587992Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0588059Z res = mod(**inputs) 2025-12-04T09:48:33.0588292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0588363Z outputs = self.model( 2025-12-04T09:48:33.0588599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0588666Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0588880Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0588950Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0589176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0589247Z return func(*args, **kwargs) 2025-12-04T09:48:33.0589494Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0589603Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0589833Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0589896Z return func(*args, **kwargs) 2025-12-04T09:48:33.0590135Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0590273Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0590276Z 2025-12-04T09:48:33.0590355Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0590448Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0590649Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0590717Z res = mod(**inputs) 2025-12-04T09:48:33.0590950Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0591013Z outputs = self.model( 2025-12-04T09:48:33.0591254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0591319Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0591531Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0591605Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0591835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0591905Z return func(*args, **kwargs) 2025-12-04T09:48:33.0592142Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0592236Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0592465Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0592528Z return func(*args, **kwargs) 2025-12-04T09:48:33.0592768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0592954Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0592962Z 2025-12-04T09:48:33.0593065Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0593254Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0593314Z res = mod(**inputs) 2025-12-04T09:48:33.0593552Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0593618Z outputs = self.model( 2025-12-04T09:48:33.0593849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0593926Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0594134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0594205Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0594442Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0594505Z return func(*args, **kwargs) 2025-12-04T09:48:33.0594746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0594836Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0595081Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0595155Z return func(*args, **kwargs) 2025-12-04T09:48:33.0595405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0595503Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0595506Z 2025-12-04T09:48:33.0595599Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0595783Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0595865Z res = mod(**inputs) 2025-12-04T09:48:33.0596098Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0596160Z outputs = self.model( 2025-12-04T09:48:33.0596404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0596488Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0596701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0596773Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0596993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0597065Z return func(*args, **kwargs) 2025-12-04T09:48:33.0597291Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0597382Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0597610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0597676Z return func(*args, **kwargs) 2025-12-04T09:48:33.0597911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0598027Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0598030Z 2025-12-04T09:48:33.0598104Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0598186Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0598279Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0598465Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0598526Z res = mod(**inputs) 2025-12-04T09:48:33.0598755Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0598825Z outputs = self.model( 2025-12-04T09:48:33.0599052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0599120Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0599333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0599407Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0599636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0599698Z return func(*args, **kwargs) 2025-12-04T09:48:33.0599924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0600045Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0600244Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0600308Z return self.act(input) 2025-12-04T09:48:33.0600320Z 2025-12-04T09:48:33.0600392Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0600477Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0600556Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0600651Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0600849Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0600917Z res = mod(**inputs) 2025-12-04T09:48:33.0601154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0601216Z outputs = self.model( 2025-12-04T09:48:33.0601472Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0601540Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0601758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0601846Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0602079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0602152Z return func(*args, **kwargs) 2025-12-04T09:48:33.0602389Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0602487Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0602714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0602779Z return func(*args, **kwargs) 2025-12-04T09:48:33.0603018Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0603121Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0603125Z 2025-12-04T09:48:33.0603220Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0603413Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0603473Z res = mod(**inputs) 2025-12-04T09:48:33.0603716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0603778Z outputs = self.model( 2025-12-04T09:48:33.0604009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0604085Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0604294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0604370Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0604597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0604662Z return func(*args, **kwargs) 2025-12-04T09:48:33.0604903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0604993Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0605222Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0605292Z return func(*args, **kwargs) 2025-12-04T09:48:33.0605524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0605607Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0605610Z 2025-12-04T09:48:33.0605879Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0606069Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0606141Z res = mod(**inputs) 2025-12-04T09:48:33.0606411Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0606477Z outputs = self.model( 2025-12-04T09:48:33.0606751Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0606818Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0607035Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0607141Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0607372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0607447Z return func(*args, **kwargs) 2025-12-04T09:48:33.0607688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0607808Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0608037Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0608101Z return func(*args, **kwargs) 2025-12-04T09:48:33.0608342Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0608466Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0608469Z 2025-12-04T09:48:33.0608545Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0608646Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0608831Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0608897Z res = mod(**inputs) 2025-12-04T09:48:33.0609129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0609195Z outputs = self.model( 2025-12-04T09:48:33.0609439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0609507Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0609718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0609799Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0610026Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0610100Z return func(*args, **kwargs) 2025-12-04T09:48:33.0610334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0610424Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0610670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0610733Z return func(*args, **kwargs) 2025-12-04T09:48:33.0610968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0611052Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0611055Z 2025-12-04T09:48:33.0611149Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0611338Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0611398Z res = mod(**inputs) 2025-12-04T09:48:33.0611631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0611699Z outputs = self.model( 2025-12-04T09:48:33.0611932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0612020Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0612230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0612316Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0612551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0612613Z return func(*args, **kwargs) 2025-12-04T09:48:33.0612843Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0612952Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0613177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0613261Z return func(*args, **kwargs) 2025-12-04T09:48:33.0613496Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0613584Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0613587Z 2025-12-04T09:48:33.0613689Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0613872Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0613936Z res = mod(**inputs) 2025-12-04T09:48:33.0614168Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0614231Z outputs = self.model( 2025-12-04T09:48:33.0614467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0614532Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0614740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0614819Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0615048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0615118Z return func(*args, **kwargs) 2025-12-04T09:48:33.0615348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0615435Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0615668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0615728Z return func(*args, **kwargs) 2025-12-04T09:48:33.0615958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0616080Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0616084Z 2025-12-04T09:48:33.0616157Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0616234Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0616326Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0616508Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0616574Z res = mod(**inputs) 2025-12-04T09:48:33.0616807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0616875Z outputs = self.model( 2025-12-04T09:48:33.0617109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0617175Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0617388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0617461Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0617703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0617787Z return func(*args, **kwargs) 2025-12-04T09:48:33.0618019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0618134Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0618334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0618413Z return self.act(input) 2025-12-04T09:48:33.0618416Z 2025-12-04T09:48:33.0618495Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0618565Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0618634Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0618750Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0618936Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0619001Z res = mod(**inputs) 2025-12-04T09:48:33.0619237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0619297Z outputs = self.model( 2025-12-04T09:48:33.0619534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0619599Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0619806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0619884Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0620111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0620180Z return func(*args, **kwargs) 2025-12-04T09:48:33.0620413Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0620502Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0620738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0620800Z return func(*args, **kwargs) 2025-12-04T09:48:33.0621048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0621153Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0621156Z 2025-12-04T09:48:33.0621249Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0621439Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0621499Z res = mod(**inputs) 2025-12-04T09:48:33.0621731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0621799Z outputs = self.model( 2025-12-04T09:48:33.0622031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0622104Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0622313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0622384Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0622617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0622680Z return func(*args, **kwargs) 2025-12-04T09:48:33.0622911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0623020Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0623246Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0623327Z return func(*args, **kwargs) 2025-12-04T09:48:33.0623559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0623632Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0623635Z 2025-12-04T09:48:33.0623736Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0623933Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0623996Z res = mod(**inputs) 2025-12-04T09:48:33.0624226Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0624301Z outputs = self.model( 2025-12-04T09:48:33.0624540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0624606Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0624813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0624891Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0625119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0625189Z return func(*args, **kwargs) 2025-12-04T09:48:33.0625428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0625515Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0625745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0625810Z return func(*args, **kwargs) 2025-12-04T09:48:33.0626045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0626169Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0626172Z 2025-12-04T09:48:33.0626242Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0626341Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0626525Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0626586Z res = mod(**inputs) 2025-12-04T09:48:33.0626826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0626888Z outputs = self.model( 2025-12-04T09:48:33.0627125Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0627194Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0627403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0627480Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0627707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0627770Z return func(*args, **kwargs) 2025-12-04T09:48:33.0628007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0628098Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0628333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0628395Z return func(*args, **kwargs) 2025-12-04T09:48:33.0628650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0628737Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0628740Z 2025-12-04T09:48:33.0628844Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0629034Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0629093Z res = mod(**inputs) 2025-12-04T09:48:33.0629324Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0629405Z outputs = self.model( 2025-12-04T09:48:33.0629642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0629707Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0629924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0630014Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0630250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0630314Z return func(*args, **kwargs) 2025-12-04T09:48:33.0630545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0630638Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0630864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0630928Z return func(*args, **kwargs) 2025-12-04T09:48:33.0631166Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0631255Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0631258Z 2025-12-04T09:48:33.0631358Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0631541Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0631600Z res = mod(**inputs) 2025-12-04T09:48:33.0631844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0631906Z outputs = self.model( 2025-12-04T09:48:33.0632144Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0632211Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0632419Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0632496Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0632730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0632793Z return func(*args, **kwargs) 2025-12-04T09:48:33.0633097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0633191Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0633419Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0633483Z return func(*args, **kwargs) 2025-12-04T09:48:33.0633711Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0633834Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0633838Z 2025-12-04T09:48:33.0633910Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0633988Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0634085Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0634283Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0634350Z res = mod(**inputs) 2025-12-04T09:48:33.0634600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0634663Z outputs = self.model( 2025-12-04T09:48:33.0634901Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0634983Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0635196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0635268Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0635497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0635585Z return func(*args, **kwargs) 2025-12-04T09:48:33.0635819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0635932Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0636140Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0636205Z return self.act(input) 2025-12-04T09:48:33.0636208Z 2025-12-04T09:48:33.0636284Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0636357Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0636426Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0636526Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0636708Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0636769Z res = mod(**inputs) 2025-12-04T09:48:33.0637014Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0637077Z outputs = self.model( 2025-12-04T09:48:33.0637314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0637380Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0637597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0637676Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0637910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0637973Z return func(*args, **kwargs) 2025-12-04T09:48:33.0638212Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0638301Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0638534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0638596Z return func(*args, **kwargs) 2025-12-04T09:48:33.0638833Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0638944Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0638948Z 2025-12-04T09:48:33.0639041Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0639231Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0639289Z res = mod(**inputs) 2025-12-04T09:48:33.0639522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0639594Z outputs = self.model( 2025-12-04T09:48:33.0639839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0639905Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0640132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0640204Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0640438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0640516Z return func(*args, **kwargs) 2025-12-04T09:48:33.0640741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0640837Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0641058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0641144Z return func(*args, **kwargs) 2025-12-04T09:48:33.0641370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0641441Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0641444Z 2025-12-04T09:48:33.0641543Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0641726Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0641785Z res = mod(**inputs) 2025-12-04T09:48:33.0642019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0642080Z outputs = self.model( 2025-12-04T09:48:33.0642313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0642380Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0642585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0642660Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0642882Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0642945Z return func(*args, **kwargs) 2025-12-04T09:48:33.0643177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0643264Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0643489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0643551Z return func(*args, **kwargs) 2025-12-04T09:48:33.0643774Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0643905Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0643908Z 2025-12-04T09:48:33.0643977Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0644074Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0644254Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0644312Z res = mod(**inputs) 2025-12-04T09:48:33.0644544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0644607Z outputs = self.model( 2025-12-04T09:48:33.0644832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0644903Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0645106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0645198Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0645423Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0645499Z return func(*args, **kwargs) 2025-12-04T09:48:33.0645737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0645825Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0646058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0646137Z return func(*args, **kwargs) 2025-12-04T09:48:33.0646369Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0646474Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0646477Z 2025-12-04T09:48:33.0646572Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0646753Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0646820Z res = mod(**inputs) 2025-12-04T09:48:33.0647052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0647122Z outputs = self.model( 2025-12-04T09:48:33.0647352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0647420Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0647636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0647705Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0647929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0648001Z return func(*args, **kwargs) 2025-12-04T09:48:33.0648233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0648328Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0648555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0648619Z return func(*args, **kwargs) 2025-12-04T09:48:33.0648858Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0648947Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0648950Z 2025-12-04T09:48:33.0649045Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0649224Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0649285Z res = mod(**inputs) 2025-12-04T09:48:33.0649525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0649591Z outputs = self.model( 2025-12-04T09:48:33.0649825Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0649897Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0650102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0650180Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0650404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0650465Z return func(*args, **kwargs) 2025-12-04T09:48:33.0650704Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0650846Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0651090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0651160Z return func(*args, **kwargs) 2025-12-04T09:48:33.0651391Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0651512Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0651531Z 2025-12-04T09:48:33.0651604Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0651674Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0651776Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0651963Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0652045Z res = mod(**inputs) 2025-12-04T09:48:33.0652274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0652336Z outputs = self.model( 2025-12-04T09:48:33.0652570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0652634Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0652839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0652919Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0653141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0653210Z return func(*args, **kwargs) 2025-12-04T09:48:33.0653436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0653550Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0653756Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0653821Z return self.act(input) 2025-12-04T09:48:33.0653824Z 2025-12-04T09:48:33.0653902Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0653972Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0654040Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0654138Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0654318Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0654376Z res = mod(**inputs) 2025-12-04T09:48:33.0654616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0654679Z outputs = self.model( 2025-12-04T09:48:33.0654908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0654980Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0655189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0655265Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0655487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0655551Z return func(*args, **kwargs) 2025-12-04T09:48:33.0655789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0655879Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0656109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0656174Z return func(*args, **kwargs) 2025-12-04T09:48:33.0656413Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0656538Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0656541Z 2025-12-04T09:48:33.0656633Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0656816Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0656880Z res = mod(**inputs) 2025-12-04T09:48:33.0657126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0657193Z outputs = self.model( 2025-12-04T09:48:33.0657433Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0657513Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0657730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0657801Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0658032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0658102Z return func(*args, **kwargs) 2025-12-04T09:48:33.0658333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0658429Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0658657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0658719Z return func(*args, **kwargs) 2025-12-04T09:48:33.0658958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0659035Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0659038Z 2025-12-04T09:48:33.0659139Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0659323Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0659382Z res = mod(**inputs) 2025-12-04T09:48:33.0659622Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0659683Z outputs = self.model( 2025-12-04T09:48:33.0659915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0659987Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0660195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0660275Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0660505Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0660568Z return func(*args, **kwargs) 2025-12-04T09:48:33.0660808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0660896Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0661129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0661192Z return func(*args, **kwargs) 2025-12-04T09:48:33.0661422Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0661553Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0661557Z 2025-12-04T09:48:33.0661631Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0661743Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0661932Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0661989Z res = mod(**inputs) 2025-12-04T09:48:33.0662243Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0662305Z outputs = self.model( 2025-12-04T09:48:33.0662534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0662625Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0662828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0662901Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0663129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0663207Z return func(*args, **kwargs) 2025-12-04T09:48:33.0663444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0663535Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0663767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0663838Z return func(*args, **kwargs) 2025-12-04T09:48:33.0664069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0664156Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0664159Z 2025-12-04T09:48:33.0664252Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0664435Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0664501Z res = mod(**inputs) 2025-12-04T09:48:33.0664760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0664822Z outputs = self.model( 2025-12-04T09:48:33.0665060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0665126Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0665343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0665415Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0665640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0665708Z return func(*args, **kwargs) 2025-12-04T09:48:33.0665941Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0666037Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0666269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0666331Z return func(*args, **kwargs) 2025-12-04T09:48:33.0666570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0666658Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0666662Z 2025-12-04T09:48:33.0666758Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0666948Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0667007Z res = mod(**inputs) 2025-12-04T09:48:33.0667244Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0667309Z outputs = self.model( 2025-12-04T09:48:33.0667576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0667663Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0667871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0667941Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0668175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0668259Z return func(*args, **kwargs) 2025-12-04T09:48:33.0668497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0668586Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0668830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0668901Z return func(*args, **kwargs) 2025-12-04T09:48:33.0669134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0669254Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0669257Z 2025-12-04T09:48:33.0669328Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0669399Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0669499Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0669682Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0669741Z res = mod(**inputs) 2025-12-04T09:48:33.0669981Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0670044Z outputs = self.model( 2025-12-04T09:48:33.0670283Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0670347Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0670557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0670634Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0670858Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0670923Z return func(*args, **kwargs) 2025-12-04T09:48:33.0671162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0671273Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0671481Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0671545Z return self.act(input) 2025-12-04T09:48:33.0671549Z 2025-12-04T09:48:33.0671620Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0671696Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0671767Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0671861Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0672048Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0672106Z res = mod(**inputs) 2025-12-04T09:48:33.0672344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0672405Z outputs = self.model( 2025-12-04T09:48:33.0672635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0672710Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0673015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0673100Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0673362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0673428Z return func(*args, **kwargs) 2025-12-04T09:48:33.0673673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0673788Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0674017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0674087Z return func(*args, **kwargs) 2025-12-04T09:48:33.0674318Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0674448Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0674452Z 2025-12-04T09:48:33.0674544Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0674730Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0674797Z res = mod(**inputs) 2025-12-04T09:48:33.0675032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0675094Z outputs = self.model( 2025-12-04T09:48:33.0675338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0675405Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0675624Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0675697Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0675931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0676003Z return func(*args, **kwargs) 2025-12-04T09:48:33.0676236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0676332Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0676564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0676627Z return func(*args, **kwargs) 2025-12-04T09:48:33.0676870Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0676943Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0676946Z 2025-12-04T09:48:33.0677043Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0677235Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0677292Z res = mod(**inputs) 2025-12-04T09:48:33.0677536Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0677599Z outputs = self.model( 2025-12-04T09:48:33.0677832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0677904Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0678114Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0678185Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0678423Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0678487Z return func(*args, **kwargs) 2025-12-04T09:48:33.0678742Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0678832Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0679099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0679171Z return func(*args, **kwargs) 2025-12-04T09:48:33.0679402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0679549Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0679552Z 2025-12-04T09:48:33.0679624Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0679717Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0679908Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0679982Z res = mod(**inputs) 2025-12-04T09:48:33.0680219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0680287Z outputs = self.model( 2025-12-04T09:48:33.0680525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0680596Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0680806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0680879Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0681116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0681179Z return func(*args, **kwargs) 2025-12-04T09:48:33.0681421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0681513Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0681747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0681816Z return func(*args, **kwargs) 2025-12-04T09:48:33.0682054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0682133Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0682137Z 2025-12-04T09:48:33.0682238Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0682426Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0682490Z res = mod(**inputs) 2025-12-04T09:48:33.0682727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0682789Z outputs = self.model( 2025-12-04T09:48:33.0683031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0683095Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0683308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0683385Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0683616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0683686Z return func(*args, **kwargs) 2025-12-04T09:48:33.0683923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0684012Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0684264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0684327Z return func(*args, **kwargs) 2025-12-04T09:48:33.0684581Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0684670Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0684673Z 2025-12-04T09:48:33.0684764Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0684950Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0685024Z res = mod(**inputs) 2025-12-04T09:48:33.0685257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0685325Z outputs = self.model( 2025-12-04T09:48:33.0685557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0685643Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0685852Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0685924Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0686159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0686222Z return func(*args, **kwargs) 2025-12-04T09:48:33.0686453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0686550Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0686780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0686849Z return func(*args, **kwargs) 2025-12-04T09:48:33.0687088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0687205Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0687210Z 2025-12-04T09:48:33.0687290Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0687362Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0687461Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0687644Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0687703Z res = mod(**inputs) 2025-12-04T09:48:33.0687941Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0688005Z outputs = self.model( 2025-12-04T09:48:33.0688236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0688312Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0688520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0688598Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0688826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0688888Z return func(*args, **kwargs) 2025-12-04T09:48:33.0689130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0689243Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0689446Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0689515Z return self.act(input) 2025-12-04T09:48:33.0689519Z 2025-12-04T09:48:33.0689590Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0689667Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0689751Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0689847Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0690058Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0690117Z res = mod(**inputs) 2025-12-04T09:48:33.0690343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0690411Z outputs = self.model( 2025-12-04T09:48:33.0690657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0690730Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0690939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0691040Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0691279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0691342Z return func(*args, **kwargs) 2025-12-04T09:48:33.0691582Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0691672Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0691898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0691969Z return func(*args, **kwargs) 2025-12-04T09:48:33.0692204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0692309Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0692321Z 2025-12-04T09:48:33.0692417Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0692598Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0692665Z res = mod(**inputs) 2025-12-04T09:48:33.0692899Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0692960Z outputs = self.model( 2025-12-04T09:48:33.0693199Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0693264Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0693480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0693551Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0693779Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0693850Z return func(*args, **kwargs) 2025-12-04T09:48:33.0694083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0694171Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0694407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0694469Z return func(*args, **kwargs) 2025-12-04T09:48:33.0694706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0694780Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0694783Z 2025-12-04T09:48:33.0694877Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0695062Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0695121Z res = mod(**inputs) 2025-12-04T09:48:33.0695368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0695438Z outputs = self.model( 2025-12-04T09:48:33.0695683Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0695757Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0695967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0696037Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0696287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0696349Z return func(*args, **kwargs) 2025-12-04T09:48:33.0696597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0696699Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0696929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0696996Z return func(*args, **kwargs) 2025-12-04T09:48:33.0697233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0697356Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0697365Z 2025-12-04T09:48:33.0697436Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0697533Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0697723Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0697782Z res = mod(**inputs) 2025-12-04T09:48:33.0698016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0698085Z outputs = self.model( 2025-12-04T09:48:33.0698321Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0698386Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0698603Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0698674Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0698910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0698974Z return func(*args, **kwargs) 2025-12-04T09:48:33.0699206Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0699302Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0699532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0699602Z return func(*args, **kwargs) 2025-12-04T09:48:33.0699838Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0699917Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0699920Z 2025-12-04T09:48:33.0700021Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0700207Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0700267Z res = mod(**inputs) 2025-12-04T09:48:33.0700507Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0700568Z outputs = self.model( 2025-12-04T09:48:33.0700809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0700877Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0701108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0701200Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0701428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0701499Z return func(*args, **kwargs) 2025-12-04T09:48:33.0701730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0701832Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0702068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0702134Z return func(*args, **kwargs) 2025-12-04T09:48:33.0702381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0702478Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0702481Z 2025-12-04T09:48:33.0702575Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0702767Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0702826Z res = mod(**inputs) 2025-12-04T09:48:33.0703058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0703129Z outputs = self.model( 2025-12-04T09:48:33.0703359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0703424Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0703634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0703708Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0703940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0704002Z return func(*args, **kwargs) 2025-12-04T09:48:33.0704233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0704328Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0704552Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0704642Z return func(*args, **kwargs) 2025-12-04T09:48:33.0704871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0704987Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0704992Z 2025-12-04T09:48:33.0705071Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0705143Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0705240Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0705433Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0705491Z res = mod(**inputs) 2025-12-04T09:48:33.0705849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0705918Z outputs = self.model( 2025-12-04T09:48:33.0706155Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0706230Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0706436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0706509Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0706780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0706845Z return func(*args, **kwargs) 2025-12-04T09:48:33.0707103Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0707216Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0707420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0707515Z return self.act(input) 2025-12-04T09:48:33.0707518Z 2025-12-04T09:48:33.0707590Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0707665Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0707738Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0707832Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0708045Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0708105Z res = mod(**inputs) 2025-12-04T09:48:33.0708340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0708408Z outputs = self.model( 2025-12-04T09:48:33.0708642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0708713Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0708924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0708994Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0709229Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0709294Z return func(*args, **kwargs) 2025-12-04T09:48:33.0709526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0709622Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0709851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0709919Z return func(*args, **kwargs) 2025-12-04T09:48:33.0710154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0710258Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0710262Z 2025-12-04T09:48:33.0710361Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0710541Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0710601Z res = mod(**inputs) 2025-12-04T09:48:33.0710843Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0710904Z outputs = self.model( 2025-12-04T09:48:33.0711143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0711208Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0711414Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0711492Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0711721Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0711790Z return func(*args, **kwargs) 2025-12-04T09:48:33.0712022Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0712114Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0712371Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0712435Z return func(*args, **kwargs) 2025-12-04T09:48:33.0712681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0712760Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0712763Z 2025-12-04T09:48:33.0712901Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0713116Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0713176Z res = mod(**inputs) 2025-12-04T09:48:33.0713409Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0713479Z outputs = self.model( 2025-12-04T09:48:33.0713728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0713801Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0714014Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0714085Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0714319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0714381Z return func(*args, **kwargs) 2025-12-04T09:48:33.0714615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0714709Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0714937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0715007Z return func(*args, **kwargs) 2025-12-04T09:48:33.0715239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0715363Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0715367Z 2025-12-04T09:48:33.0715446Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0715542Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0715727Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0715798Z res = mod(**inputs) 2025-12-04T09:48:33.0716030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0716099Z outputs = self.model( 2025-12-04T09:48:33.0716330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0716396Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0716615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0716687Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0716925Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0716988Z return func(*args, **kwargs) 2025-12-04T09:48:33.0717220Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0717315Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0717545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0717607Z return func(*args, **kwargs) 2025-12-04T09:48:33.0717854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0717947Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0717950Z 2025-12-04T09:48:33.0718050Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0718244Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0718303Z res = mod(**inputs) 2025-12-04T09:48:33.0718536Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0718899Z outputs = self.model( 2025-12-04T09:48:33.0719130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0719203Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0719410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0719504Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0719727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0719792Z return func(*args, **kwargs) 2025-12-04T09:48:33.0720031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0720119Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0720349Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0720413Z return func(*args, **kwargs) 2025-12-04T09:48:33.0720644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0720740Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0720745Z 2025-12-04T09:48:33.0720837Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0721020Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0721086Z res = mod(**inputs) 2025-12-04T09:48:33.0721319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0721387Z outputs = self.model( 2025-12-04T09:48:33.0721617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0721685Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0721900Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0721970Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0722204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0722268Z return func(*args, **kwargs) 2025-12-04T09:48:33.0722505Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0722600Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0722826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0722887Z return func(*args, **kwargs) 2025-12-04T09:48:33.0723124Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0723238Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0723242Z 2025-12-04T09:48:33.0723320Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0723392Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0723487Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0723691Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0723750Z res = mod(**inputs) 2025-12-04T09:48:33.0723995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0724064Z outputs = self.model( 2025-12-04T09:48:33.0724292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0724363Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0724587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0724658Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0724890Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0724966Z return func(*args, **kwargs) 2025-12-04T09:48:33.0725197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0725315Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0725520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0725590Z return self.act(input) 2025-12-04T09:48:33.0725593Z 2025-12-04T09:48:33.0725665Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0725735Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0725813Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0725908Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0726101Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0726159Z res = mod(**inputs) 2025-12-04T09:48:33.0726396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0726465Z outputs = self.model( 2025-12-04T09:48:33.0726700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0726768Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0726986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0727059Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0727295Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0727359Z return func(*args, **kwargs) 2025-12-04T09:48:33.0727590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0727689Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0727919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0727983Z return func(*args, **kwargs) 2025-12-04T09:48:33.0728225Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:48:33.0728329Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:48:33.0728332Z 2025-12-04T09:48:33.0728433Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0728617Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0728677Z res = mod(**inputs) 2025-12-04T09:48:33.0728918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0728981Z outputs = self.model( 2025-12-04T09:48:33.0729237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0729305Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0729529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0729610Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0729839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0729901Z return func(*args, **kwargs) 2025-12-04T09:48:33.0730157Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0730246Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0730480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0730565Z return func(*args, **kwargs) 2025-12-04T09:48:33.0730800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:48:33.0730879Z key_states = self.k_proj(current_states) 2025-12-04T09:48:33.0730883Z 2025-12-04T09:48:33.0730977Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0731167Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0731225Z res = mod(**inputs) 2025-12-04T09:48:33.0731457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0731526Z outputs = self.model( 2025-12-04T09:48:33.0731757Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0731823Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0732041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0732111Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0732346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0732409Z return func(*args, **kwargs) 2025-12-04T09:48:33.0732642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0732738Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0732968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0733029Z return func(*args, **kwargs) 2025-12-04T09:48:33.0733271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:48:33.0733396Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:48:33.0733400Z 2025-12-04T09:48:33.0733478Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0733572Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0733758Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0733824Z res = mod(**inputs) 2025-12-04T09:48:33.0734059Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0734122Z outputs = self.model( 2025-12-04T09:48:33.0734360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0734425Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0734640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0734712Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0734956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0735028Z return func(*args, **kwargs) 2025-12-04T09:48:33.0735277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0735374Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0735601Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0735678Z return func(*args, **kwargs) 2025-12-04T09:48:33.0735912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:48:33.0735990Z value_states = self.v_proj(current_states) 2025-12-04T09:48:33.0736008Z 2025-12-04T09:48:33.0736103Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0736293Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0736351Z res = mod(**inputs) 2025-12-04T09:48:33.0736593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0736654Z outputs = self.model( 2025-12-04T09:48:33.0736889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0736965Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0737176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0737252Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0737485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0737550Z return func(*args, **kwargs) 2025-12-04T09:48:33.0737789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0737879Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0738113Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0738182Z return func(*args, **kwargs) 2025-12-04T09:48:33.0738418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:48:33.0738516Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:48:33.0738519Z 2025-12-04T09:48:33.0738612Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0738795Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0738862Z res = mod(**inputs) 2025-12-04T09:48:33.0739096Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0739158Z outputs = self.model( 2025-12-04T09:48:33.0739396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0739462Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0739675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0739748Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0739976Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0740046Z return func(*args, **kwargs) 2025-12-04T09:48:33.0740278Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:48:33.0740390Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:33.0740620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0740698Z return func(*args, **kwargs) 2025-12-04T09:48:33.0740938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:48:33.0741053Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:48:33.0741056Z 2025-12-04T09:48:33.0741145Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0741224Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0741318Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0741507Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0744174Z res = mod(**inputs) 2025-12-04T09:48:33.0744424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:48:33.0744498Z outputs = self.model( 2025-12-04T09:48:33.0744738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:48:33.0744804Z layer_outputs = decoder_layer( 2025-12-04T09:48:33.0745023Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:33.0745096Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:33.0745327Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:33.0745400Z return func(*args, **kwargs) 2025-12-04T09:48:33.0745634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:48:33.0745786Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:33.0745998Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:33.0746063Z return self.act(input) 2025-12-04T09:48:33.0746067Z 2025-12-04T09:48:33.0746140Z cudagraph partition due to non gpu ops 2025-12-04T09:48:33.0746244Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0746429Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0746487Z res = mod(**inputs) 2025-12-04T09:48:33.0746731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 685, in forward 2025-12-04T09:48:33.0746801Z logits = self.lm_head(outputs[0]) 2025-12-04T09:48:33.0746804Z 2025-12-04T09:48:33.0746903Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:33.0747086Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:33.0747145Z res = mod(**inputs) 2025-12-04T09:48:33.0747385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 689, in forward 2025-12-04T09:48:33.0747453Z loss = self.loss_function( 2025-12-04T09:48:33.0747691Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 67, in ForCausalLMLoss 2025-12-04T09:48:33.0747853Z loss = fixed_cross_entropy(logits, shift_labels, num_items_in_batch, ignore_index, **kwargs) 2025-12-04T09:48:33.0748094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 36, in fixed_cross_entropy 2025-12-04T09:48:33.0748280Z loss = nn.functional.cross_entropy(source, target, ignore_index=ignore_index, reduction=reduction) 2025-12-04T09:48:33.0748284Z 2025-12-04T09:48:45.0460121Z Compilation time (from dynamo_timed): 27.737850201 2025-12-04T09:48:45.0541121Z pass 2025-12-04T09:48:45.0541832Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:48:45.0543487Z TIMING: _recursive_pre_grad_passes:0.0618 _recursive_joint_graph_passes:1.04621 _recursive_post_grad_passes:0.21852 async_compile.wait:0.7766 code_gen:11.74808 inductor_compile:15.19693 backend_compile:23.28307 gc:0.00013 entire_frame_compile:27.73785 total_wall_time:27.73785 2025-12-04T09:48:45.0544411Z STATS: call_* op count: 921 | FakeTensorMode.__torch_dispatch__:44773 | FakeTensor.__torch_dispatch__:8380 | ProxyTorchDispatchMode.__torch_dispatch__:6458 2025-12-04T09:48:45.0544898Z Dynamo produced 1 graphs covering 921 ops with 0 graph breaks (0 unique) 2025-12-04T09:48:47.9034553Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:48:47.9035853Z import pynvml # type: ignore[import] 2025-12-04T09:48:51.0030124Z 2025-12-04T09:48:53.6679707Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:48:53.6681947Z loading model: 0it [00:02, ?it/s] 2025-12-04T09:48:53.6702003Z cpu eval XLNetLMHeadModel 2025-12-04T09:48:56.0652035Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:48:56.9009055Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:48:57.7424758Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:49:21.0886100Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0887369Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0887744Z res = mod(**inputs) 2025-12-04T09:49:21.0888201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0888615Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0889030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1307, in forward 2025-12-04T09:49:21.0889439Z word_emb_k = self.word_embedding(input_ids) 2025-12-04T09:49:21.0889586Z 2025-12-04T09:49:21.0889701Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0890054Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0890374Z res = mod(**inputs) 2025-12-04T09:49:21.0890782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0891163Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0891556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1334, in forward 2025-12-04T09:49:21.0891982Z pos_emb = self.relative_positional_encoding(qlen, klen, bsz=bsz) 2025-12-04T09:49:21.0892454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1157, in relative_positional_encoding 2025-12-04T09:49:21.0892952Z pos_emb = self.positional_embedding(fwd_pos_seq, inv_freq, bsz) 2025-12-04T09:49:21.0893488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1115, in positional_embedding 2025-12-04T09:49:21.0893992Z pos_emb = torch.cat([torch.sin(sinusoid_inp), torch.cos(sinusoid_inp)], dim=-1) 2025-12-04T09:49:21.0894199Z 2025-12-04T09:49:21.0894312Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0894669Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0895016Z res = mod(**inputs) 2025-12-04T09:49:21.0896409Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0896836Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0897293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1334, in forward 2025-12-04T09:49:21.0897733Z pos_emb = self.relative_positional_encoding(qlen, klen, bsz=bsz) 2025-12-04T09:49:21.0898223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1157, in relative_positional_encoding 2025-12-04T09:49:21.0898705Z pos_emb = self.positional_embedding(fwd_pos_seq, inv_freq, bsz) 2025-12-04T09:49:21.0899159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1115, in positional_embedding 2025-12-04T09:49:21.0899690Z pos_emb = torch.cat([torch.sin(sinusoid_inp), torch.cos(sinusoid_inp)], dim=-1) 2025-12-04T09:49:21.0899958Z 2025-12-04T09:49:21.0900066Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0900410Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0900735Z res = mod(**inputs) 2025-12-04T09:49:21.0901100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0901517Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0901908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.0902297Z outputs = layer_module( 2025-12-04T09:49:21.0902682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.0903067Z outputs = self.rel_attn( 2025-12-04T09:49:21.0903444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.0903861Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.0904009Z 2025-12-04T09:49:21.0904121Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0904464Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0904780Z res = mod(**inputs) 2025-12-04T09:49:21.0905133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0905523Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0906063Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.0906447Z outputs = layer_module( 2025-12-04T09:49:21.0906814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.0907189Z outputs = self.rel_attn( 2025-12-04T09:49:21.0907556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.0907969Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.0908119Z 2025-12-04T09:49:21.0908227Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0908572Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0908889Z res = mod(**inputs) 2025-12-04T09:49:21.0909241Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0909633Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0910021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.0910430Z outputs = layer_module( 2025-12-04T09:49:21.0910784Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.0911169Z outputs = self.rel_attn( 2025-12-04T09:49:21.0911537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.0911918Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.0912311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.0912763Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.0912955Z 2025-12-04T09:49:21.0913154Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0913528Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0913919Z res = mod(**inputs) 2025-12-04T09:49:21.0914299Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0914711Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0915104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1334, in forward 2025-12-04T09:49:21.0915532Z pos_emb = self.relative_positional_encoding(qlen, klen, bsz=bsz) 2025-12-04T09:49:21.0916008Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1157, in relative_positional_encoding 2025-12-04T09:49:21.0916492Z pos_emb = self.positional_embedding(fwd_pos_seq, inv_freq, bsz) 2025-12-04T09:49:21.0916953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1115, in positional_embedding 2025-12-04T09:49:21.0917439Z pos_emb = torch.cat([torch.sin(sinusoid_inp), torch.cos(sinusoid_inp)], dim=-1) 2025-12-04T09:49:21.0917648Z 2025-12-04T09:49:21.0917747Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0918098Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0918410Z res = mod(**inputs) 2025-12-04T09:49:21.0918758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0919149Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0919535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.0919905Z outputs = layer_module( 2025-12-04T09:49:21.0920275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.0920654Z outputs = self.rel_attn( 2025-12-04T09:49:21.0921019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.0921448Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.0921641Z 2025-12-04T09:49:21.0921743Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0922090Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0922398Z res = mod(**inputs) 2025-12-04T09:49:21.0922753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0923217Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0923604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.0923972Z outputs = layer_module( 2025-12-04T09:49:21.0924359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.0924723Z outputs = self.rel_attn( 2025-12-04T09:49:21.0925105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.0925486Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.0925870Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.0926309Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.0926485Z 2025-12-04T09:49:21.0926583Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0926922Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0927225Z res = mod(**inputs) 2025-12-04T09:49:21.0927608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0927987Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0928376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.0928763Z outputs = layer_module( 2025-12-04T09:49:21.0929126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.0929496Z outputs = self.rel_attn( 2025-12-04T09:49:21.0929855Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.0930260Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.0930410Z 2025-12-04T09:49:21.0930508Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0930858Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0931172Z res = mod(**inputs) 2025-12-04T09:49:21.0931524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0931910Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0932302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.0932684Z outputs = layer_module( 2025-12-04T09:49:21.0933044Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.0933422Z outputs = self.rel_attn( 2025-12-04T09:49:21.0933787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.0934187Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.0934570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.0935046Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.0935219Z 2025-12-04T09:49:21.0935320Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0935676Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0935991Z res = mod(**inputs) 2025-12-04T09:49:21.0936345Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0936727Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0937108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.0937479Z outputs = layer_module( 2025-12-04T09:49:21.0937845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.0938212Z outputs = self.rel_attn( 2025-12-04T09:49:21.0938581Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.0938967Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.0939372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.0939819Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.0939984Z 2025-12-04T09:49:21.0940081Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0940420Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0940721Z res = mod(**inputs) 2025-12-04T09:49:21.0941130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0941516Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0941895Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.0942265Z outputs = layer_module( 2025-12-04T09:49:21.0942621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.0942994Z outputs = self.rel_attn( 2025-12-04T09:49:21.0943348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.0943739Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.0944148Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.0944590Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.0944755Z 2025-12-04T09:49:21.0944836Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.0945069Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0945420Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0945725Z res = mod(**inputs) 2025-12-04T09:49:21.0946079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0946470Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0946848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.0947213Z outputs = layer_module( 2025-12-04T09:49:21.0947572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.0948093Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.0948636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.0949022Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.0949407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.0949790Z output_x = self.ff(output_x) 2025-12-04T09:49:21.0950150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.0950543Z output = self.activation_function(output) 2025-12-04T09:49:21.0950894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.0951236Z return self.act(input) 2025-12-04T09:49:21.0951342Z 2025-12-04T09:49:21.0951442Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.0951674Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0952039Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0952346Z res = mod(**inputs) 2025-12-04T09:49:21.0952698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0953171Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0953565Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.0953934Z outputs = layer_module( 2025-12-04T09:49:21.0954296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.0954709Z outputs = self.rel_attn( 2025-12-04T09:49:21.0955063Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.0955468Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.0955624Z 2025-12-04T09:49:21.0955725Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0956076Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0956383Z res = mod(**inputs) 2025-12-04T09:49:21.0956735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0957129Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0957520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.0957904Z outputs = layer_module( 2025-12-04T09:49:21.0958267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.0958643Z outputs = self.rel_attn( 2025-12-04T09:49:21.0958993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.0959393Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.0959549Z 2025-12-04T09:49:21.0959647Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0959996Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0960300Z res = mod(**inputs) 2025-12-04T09:49:21.0960648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0961041Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0961417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.0961797Z outputs = layer_module( 2025-12-04T09:49:21.0962155Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.0962541Z outputs = self.rel_attn( 2025-12-04T09:49:21.0962899Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.0963274Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.0963659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.0964106Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.0964284Z 2025-12-04T09:49:21.0964380Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0964723Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0965045Z res = mod(**inputs) 2025-12-04T09:49:21.0965385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0965780Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0966149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.0966511Z outputs = layer_module( 2025-12-04T09:49:21.0966855Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.0967231Z outputs = self.rel_attn( 2025-12-04T09:49:21.0967588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.0968018Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.0968228Z 2025-12-04T09:49:21.0968328Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0968666Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0968967Z res = mod(**inputs) 2025-12-04T09:49:21.0969304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0969687Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0970064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.0970430Z outputs = layer_module( 2025-12-04T09:49:21.0970770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.0971142Z outputs = self.rel_attn( 2025-12-04T09:49:21.0971500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.0971868Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.0972253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.0972690Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.0972863Z 2025-12-04T09:49:21.0972968Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0973299Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0973596Z res = mod(**inputs) 2025-12-04T09:49:21.0973942Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0974320Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0974690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.0975059Z outputs = layer_module( 2025-12-04T09:49:21.0975407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.0975761Z outputs = self.rel_attn( 2025-12-04T09:49:21.0976113Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.0976506Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.0976651Z 2025-12-04T09:49:21.0976756Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0977085Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0977384Z res = mod(**inputs) 2025-12-04T09:49:21.0977729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0978112Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0978497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.0978878Z outputs = layer_module( 2025-12-04T09:49:21.0979234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.0979591Z outputs = self.rel_attn( 2025-12-04T09:49:21.0979944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.0980314Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.0980691Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.0981118Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.0981324Z 2025-12-04T09:49:21.0981422Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0981769Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0982064Z res = mod(**inputs) 2025-12-04T09:49:21.0982412Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0982796Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0983174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.0983534Z outputs = layer_module( 2025-12-04T09:49:21.0983890Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.0984259Z outputs = self.rel_attn( 2025-12-04T09:49:21.0984608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.0984995Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.0985404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.0985832Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.0985991Z 2025-12-04T09:49:21.0986087Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0986431Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0986734Z res = mod(**inputs) 2025-12-04T09:49:21.0987078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0987448Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0987820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.0988188Z outputs = layer_module( 2025-12-04T09:49:21.0988527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.0988895Z outputs = self.rel_attn( 2025-12-04T09:49:21.0989243Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.0989625Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.0990019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.0990445Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.0990606Z 2025-12-04T09:49:21.0990683Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.0990906Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0991254Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0991559Z res = mod(**inputs) 2025-12-04T09:49:21.0991928Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0992302Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0992720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.0993203Z outputs = layer_module( 2025-12-04T09:49:21.0993590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.0994131Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.0994660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.0995110Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.0995519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.0995920Z output_x = self.ff(output_x) 2025-12-04T09:49:21.0996315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.0996737Z output = self.activation_function(output) 2025-12-04T09:49:21.0997110Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.0997485Z return self.act(input) 2025-12-04T09:49:21.0997607Z 2025-12-04T09:49:21.0997691Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.0997935Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.0998297Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.0998631Z res = mod(**inputs) 2025-12-04T09:49:21.0999003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.0999415Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.0999826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1000222Z outputs = layer_module( 2025-12-04T09:49:21.1000600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1000994Z outputs = self.rel_attn( 2025-12-04T09:49:21.1001376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.1001801Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.1001961Z 2025-12-04T09:49:21.1002071Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1002441Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1002748Z res = mod(**inputs) 2025-12-04T09:49:21.1003093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1003467Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1003845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1004209Z outputs = layer_module( 2025-12-04T09:49:21.1004556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1004914Z outputs = self.rel_attn( 2025-12-04T09:49:21.1005262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.1005881Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.1006040Z 2025-12-04T09:49:21.1006141Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1006519Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1006827Z res = mod(**inputs) 2025-12-04T09:49:21.1007171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1007542Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1007923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1008295Z outputs = layer_module( 2025-12-04T09:49:21.1008645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1009056Z outputs = self.rel_attn( 2025-12-04T09:49:21.1009407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1009780Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1010158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.1010597Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.1010779Z 2025-12-04T09:49:21.1010878Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1011220Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1011516Z res = mod(**inputs) 2025-12-04T09:49:21.1011862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1012247Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1012616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1012984Z outputs = layer_module( 2025-12-04T09:49:21.1013340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1013709Z outputs = self.rel_attn( 2025-12-04T09:49:21.1014054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.1014480Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.1014661Z 2025-12-04T09:49:21.1014761Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1015098Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1015396Z res = mod(**inputs) 2025-12-04T09:49:21.1015739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1016123Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1016495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1016860Z outputs = layer_module( 2025-12-04T09:49:21.1017210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1017576Z outputs = self.rel_attn( 2025-12-04T09:49:21.1017915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1018284Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1018667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.1019126Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.1019298Z 2025-12-04T09:49:21.1019395Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1019756Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1020062Z res = mod(**inputs) 2025-12-04T09:49:21.1020396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1020774Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1021148Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1021511Z outputs = layer_module( 2025-12-04T09:49:21.1021853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1022248Z outputs = self.rel_attn( 2025-12-04T09:49:21.1022597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.1022985Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.1023139Z 2025-12-04T09:49:21.1023236Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1023570Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1023874Z res = mod(**inputs) 2025-12-04T09:49:21.1024210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1024591Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1024967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1025334Z outputs = layer_module( 2025-12-04T09:49:21.1025674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1026040Z outputs = self.rel_attn( 2025-12-04T09:49:21.1026391Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1026756Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1027136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.1027572Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.1027741Z 2025-12-04T09:49:21.1027844Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1028176Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1028485Z res = mod(**inputs) 2025-12-04T09:49:21.1028833Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1029207Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1029587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1029959Z outputs = layer_module( 2025-12-04T09:49:21.1030310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1030673Z outputs = self.rel_attn( 2025-12-04T09:49:21.1031026Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1031414Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1031815Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1032256Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1032421Z 2025-12-04T09:49:21.1032517Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1032867Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1033234Z res = mod(**inputs) 2025-12-04T09:49:21.1033595Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1034062Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1034456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1034830Z outputs = layer_module( 2025-12-04T09:49:21.1035185Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1035588Z outputs = self.rel_attn( 2025-12-04T09:49:21.1035936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1036319Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1036723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1037147Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1037308Z 2025-12-04T09:49:21.1037386Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1037617Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1037949Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1038252Z res = mod(**inputs) 2025-12-04T09:49:21.1038588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1038971Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1039345Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1039700Z outputs = layer_module( 2025-12-04T09:49:21.1040047Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.1040545Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.1041049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.1041422Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.1041791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.1042160Z output_x = self.ff(output_x) 2025-12-04T09:49:21.1042522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.1042896Z output = self.activation_function(output) 2025-12-04T09:49:21.1043243Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.1043577Z return self.act(input) 2025-12-04T09:49:21.1043679Z 2025-12-04T09:49:21.1043754Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1043976Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1044315Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1044617Z res = mod(**inputs) 2025-12-04T09:49:21.1044955Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1045332Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1045734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1046095Z outputs = layer_module( 2025-12-04T09:49:21.1046462Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1046835Z outputs = self.rel_attn( 2025-12-04T09:49:21.1047189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.1047574Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.1047725Z 2025-12-04T09:49:21.1047823Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1048165Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1048472Z res = mod(**inputs) 2025-12-04T09:49:21.1048840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1049221Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1049596Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1049956Z outputs = layer_module( 2025-12-04T09:49:21.1050304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1050667Z outputs = self.rel_attn( 2025-12-04T09:49:21.1051017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.1051403Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.1051555Z 2025-12-04T09:49:21.1051652Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1051997Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1052293Z res = mod(**inputs) 2025-12-04T09:49:21.1052636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1053015Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1053393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1053752Z outputs = layer_module( 2025-12-04T09:49:21.1054101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1054470Z outputs = self.rel_attn( 2025-12-04T09:49:21.1054811Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1055184Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1055566Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.1056007Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.1056183Z 2025-12-04T09:49:21.1056725Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1057068Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1057371Z res = mod(**inputs) 2025-12-04T09:49:21.1057720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1058095Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1058469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1058843Z outputs = layer_module( 2025-12-04T09:49:21.1059211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1059574Z outputs = self.rel_attn( 2025-12-04T09:49:21.1059939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.1060367Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.1060547Z 2025-12-04T09:49:21.1060643Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1060985Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1061290Z res = mod(**inputs) 2025-12-04T09:49:21.1061634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1062012Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1062423Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1062791Z outputs = layer_module( 2025-12-04T09:49:21.1063133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1063508Z outputs = self.rel_attn( 2025-12-04T09:49:21.1063862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1064232Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1064606Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.1065048Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.1065230Z 2025-12-04T09:49:21.1065327Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1065671Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1065969Z res = mod(**inputs) 2025-12-04T09:49:21.1066313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1066693Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1067069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1067435Z outputs = layer_module( 2025-12-04T09:49:21.1067785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1068150Z outputs = self.rel_attn( 2025-12-04T09:49:21.1068493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.1068892Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.1069037Z 2025-12-04T09:49:21.1069141Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1069478Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1069772Z res = mod(**inputs) 2025-12-04T09:49:21.1070115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1070492Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1070862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1071224Z outputs = layer_module( 2025-12-04T09:49:21.1071570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1071935Z outputs = self.rel_attn( 2025-12-04T09:49:21.1072292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1072664Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1073141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.1073591Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.1073772Z 2025-12-04T09:49:21.1073872Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1074231Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1074539Z res = mod(**inputs) 2025-12-04T09:49:21.1074879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1075260Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1075655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1076040Z outputs = layer_module( 2025-12-04T09:49:21.1076389Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1076757Z outputs = self.rel_attn( 2025-12-04T09:49:21.1077107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1077485Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1077888Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1078311Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1078469Z 2025-12-04T09:49:21.1078574Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1078906Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1079215Z res = mod(**inputs) 2025-12-04T09:49:21.1079559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1079931Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1080307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1080675Z outputs = layer_module( 2025-12-04T09:49:21.1081025Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1081382Z outputs = self.rel_attn( 2025-12-04T09:49:21.1081735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1082117Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1082522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1082940Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1083104Z 2025-12-04T09:49:21.1083184Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1083408Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1083736Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1084045Z res = mod(**inputs) 2025-12-04T09:49:21.1084389Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1084772Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1085139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1085510Z outputs = layer_module( 2025-12-04T09:49:21.1085873Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.1086390Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.1086891Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.1087275Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.1087645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.1088005Z output_x = self.ff(output_x) 2025-12-04T09:49:21.1088366Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.1088747Z output = self.activation_function(output) 2025-12-04T09:49:21.1089135Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.1089459Z return self.act(input) 2025-12-04T09:49:21.1089572Z 2025-12-04T09:49:21.1089647Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1089873Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1090204Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1090511Z res = mod(**inputs) 2025-12-04T09:49:21.1090852Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1091227Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1091595Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1091960Z outputs = layer_module( 2025-12-04T09:49:21.1092319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1092675Z outputs = self.rel_attn( 2025-12-04T09:49:21.1093028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.1093418Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.1093562Z 2025-12-04T09:49:21.1093666Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1093996Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1094300Z res = mod(**inputs) 2025-12-04T09:49:21.1094646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1095024Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1095391Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1095758Z outputs = layer_module( 2025-12-04T09:49:21.1096108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1096469Z outputs = self.rel_attn( 2025-12-04T09:49:21.1096822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.1097219Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.1097364Z 2025-12-04T09:49:21.1097469Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1097800Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1098102Z res = mod(**inputs) 2025-12-04T09:49:21.1098442Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1098839Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1099210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1099629Z outputs = layer_module( 2025-12-04T09:49:21.1099977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1100341Z outputs = self.rel_attn( 2025-12-04T09:49:21.1100693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1101062Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1101444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.1101881Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.1102131Z 2025-12-04T09:49:21.1102230Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1102571Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1102868Z res = mod(**inputs) 2025-12-04T09:49:21.1103217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1103595Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1103973Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1104336Z outputs = layer_module( 2025-12-04T09:49:21.1104690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1105054Z outputs = self.rel_attn( 2025-12-04T09:49:21.1105406Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.1105992Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.1106179Z 2025-12-04T09:49:21.1106280Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1106628Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1106927Z res = mod(**inputs) 2025-12-04T09:49:21.1107276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1107665Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1108044Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1108401Z outputs = layer_module( 2025-12-04T09:49:21.1108753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1109124Z outputs = self.rel_attn( 2025-12-04T09:49:21.1109472Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1109849Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1110236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.1110678Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.1110852Z 2025-12-04T09:49:21.1110951Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1111293Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1111596Z res = mod(**inputs) 2025-12-04T09:49:21.1111944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1112359Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1112738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1113198Z outputs = layer_module( 2025-12-04T09:49:21.1113555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1113930Z outputs = self.rel_attn( 2025-12-04T09:49:21.1114288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.1114696Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.1114842Z 2025-12-04T09:49:21.1114938Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1115282Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1115685Z res = mod(**inputs) 2025-12-04T09:49:21.1116031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1116426Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1116818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1117196Z outputs = layer_module( 2025-12-04T09:49:21.1117549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1117938Z outputs = self.rel_attn( 2025-12-04T09:49:21.1118304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1118686Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1119068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.1119519Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.1119691Z 2025-12-04T09:49:21.1119801Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1120148Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1120463Z res = mod(**inputs) 2025-12-04T09:49:21.1120832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1121225Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1121606Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1121981Z outputs = layer_module( 2025-12-04T09:49:21.1122341Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1122721Z outputs = self.rel_attn( 2025-12-04T09:49:21.1123080Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1123477Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1123898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1124329Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1124498Z 2025-12-04T09:49:21.1124598Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1124946Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1125261Z res = mod(**inputs) 2025-12-04T09:49:21.1125605Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1126003Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1126410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1126780Z outputs = layer_module( 2025-12-04T09:49:21.1127150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1127533Z outputs = self.rel_attn( 2025-12-04T09:49:21.1127893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1128283Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1128685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1129108Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1129295Z 2025-12-04T09:49:21.1129379Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1129602Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1129939Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1130256Z res = mod(**inputs) 2025-12-04T09:49:21.1130609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1130988Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1131358Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1131721Z outputs = layer_module( 2025-12-04T09:49:21.1132063Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.1132560Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.1133065Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.1133447Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.1133814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.1134184Z output_x = self.ff(output_x) 2025-12-04T09:49:21.1134544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.1134919Z output = self.activation_function(output) 2025-12-04T09:49:21.1135265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.1135591Z return self.act(input) 2025-12-04T09:49:21.1135697Z 2025-12-04T09:49:21.1135782Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1136004Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1136341Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1136645Z res = mod(**inputs) 2025-12-04T09:49:21.1136983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1137361Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1137743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1138109Z outputs = layer_module( 2025-12-04T09:49:21.1138448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1138813Z outputs = self.rel_attn( 2025-12-04T09:49:21.1139163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.1139561Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.1139714Z 2025-12-04T09:49:21.1139809Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1140164Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1140470Z res = mod(**inputs) 2025-12-04T09:49:21.1140802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1141181Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1141558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1141917Z outputs = layer_module( 2025-12-04T09:49:21.1142260Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1142669Z outputs = self.rel_attn( 2025-12-04T09:49:21.1143023Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.1143413Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.1143567Z 2025-12-04T09:49:21.1143666Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1144008Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1144311Z res = mod(**inputs) 2025-12-04T09:49:21.1144647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1145023Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1145399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1145765Z outputs = layer_module( 2025-12-04T09:49:21.1146106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1146472Z outputs = self.rel_attn( 2025-12-04T09:49:21.1146826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1147190Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1147572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.1148015Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.1148190Z 2025-12-04T09:49:21.1148296Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1148629Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1148938Z res = mod(**inputs) 2025-12-04T09:49:21.1149285Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1149658Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1150043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1150409Z outputs = layer_module( 2025-12-04T09:49:21.1150760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1151118Z outputs = self.rel_attn( 2025-12-04T09:49:21.1151468Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.1151891Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.1152068Z 2025-12-04T09:49:21.1152176Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1152524Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1152832Z res = mod(**inputs) 2025-12-04T09:49:21.1153272Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1153664Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1154052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1154424Z outputs = layer_module( 2025-12-04T09:49:21.1154792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1155152Z outputs = self.rel_attn( 2025-12-04T09:49:21.1155504Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1155907Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1156290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.1156731Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.1156912Z 2025-12-04T09:49:21.1157010Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1157353Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1157659Z res = mod(**inputs) 2025-12-04T09:49:21.1158008Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1158388Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1158767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1159131Z outputs = layer_module( 2025-12-04T09:49:21.1159487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1159856Z outputs = self.rel_attn( 2025-12-04T09:49:21.1160204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.1160604Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.1160761Z 2025-12-04T09:49:21.1160861Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1161203Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1161502Z res = mod(**inputs) 2025-12-04T09:49:21.1161854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1162236Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1162613Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1162986Z outputs = layer_module( 2025-12-04T09:49:21.1163344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1163717Z outputs = self.rel_attn( 2025-12-04T09:49:21.1164064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1164431Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1164813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.1165252Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.1165421Z 2025-12-04T09:49:21.1165519Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1165886Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1166201Z res = mod(**inputs) 2025-12-04T09:49:21.1166564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1166952Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1167334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1167706Z outputs = layer_module( 2025-12-04T09:49:21.1168054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1168434Z outputs = self.rel_attn( 2025-12-04T09:49:21.1168783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1169194Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1169588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1170017Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1170175Z 2025-12-04T09:49:21.1170280Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1170610Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1170914Z res = mod(**inputs) 2025-12-04T09:49:21.1171253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1171632Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1172001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1172367Z outputs = layer_module( 2025-12-04T09:49:21.1172716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1173070Z outputs = self.rel_attn( 2025-12-04T09:49:21.1173420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1173801Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1174200Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1174611Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1174772Z 2025-12-04T09:49:21.1174847Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1175072Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1175405Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1175704Z res = mod(**inputs) 2025-12-04T09:49:21.1176054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1176433Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1176801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1177163Z outputs = layer_module( 2025-12-04T09:49:21.1177517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.1178018Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.1178517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.1178897Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.1179290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.1179662Z output_x = self.ff(output_x) 2025-12-04T09:49:21.1180036Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.1180422Z output = self.activation_function(output) 2025-12-04T09:49:21.1180769Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.1181091Z return self.act(input) 2025-12-04T09:49:21.1181204Z 2025-12-04T09:49:21.1181281Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1181507Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1181848Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1182166Z res = mod(**inputs) 2025-12-04T09:49:21.1182529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1182908Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1183276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1183638Z outputs = layer_module( 2025-12-04T09:49:21.1183987Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1184351Z outputs = self.rel_attn( 2025-12-04T09:49:21.1184697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.1185083Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.1185228Z 2025-12-04T09:49:21.1185336Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1185683Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1185982Z res = mod(**inputs) 2025-12-04T09:49:21.1186324Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1186701Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1187084Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1187446Z outputs = layer_module( 2025-12-04T09:49:21.1187794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1188158Z outputs = self.rel_attn( 2025-12-04T09:49:21.1188504Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.1188899Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.1189045Z 2025-12-04T09:49:21.1189151Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1189485Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1189790Z res = mod(**inputs) 2025-12-04T09:49:21.1190129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1190508Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1190872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1191235Z outputs = layer_module( 2025-12-04T09:49:21.1191584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1191940Z outputs = self.rel_attn( 2025-12-04T09:49:21.1192314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1192686Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1193176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.1193640Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.1193834Z 2025-12-04T09:49:21.1193943Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1194285Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1194588Z res = mod(**inputs) 2025-12-04T09:49:21.1194927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1195309Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1195717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1196071Z outputs = layer_module( 2025-12-04T09:49:21.1196435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1196812Z outputs = self.rel_attn( 2025-12-04T09:49:21.1197181Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.1197606Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.1197795Z 2025-12-04T09:49:21.1197893Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1198237Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1198547Z res = mod(**inputs) 2025-12-04T09:49:21.1198889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1199279Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1199663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1200029Z outputs = layer_module( 2025-12-04T09:49:21.1200382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1200752Z outputs = self.rel_attn( 2025-12-04T09:49:21.1201107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1201476Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1201866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.1202313Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.1202490Z 2025-12-04T09:49:21.1202600Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1202940Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1203250Z res = mod(**inputs) 2025-12-04T09:49:21.1203599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1203976Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1204360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1204726Z outputs = layer_module( 2025-12-04T09:49:21.1205078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1205442Z outputs = self.rel_attn( 2025-12-04T09:49:21.1206027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.1206440Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.1206590Z 2025-12-04T09:49:21.1206718Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1207067Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1207385Z res = mod(**inputs) 2025-12-04T09:49:21.1207730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1208100Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1208474Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1208841Z outputs = layer_module( 2025-12-04T09:49:21.1209190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1209603Z outputs = self.rel_attn( 2025-12-04T09:49:21.1209957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1210336Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1210712Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.1211146Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.1211321Z 2025-12-04T09:49:21.1211419Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1211755Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1212049Z res = mod(**inputs) 2025-12-04T09:49:21.1212426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1212811Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1213178Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1213544Z outputs = layer_module( 2025-12-04T09:49:21.1213893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1214256Z outputs = self.rel_attn( 2025-12-04T09:49:21.1214600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1214980Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1215384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1215812Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1215970Z 2025-12-04T09:49:21.1216068Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1216415Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1216720Z res = mod(**inputs) 2025-12-04T09:49:21.1217054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1217431Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1217808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1218172Z outputs = layer_module( 2025-12-04T09:49:21.1218509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1218877Z outputs = self.rel_attn( 2025-12-04T09:49:21.1219249Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1219631Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1220048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1220470Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1220626Z 2025-12-04T09:49:21.1220710Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1220931Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1221264Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1221567Z res = mod(**inputs) 2025-12-04T09:49:21.1221909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1222299Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1222692Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1223055Z outputs = layer_module( 2025-12-04T09:49:21.1223396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.1223897Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.1224405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.1224787Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.1225150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.1225521Z output_x = self.ff(output_x) 2025-12-04T09:49:21.1225890Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.1226277Z output = self.activation_function(output) 2025-12-04T09:49:21.1226616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.1226950Z return self.act(input) 2025-12-04T09:49:21.1227052Z 2025-12-04T09:49:21.1227137Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1227353Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1227693Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1227997Z res = mod(**inputs) 2025-12-04T09:49:21.1228339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1228708Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1229090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1229461Z outputs = layer_module( 2025-12-04T09:49:21.1229812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1230180Z outputs = self.rel_attn( 2025-12-04T09:49:21.1230533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.1230924Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.1231069Z 2025-12-04T09:49:21.1231166Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1231506Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1231810Z res = mod(**inputs) 2025-12-04T09:49:21.1232150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1232553Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1232939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1233389Z outputs = layer_module( 2025-12-04T09:49:21.1233747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1234119Z outputs = self.rel_attn( 2025-12-04T09:49:21.1234485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.1234874Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.1235018Z 2025-12-04T09:49:21.1235114Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1235451Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1235789Z res = mod(**inputs) 2025-12-04T09:49:21.1236124Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1236506Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1236894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1237262Z outputs = layer_module( 2025-12-04T09:49:21.1237605Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1237973Z outputs = self.rel_attn( 2025-12-04T09:49:21.1238324Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1238685Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1239068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.1239513Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.1239687Z 2025-12-04T09:49:21.1239790Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1240123Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1240425Z res = mod(**inputs) 2025-12-04T09:49:21.1240768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1241150Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1241520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1241888Z outputs = layer_module( 2025-12-04T09:49:21.1242236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1242606Z outputs = self.rel_attn( 2025-12-04T09:49:21.1242957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.1243385Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.1243563Z 2025-12-04T09:49:21.1243668Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1243998Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1244301Z res = mod(**inputs) 2025-12-04T09:49:21.1244650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1245033Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1245399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1245769Z outputs = layer_module( 2025-12-04T09:49:21.1246141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1246507Z outputs = self.rel_attn( 2025-12-04T09:49:21.1246876Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1247250Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1247634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.1248067Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.1248248Z 2025-12-04T09:49:21.1248345Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1248690Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1249015Z res = mod(**inputs) 2025-12-04T09:49:21.1249358Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1249741Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1250125Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1250485Z outputs = layer_module( 2025-12-04T09:49:21.1250837Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1251202Z outputs = self.rel_attn( 2025-12-04T09:49:21.1251554Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.1251941Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.1252096Z 2025-12-04T09:49:21.1252194Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1252532Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1252832Z res = mod(**inputs) 2025-12-04T09:49:21.1253178Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1253561Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1253938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1254297Z outputs = layer_module( 2025-12-04T09:49:21.1254645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1255012Z outputs = self.rel_attn( 2025-12-04T09:49:21.1255353Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1255725Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1256105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.1256541Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.1256713Z 2025-12-04T09:49:21.1256809Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1257147Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1257450Z res = mod(**inputs) 2025-12-04T09:49:21.1257794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1258165Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1258540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1258608Z outputs = layer_module( 2025-12-04T09:49:21.1258880Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1258945Z outputs = self.rel_attn( 2025-12-04T09:49:21.1259207Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1259295Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1259561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1259675Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1259679Z 2025-12-04T09:49:21.1259776Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1259964Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1260060Z res = mod(**inputs) 2025-12-04T09:49:21.1260305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1260390Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1260644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1260708Z outputs = layer_module( 2025-12-04T09:49:21.1260960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1261025Z outputs = self.rel_attn( 2025-12-04T09:49:21.1261277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1261360Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1261621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1261737Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1261740Z 2025-12-04T09:49:21.1261816Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1261915Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1262110Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1262172Z res = mod(**inputs) 2025-12-04T09:49:21.1262422Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1262500Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1262743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1262814Z outputs = layer_module( 2025-12-04T09:49:21.1263056Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.1263262Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.1263532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.1263607Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.1263860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.1263929Z output_x = self.ff(output_x) 2025-12-04T09:49:21.1264171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.1264260Z output = self.activation_function(output) 2025-12-04T09:49:21.1264465Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.1264539Z return self.act(input) 2025-12-04T09:49:21.1264560Z 2025-12-04T09:49:21.1264637Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1264734Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1264939Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1265000Z res = mod(**inputs) 2025-12-04T09:49:21.1265244Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1265326Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1265564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1265635Z outputs = layer_module( 2025-12-04T09:49:21.1265875Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1265969Z outputs = self.rel_attn( 2025-12-04T09:49:21.1266217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.1266307Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.1266310Z 2025-12-04T09:49:21.1266413Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1266598Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1266657Z res = mod(**inputs) 2025-12-04T09:49:21.1266909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1266986Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1267225Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1267298Z outputs = layer_module( 2025-12-04T09:49:21.1267536Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1267608Z outputs = self.rel_attn( 2025-12-04T09:49:21.1267849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.1267941Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.1267945Z 2025-12-04T09:49:21.1268048Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1268233Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1268301Z res = mod(**inputs) 2025-12-04T09:49:21.1268544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1268622Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1268872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1268934Z outputs = layer_module( 2025-12-04T09:49:21.1269172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1269243Z outputs = self.rel_attn( 2025-12-04T09:49:21.1269481Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1269556Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1269812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.1269935Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.1269939Z 2025-12-04T09:49:21.1270042Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1270246Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1270307Z res = mod(**inputs) 2025-12-04T09:49:21.1270567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1270642Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1270892Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1270954Z outputs = layer_module( 2025-12-04T09:49:21.1271192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1271261Z outputs = self.rel_attn( 2025-12-04T09:49:21.1271498Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.1271670Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.1271675Z 2025-12-04T09:49:21.1271770Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1271956Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1272023Z res = mod(**inputs) 2025-12-04T09:49:21.1272262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1272338Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1272585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1272648Z outputs = layer_module( 2025-12-04T09:49:21.1272902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1272971Z outputs = self.rel_attn( 2025-12-04T09:49:21.1273320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1273403Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1273667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.1273802Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.1273806Z 2025-12-04T09:49:21.1273906Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1274097Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1274167Z res = mod(**inputs) 2025-12-04T09:49:21.1274427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1274506Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1274757Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1274821Z outputs = layer_module( 2025-12-04T09:49:21.1275067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1275131Z outputs = self.rel_attn( 2025-12-04T09:49:21.1275371Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.1275474Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.1275477Z 2025-12-04T09:49:21.1275573Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1275766Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1275827Z res = mod(**inputs) 2025-12-04T09:49:21.1276097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1276181Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1276433Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1276497Z outputs = layer_module( 2025-12-04T09:49:21.1276743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1276806Z outputs = self.rel_attn( 2025-12-04T09:49:21.1277050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1277118Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1277374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.1277527Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.1277531Z 2025-12-04T09:49:21.1277627Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1277818Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1277881Z res = mod(**inputs) 2025-12-04T09:49:21.1278123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1278204Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1278441Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1278502Z outputs = layer_module( 2025-12-04T09:49:21.1278747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1278811Z outputs = self.rel_attn( 2025-12-04T09:49:21.1279059Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1279141Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1279400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1279513Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1279516Z 2025-12-04T09:49:21.1279612Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1279809Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1279868Z res = mod(**inputs) 2025-12-04T09:49:21.1280108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1280188Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1280429Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1280492Z outputs = layer_module( 2025-12-04T09:49:21.1280738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1280803Z outputs = self.rel_attn( 2025-12-04T09:49:21.1281047Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1281127Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1281385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1281495Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1281498Z 2025-12-04T09:49:21.1281572Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1281671Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1281910Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1281972Z res = mod(**inputs) 2025-12-04T09:49:21.1282238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1282316Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1282557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1282628Z outputs = layer_module( 2025-12-04T09:49:21.1282869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.1283075Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.1284172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.1284247Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.1284502Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.1284571Z output_x = self.ff(output_x) 2025-12-04T09:49:21.1284814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.1284903Z output = self.activation_function(output) 2025-12-04T09:49:21.1285111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.1285186Z return self.act(input) 2025-12-04T09:49:21.1285189Z 2025-12-04T09:49:21.1285265Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1285364Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1285565Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1285626Z res = mod(**inputs) 2025-12-04T09:49:21.1285870Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1285957Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1286201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1286275Z outputs = layer_module( 2025-12-04T09:49:21.1286513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1286575Z outputs = self.rel_attn( 2025-12-04T09:49:21.1286822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.1286914Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.1286918Z 2025-12-04T09:49:21.1287019Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1287206Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1287268Z res = mod(**inputs) 2025-12-04T09:49:21.1287516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1287595Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1287840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1287908Z outputs = layer_module( 2025-12-04T09:49:21.1288148Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1288218Z outputs = self.rel_attn( 2025-12-04T09:49:21.1288479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.1288572Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.1288575Z 2025-12-04T09:49:21.1288693Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1288880Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1288948Z res = mod(**inputs) 2025-12-04T09:49:21.1289193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1289268Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1289515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1289578Z outputs = layer_module( 2025-12-04T09:49:21.1289834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1289923Z outputs = self.rel_attn( 2025-12-04T09:49:21.1290166Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1290242Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1290499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.1290622Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.1290625Z 2025-12-04T09:49:21.1290728Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1290910Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1290978Z res = mod(**inputs) 2025-12-04T09:49:21.1291217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1291298Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1291544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1291608Z outputs = layer_module( 2025-12-04T09:49:21.1291845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1291915Z outputs = self.rel_attn( 2025-12-04T09:49:21.1292150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.1292283Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.1292286Z 2025-12-04T09:49:21.1292382Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1292568Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1292639Z res = mod(**inputs) 2025-12-04T09:49:21.1292878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1292962Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1293199Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1293261Z outputs = layer_module( 2025-12-04T09:49:21.1293501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1293563Z outputs = self.rel_attn( 2025-12-04T09:49:21.1293804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1293880Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1294154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.1294286Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.1294289Z 2025-12-04T09:49:21.1294398Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1294584Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1294652Z res = mod(**inputs) 2025-12-04T09:49:21.1294895Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1294972Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1295223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1295286Z outputs = layer_module( 2025-12-04T09:49:21.1295566Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1295646Z outputs = self.rel_attn( 2025-12-04T09:49:21.1295882Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.1295982Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.1295986Z 2025-12-04T09:49:21.1296080Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1296266Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1296325Z res = mod(**inputs) 2025-12-04T09:49:21.1296560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1296643Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1296879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1296946Z outputs = layer_module( 2025-12-04T09:49:21.1297189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1297254Z outputs = self.rel_attn( 2025-12-04T09:49:21.1297505Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1297571Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1297818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.1297941Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.1297944Z 2025-12-04T09:49:21.1298036Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1298227Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1298288Z res = mod(**inputs) 2025-12-04T09:49:21.1298525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1298610Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1298847Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1298910Z outputs = layer_module( 2025-12-04T09:49:21.1299152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1299215Z outputs = self.rel_attn( 2025-12-04T09:49:21.1299455Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1299537Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1299810Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1299925Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1299929Z 2025-12-04T09:49:21.1300036Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1300228Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1300288Z res = mod(**inputs) 2025-12-04T09:49:21.1300528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1300608Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1300848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1300910Z outputs = layer_module( 2025-12-04T09:49:21.1301154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1301250Z outputs = self.rel_attn( 2025-12-04T09:49:21.1301498Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1301580Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1301844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1301954Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1301957Z 2025-12-04T09:49:21.1302033Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1302135Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1302320Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1302380Z res = mod(**inputs) 2025-12-04T09:49:21.1302636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1302712Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1302960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1303032Z outputs = layer_module( 2025-12-04T09:49:21.1303278Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.1303483Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.1303737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.1303810Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.1304064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.1304138Z output_x = self.ff(output_x) 2025-12-04T09:49:21.1304380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.1304473Z output = self.activation_function(output) 2025-12-04T09:49:21.1304680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.1304753Z return self.act(input) 2025-12-04T09:49:21.1304756Z 2025-12-04T09:49:21.1304829Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1304924Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1305119Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1305178Z res = mod(**inputs) 2025-12-04T09:49:21.1305429Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1305522Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1305888Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1306009Z outputs = layer_module( 2025-12-04T09:49:21.1306255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1306320Z outputs = self.rel_attn( 2025-12-04T09:49:21.1306569Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.1306661Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.1306665Z 2025-12-04T09:49:21.1306771Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1306960Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1307069Z res = mod(**inputs) 2025-12-04T09:49:21.1307327Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1307409Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1307655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1307728Z outputs = layer_module( 2025-12-04T09:49:21.1307968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1308044Z outputs = self.rel_attn( 2025-12-04T09:49:21.1308281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.1308373Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.1308378Z 2025-12-04T09:49:21.1308484Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1308671Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1308738Z res = mod(**inputs) 2025-12-04T09:49:21.1308982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1309058Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1309305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1309368Z outputs = layer_module( 2025-12-04T09:49:21.1309604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1309674Z outputs = self.rel_attn( 2025-12-04T09:49:21.1309912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1309991Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1310249Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.1310378Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.1310381Z 2025-12-04T09:49:21.1310483Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1310668Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1310736Z res = mod(**inputs) 2025-12-04T09:49:21.1310978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1311055Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1311301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1311367Z outputs = layer_module( 2025-12-04T09:49:21.1311624Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1311696Z outputs = self.rel_attn( 2025-12-04T09:49:21.1311948Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.1312079Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.1312082Z 2025-12-04T09:49:21.1312175Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1312365Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1312431Z res = mod(**inputs) 2025-12-04T09:49:21.1312673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1312786Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1313081Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1313153Z outputs = layer_module( 2025-12-04T09:49:21.1313405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1313470Z outputs = self.rel_attn( 2025-12-04T09:49:21.1313713Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1313790Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1314052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.1314195Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.1314200Z 2025-12-04T09:49:21.1314294Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1314479Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1314547Z res = mod(**inputs) 2025-12-04T09:49:21.1314794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1314878Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1315125Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1315190Z outputs = layer_module( 2025-12-04T09:49:21.1315444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1315509Z outputs = self.rel_attn( 2025-12-04T09:49:21.1315763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.1315872Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.1315875Z 2025-12-04T09:49:21.1315971Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1316171Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1316234Z res = mod(**inputs) 2025-12-04T09:49:21.1316489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1316576Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1316827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1316891Z outputs = layer_module( 2025-12-04T09:49:21.1317149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1317217Z outputs = self.rel_attn( 2025-12-04T09:49:21.1317488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1317557Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1317841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.1317968Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.1317972Z 2025-12-04T09:49:21.1318071Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1318270Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1318331Z res = mod(**inputs) 2025-12-04T09:49:21.1318581Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1318699Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1318953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1319017Z outputs = layer_module( 2025-12-04T09:49:21.1319278Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1319342Z outputs = self.rel_attn( 2025-12-04T09:49:21.1319593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1319677Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1319942Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1320060Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1320064Z 2025-12-04T09:49:21.1320164Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1320365Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1320428Z res = mod(**inputs) 2025-12-04T09:49:21.1320679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1320767Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1321017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1321082Z outputs = layer_module( 2025-12-04T09:49:21.1321344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1321410Z outputs = self.rel_attn( 2025-12-04T09:49:21.1321663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1321748Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1322017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1322130Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1322133Z 2025-12-04T09:49:21.1322210Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1322316Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1322505Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1322566Z res = mod(**inputs) 2025-12-04T09:49:21.1322819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1322895Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1323143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1323235Z outputs = layer_module( 2025-12-04T09:49:21.1323480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.1323706Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.1323965Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.1324040Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.1324296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.1324366Z output_x = self.ff(output_x) 2025-12-04T09:49:21.1324619Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.1324732Z output = self.activation_function(output) 2025-12-04T09:49:21.1324945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.1325018Z return self.act(input) 2025-12-04T09:49:21.1325023Z 2025-12-04T09:49:21.1325100Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1325199Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1325397Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1325456Z res = mod(**inputs) 2025-12-04T09:49:21.1325716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1325794Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1326039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1326113Z outputs = layer_module( 2025-12-04T09:49:21.1326361Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1326427Z outputs = self.rel_attn( 2025-12-04T09:49:21.1326682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.1326774Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.1326777Z 2025-12-04T09:49:21.1326881Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1327069Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1327129Z res = mod(**inputs) 2025-12-04T09:49:21.1327390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1327466Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1327718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1327778Z outputs = layer_module( 2025-12-04T09:49:21.1328015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1328085Z outputs = self.rel_attn( 2025-12-04T09:49:21.1328322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.1328413Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.1328416Z 2025-12-04T09:49:21.1328518Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1328702Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1328767Z res = mod(**inputs) 2025-12-04T09:49:21.1329032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1329110Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1329373Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1329436Z outputs = layer_module( 2025-12-04T09:49:21.1329681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1329743Z outputs = self.rel_attn( 2025-12-04T09:49:21.1329979Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1330053Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1330307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.1330458Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.1330462Z 2025-12-04T09:49:21.1330566Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1330750Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1330816Z res = mod(**inputs) 2025-12-04T09:49:21.1331057Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1331132Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1331378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1331439Z outputs = layer_module( 2025-12-04T09:49:21.1331677Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1331749Z outputs = self.rel_attn( 2025-12-04T09:49:21.1331986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.1332117Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.1332120Z 2025-12-04T09:49:21.1332215Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1332399Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1332465Z res = mod(**inputs) 2025-12-04T09:49:21.1332706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1332789Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1333029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1333092Z outputs = layer_module( 2025-12-04T09:49:21.1333343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1333405Z outputs = self.rel_attn( 2025-12-04T09:49:21.1333643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1333718Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1333978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.1334105Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.1334108Z 2025-12-04T09:49:21.1334201Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1334384Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1334453Z res = mod(**inputs) 2025-12-04T09:49:21.1334714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1334799Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1335054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1335120Z outputs = layer_module( 2025-12-04T09:49:21.1335368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1335432Z outputs = self.rel_attn( 2025-12-04T09:49:21.1335673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.1335773Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.1335776Z 2025-12-04T09:49:21.1335871Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1336092Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1336152Z res = mod(**inputs) 2025-12-04T09:49:21.1336391Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1336476Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1336717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1336788Z outputs = layer_module( 2025-12-04T09:49:21.1337033Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1337095Z outputs = self.rel_attn( 2025-12-04T09:49:21.1337340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1337404Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1337664Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.1337788Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.1337791Z 2025-12-04T09:49:21.1337888Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1338085Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1338144Z res = mod(**inputs) 2025-12-04T09:49:21.1338388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1338472Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1338715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1338786Z outputs = layer_module( 2025-12-04T09:49:21.1339034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1339099Z outputs = self.rel_attn( 2025-12-04T09:49:21.1339345Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1339427Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1339688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1339798Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1339801Z 2025-12-04T09:49:21.1339895Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1340090Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1340150Z res = mod(**inputs) 2025-12-04T09:49:21.1340394Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1340503Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1340762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1340827Z outputs = layer_module( 2025-12-04T09:49:21.1341074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1341137Z outputs = self.rel_attn( 2025-12-04T09:49:21.1341382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1341462Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1341721Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1341860Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1341864Z 2025-12-04T09:49:21.1341940Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1342041Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1342225Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1342283Z res = mod(**inputs) 2025-12-04T09:49:21.1342531Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1342606Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1342846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1342915Z outputs = layer_module( 2025-12-04T09:49:21.1343152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.1343361Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.1343613Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.1343688Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.1343935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.1344004Z output_x = self.ff(output_x) 2025-12-04T09:49:21.1344251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.1344332Z output = self.activation_function(output) 2025-12-04T09:49:21.1344538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.1344612Z return self.act(input) 2025-12-04T09:49:21.1344616Z 2025-12-04T09:49:21.1344690Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1344785Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1344974Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1345034Z res = mod(**inputs) 2025-12-04T09:49:21.1345283Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1345359Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1345598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1345669Z outputs = layer_module( 2025-12-04T09:49:21.1345907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1345975Z outputs = self.rel_attn( 2025-12-04T09:49:21.1346238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.1346331Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.1346334Z 2025-12-04T09:49:21.1346452Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1346642Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1346704Z res = mod(**inputs) 2025-12-04T09:49:21.1346953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1347028Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1347277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1347339Z outputs = layer_module( 2025-12-04T09:49:21.1347608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1347681Z outputs = self.rel_attn( 2025-12-04T09:49:21.1347921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.1348012Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.1348024Z 2025-12-04T09:49:21.1348116Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1348301Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1348369Z res = mod(**inputs) 2025-12-04T09:49:21.1348610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1348687Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1348946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1349013Z outputs = layer_module( 2025-12-04T09:49:21.1349262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1349326Z outputs = self.rel_attn( 2025-12-04T09:49:21.1349566Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1349642Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1349899Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.1350023Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.1350034Z 2025-12-04T09:49:21.1350130Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1350315Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1350383Z res = mod(**inputs) 2025-12-04T09:49:21.1350626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1350703Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1350951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1351012Z outputs = layer_module( 2025-12-04T09:49:21.1351264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1351328Z outputs = self.rel_attn( 2025-12-04T09:49:21.1351566Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.1351697Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.1351702Z 2025-12-04T09:49:21.1351814Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1351999Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1352068Z res = mod(**inputs) 2025-12-04T09:49:21.1352339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1352423Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1352672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1352735Z outputs = layer_module( 2025-12-04T09:49:21.1353053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1353133Z outputs = self.rel_attn( 2025-12-04T09:49:21.1353382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1353497Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1353759Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.1353895Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.1353898Z 2025-12-04T09:49:21.1353997Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1354197Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1354270Z res = mod(**inputs) 2025-12-04T09:49:21.1354515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1354600Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1354839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1354906Z outputs = layer_module( 2025-12-04T09:49:21.1355205Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1355271Z outputs = self.rel_attn( 2025-12-04T09:49:21.1355516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.1355617Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.1355621Z 2025-12-04T09:49:21.1355717Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1355911Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1355972Z res = mod(**inputs) 2025-12-04T09:49:21.1356230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1356321Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1356571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1356646Z outputs = layer_module( 2025-12-04T09:49:21.1356893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1356956Z outputs = self.rel_attn( 2025-12-04T09:49:21.1357209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1357276Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1357548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.1357670Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.1357676Z 2025-12-04T09:49:21.1357773Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1357989Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1358050Z res = mod(**inputs) 2025-12-04T09:49:21.1358313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1358398Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1358649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1358719Z outputs = layer_module( 2025-12-04T09:49:21.1358963Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1359028Z outputs = self.rel_attn( 2025-12-04T09:49:21.1359277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1359397Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1359664Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1359777Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1359781Z 2025-12-04T09:49:21.1359876Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1360072Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1360131Z res = mod(**inputs) 2025-12-04T09:49:21.1360380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1360467Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1360717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1360788Z outputs = layer_module( 2025-12-04T09:49:21.1361033Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1361099Z outputs = self.rel_attn( 2025-12-04T09:49:21.1361352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1361437Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1361708Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1361820Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1361824Z 2025-12-04T09:49:21.1361904Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1362008Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1362199Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1362262Z res = mod(**inputs) 2025-12-04T09:49:21.1362518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1362597Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1362846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1362918Z outputs = layer_module( 2025-12-04T09:49:21.1363161Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.1363372Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.1363630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.1363707Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.1363978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.1364065Z output_x = self.ff(output_x) 2025-12-04T09:49:21.1364320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.1364404Z output = self.activation_function(output) 2025-12-04T09:49:21.1364616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.1364690Z return self.act(input) 2025-12-04T09:49:21.1364693Z 2025-12-04T09:49:21.1364770Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1364867Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1365065Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1365158Z res = mod(**inputs) 2025-12-04T09:49:21.1365418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1365496Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1365746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1365817Z outputs = layer_module( 2025-12-04T09:49:21.1366065Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1366137Z outputs = self.rel_attn( 2025-12-04T09:49:21.1366388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.1366481Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.1366487Z 2025-12-04T09:49:21.1366590Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1366779Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1366839Z res = mod(**inputs) 2025-12-04T09:49:21.1367095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1367171Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1367427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1367492Z outputs = layer_module( 2025-12-04T09:49:21.1367747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1367818Z outputs = self.rel_attn( 2025-12-04T09:49:21.1368059Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.1368154Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.1368165Z 2025-12-04T09:49:21.1368258Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1368444Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1368510Z res = mod(**inputs) 2025-12-04T09:49:21.1368752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1368829Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1369080Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1369142Z outputs = layer_module( 2025-12-04T09:49:21.1369388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1369454Z outputs = self.rel_attn( 2025-12-04T09:49:21.1369725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1369799Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1370073Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.1370197Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.1370207Z 2025-12-04T09:49:21.1370301Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1370484Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1370549Z res = mod(**inputs) 2025-12-04T09:49:21.1370788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1370917Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1371162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1371225Z outputs = layer_module( 2025-12-04T09:49:21.1371466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1371529Z outputs = self.rel_attn( 2025-12-04T09:49:21.1371762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.1371891Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.1371894Z 2025-12-04T09:49:21.1371987Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1372167Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1372234Z res = mod(**inputs) 2025-12-04T09:49:21.1372479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1372562Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1372796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1372859Z outputs = layer_module( 2025-12-04T09:49:21.1373099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1373161Z outputs = self.rel_attn( 2025-12-04T09:49:21.1373397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1373465Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1373712Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.1373840Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.1373843Z 2025-12-04T09:49:21.1373939Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1374120Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1374187Z res = mod(**inputs) 2025-12-04T09:49:21.1374420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1374501Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1374733Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1374794Z outputs = layer_module( 2025-12-04T09:49:21.1375033Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1375097Z outputs = self.rel_attn( 2025-12-04T09:49:21.1375349Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.1375442Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.1375459Z 2025-12-04T09:49:21.1375554Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1375752Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1375811Z res = mod(**inputs) 2025-12-04T09:49:21.1376049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1376132Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1376371Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1376479Z outputs = layer_module( 2025-12-04T09:49:21.1376719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1376782Z outputs = self.rel_attn( 2025-12-04T09:49:21.1377029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1377095Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1377349Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.1377471Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.1377474Z 2025-12-04T09:49:21.1377568Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1377761Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1377821Z res = mod(**inputs) 2025-12-04T09:49:21.1378069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1378154Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1378402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1378473Z outputs = layer_module( 2025-12-04T09:49:21.1378713Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1378776Z outputs = self.rel_attn( 2025-12-04T09:49:21.1379021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1379101Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1379361Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1379475Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1379478Z 2025-12-04T09:49:21.1379572Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1379764Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1379824Z res = mod(**inputs) 2025-12-04T09:49:21.1380067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1380149Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1380390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1380460Z outputs = layer_module( 2025-12-04T09:49:21.1380698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1380764Z outputs = self.rel_attn( 2025-12-04T09:49:21.1381026Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1381107Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1381387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1381498Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1381501Z 2025-12-04T09:49:21.1381575Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1381675Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1381859Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1381917Z res = mod(**inputs) 2025-12-04T09:49:21.1382161Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1382272Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1382522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1382586Z outputs = layer_module( 2025-12-04T09:49:21.1382827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.1383031Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.1383282Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.1383356Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.1383604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.1383675Z output_x = self.ff(output_x) 2025-12-04T09:49:21.1383923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.1384003Z output = self.activation_function(output) 2025-12-04T09:49:21.1384208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.1384282Z return self.act(input) 2025-12-04T09:49:21.1384285Z 2025-12-04T09:49:21.1384359Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1384461Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1384645Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1384702Z res = mod(**inputs) 2025-12-04T09:49:21.1384951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1385030Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1385274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1385345Z outputs = layer_module( 2025-12-04T09:49:21.1385588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1385660Z outputs = self.rel_attn( 2025-12-04T09:49:21.1385901Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.1385991Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.1385995Z 2025-12-04T09:49:21.1386094Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1386278Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1386337Z res = mod(**inputs) 2025-12-04T09:49:21.1386601Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1386677Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1386936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1387002Z outputs = layer_module( 2025-12-04T09:49:21.1387240Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1387314Z outputs = self.rel_attn( 2025-12-04T09:49:21.1387554Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.1387657Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.1387660Z 2025-12-04T09:49:21.1387755Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1387971Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1388040Z res = mod(**inputs) 2025-12-04T09:49:21.1388284Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1388361Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1388613Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1388674Z outputs = layer_module( 2025-12-04T09:49:21.1388921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1388986Z outputs = self.rel_attn( 2025-12-04T09:49:21.1389225Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1389301Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1389561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.1389694Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.1389697Z 2025-12-04T09:49:21.1389792Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1389978Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1390047Z res = mod(**inputs) 2025-12-04T09:49:21.1390291Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1390368Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1390618Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1390680Z outputs = layer_module( 2025-12-04T09:49:21.1390932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1390997Z outputs = self.rel_attn( 2025-12-04T09:49:21.1391240Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.1391375Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.1391378Z 2025-12-04T09:49:21.1391473Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1391666Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1391726Z res = mod(**inputs) 2025-12-04T09:49:21.1391968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1392052Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1392315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1392379Z outputs = layer_module( 2025-12-04T09:49:21.1392642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1392707Z outputs = self.rel_attn( 2025-12-04T09:49:21.1392962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1393102Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1393373Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.1393510Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.1393513Z 2025-12-04T09:49:21.1393611Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1393841Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1393906Z res = mod(**inputs) 2025-12-04T09:49:21.1394169Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1394258Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1394501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1394566Z outputs = layer_module( 2025-12-04T09:49:21.1394814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1394877Z outputs = self.rel_attn( 2025-12-04T09:49:21.1395124Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.1395217Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.1395222Z 2025-12-04T09:49:21.1395323Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1395518Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1395579Z res = mod(**inputs) 2025-12-04T09:49:21.1395822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1395907Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1396150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1396222Z outputs = layer_module( 2025-12-04T09:49:21.1396462Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1396525Z outputs = self.rel_attn( 2025-12-04T09:49:21.1396776Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1396843Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1397105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.1397220Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.1397223Z 2025-12-04T09:49:21.1397318Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1397514Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1397573Z res = mod(**inputs) 2025-12-04T09:49:21.1397815Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1397901Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1398138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1398224Z outputs = layer_module( 2025-12-04T09:49:21.1398466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1398557Z outputs = self.rel_attn( 2025-12-04T09:49:21.1398804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1398888Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1399156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1399260Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1399263Z 2025-12-04T09:49:21.1399358Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1399566Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1399643Z res = mod(**inputs) 2025-12-04T09:49:21.1399887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1399970Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1400214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1421556Z outputs = layer_module( 2025-12-04T09:49:21.1422060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1422138Z outputs = self.rel_attn( 2025-12-04T09:49:21.1422420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1422515Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1422810Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1422935Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1422942Z 2025-12-04T09:49:21.1423025Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1423139Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1423342Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1423409Z res = mod(**inputs) 2025-12-04T09:49:21.1423671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1423755Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1423997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1424076Z outputs = layer_module( 2025-12-04T09:49:21.1424323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.1424535Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.1424789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.1424866Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.1425116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.1425188Z output_x = self.ff(output_x) 2025-12-04T09:49:21.1425436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.1425520Z output = self.activation_function(output) 2025-12-04T09:49:21.1425864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.1425947Z return self.act(input) 2025-12-04T09:49:21.1425952Z 2025-12-04T09:49:21.1426030Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1426176Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1426383Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1426445Z res = mod(**inputs) 2025-12-04T09:49:21.1426697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1426781Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1427024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1427100Z outputs = layer_module( 2025-12-04T09:49:21.1427413Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1427479Z outputs = self.rel_attn( 2025-12-04T09:49:21.1427732Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.1427828Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.1427832Z 2025-12-04T09:49:21.1427937Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1428131Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1428193Z res = mod(**inputs) 2025-12-04T09:49:21.1428447Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1428525Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1428776Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1428845Z outputs = layer_module( 2025-12-04T09:49:21.1429089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1429167Z outputs = self.rel_attn( 2025-12-04T09:49:21.1429406Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.1429504Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.1429516Z 2025-12-04T09:49:21.1429614Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1429802Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1429871Z res = mod(**inputs) 2025-12-04T09:49:21.1430118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1430199Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1430454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1430518Z outputs = layer_module( 2025-12-04T09:49:21.1430769Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1430832Z outputs = self.rel_attn( 2025-12-04T09:49:21.1431074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1431153Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1431414Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.1431544Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.1431558Z 2025-12-04T09:49:21.1431654Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1431859Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1431927Z res = mod(**inputs) 2025-12-04T09:49:21.1432192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1432270Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1432519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1432581Z outputs = layer_module( 2025-12-04T09:49:21.1432833Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1432899Z outputs = self.rel_attn( 2025-12-04T09:49:21.1433224Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.1433416Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.1433421Z 2025-12-04T09:49:21.1433523Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1433713Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1433783Z res = mod(**inputs) 2025-12-04T09:49:21.1434041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1434127Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1434369Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1434434Z outputs = layer_module( 2025-12-04T09:49:21.1434680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1434747Z outputs = self.rel_attn( 2025-12-04T09:49:21.1434990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1435069Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1435327Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.1435460Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.1435463Z 2025-12-04T09:49:21.1435559Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1435747Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1435818Z res = mod(**inputs) 2025-12-04T09:49:21.1436060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1436149Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1436393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1436457Z outputs = layer_module( 2025-12-04T09:49:21.1436708Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1436773Z outputs = self.rel_attn( 2025-12-04T09:49:21.1437015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.1437118Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.1437121Z 2025-12-04T09:49:21.1437217Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1437412Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1437474Z res = mod(**inputs) 2025-12-04T09:49:21.1437730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1437816Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1438070Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1438142Z outputs = layer_module( 2025-12-04T09:49:21.1438384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1438447Z outputs = self.rel_attn( 2025-12-04T09:49:21.1438698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1438766Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1439025Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.1439181Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.1439184Z 2025-12-04T09:49:21.1439279Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1439475Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1439534Z res = mod(**inputs) 2025-12-04T09:49:21.1439773Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1439857Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1440100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1440170Z outputs = layer_module( 2025-12-04T09:49:21.1440411Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1440477Z outputs = self.rel_attn( 2025-12-04T09:49:21.1440724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1440810Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1441079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1441186Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1441190Z 2025-12-04T09:49:21.1441284Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1441476Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1441535Z res = mod(**inputs) 2025-12-04T09:49:21.1441788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1441875Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1442116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1442187Z outputs = layer_module( 2025-12-04T09:49:21.1442425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1442488Z outputs = self.rel_attn( 2025-12-04T09:49:21.1442738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1442821Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1443089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1443194Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1443200Z 2025-12-04T09:49:21.1443275Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1443392Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1443576Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1443650Z res = mod(**inputs) 2025-12-04T09:49:21.1443901Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1443976Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1444222Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1444284Z outputs = layer_module( 2025-12-04T09:49:21.1444523Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.1444730Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.1445016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.1445099Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.1445335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.1445404Z output_x = self.ff(output_x) 2025-12-04T09:49:21.1445644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.1445725Z output = self.activation_function(output) 2025-12-04T09:49:21.1445925Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.1445998Z return self.act(input) 2025-12-04T09:49:21.1446002Z 2025-12-04T09:49:21.1446074Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1446178Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1446362Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1446420Z res = mod(**inputs) 2025-12-04T09:49:21.1446666Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1446741Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1446977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1447047Z outputs = layer_module( 2025-12-04T09:49:21.1447284Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1447354Z outputs = self.rel_attn( 2025-12-04T09:49:21.1447586Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.1447681Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.1447684Z 2025-12-04T09:49:21.1447785Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1447969Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1448035Z res = mod(**inputs) 2025-12-04T09:49:21.1448274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1448350Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1448594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1448656Z outputs = layer_module( 2025-12-04T09:49:21.1448889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1448965Z outputs = self.rel_attn( 2025-12-04T09:49:21.1449214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.1449331Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.1449334Z 2025-12-04T09:49:21.1449432Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1449618Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1449688Z res = mod(**inputs) 2025-12-04T09:49:21.1449932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1450016Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1450261Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1450353Z outputs = layer_module( 2025-12-04T09:49:21.1450605Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1450672Z outputs = self.rel_attn( 2025-12-04T09:49:21.1450921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1451002Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1451265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.1451401Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.1451404Z 2025-12-04T09:49:21.1451503Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1451693Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1451766Z res = mod(**inputs) 2025-12-04T09:49:21.1452013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1452099Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1452343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1452409Z outputs = layer_module( 2025-12-04T09:49:21.1452654Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1452721Z outputs = self.rel_attn( 2025-12-04T09:49:21.1452960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.1453098Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.1453101Z 2025-12-04T09:49:21.1453198Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1453398Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1453461Z res = mod(**inputs) 2025-12-04T09:49:21.1453705Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1453794Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1454038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1454105Z outputs = layer_module( 2025-12-04T09:49:21.1454354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1454420Z outputs = self.rel_attn( 2025-12-04T09:49:21.1454668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1454742Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1455017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.1455161Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.1455164Z 2025-12-04T09:49:21.1455259Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1455452Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1455512Z res = mod(**inputs) 2025-12-04T09:49:21.1455754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1455836Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1456074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1456161Z outputs = layer_module( 2025-12-04T09:49:21.1456410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1456473Z outputs = self.rel_attn( 2025-12-04T09:49:21.1456720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.1456812Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.1456816Z 2025-12-04T09:49:21.1456909Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1457103Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1457163Z res = mod(**inputs) 2025-12-04T09:49:21.1457410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1457485Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1457728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1457797Z outputs = layer_module( 2025-12-04T09:49:21.1458038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1458103Z outputs = self.rel_attn( 2025-12-04T09:49:21.1458352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1458417Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1458682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.1458799Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.1458803Z 2025-12-04T09:49:21.1458898Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1459098Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1459157Z res = mod(**inputs) 2025-12-04T09:49:21.1459411Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1459487Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1459730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1459799Z outputs = layer_module( 2025-12-04T09:49:21.1460038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1460102Z outputs = self.rel_attn( 2025-12-04T09:49:21.1460350Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1460435Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1460719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1460826Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1460843Z 2025-12-04T09:49:21.1460939Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1461131Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1461193Z res = mod(**inputs) 2025-12-04T09:49:21.1461446Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1461526Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1461770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1461855Z outputs = layer_module( 2025-12-04T09:49:21.1462111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1462174Z outputs = self.rel_attn( 2025-12-04T09:49:21.1462420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1462501Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1462768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1462870Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1462874Z 2025-12-04T09:49:21.1462949Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1463053Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1463237Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1463298Z res = mod(**inputs) 2025-12-04T09:49:21.1463549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1463627Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1463876Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1463937Z outputs = layer_module( 2025-12-04T09:49:21.1464178Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.1464387Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.1464641Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.1464723Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.1464969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.1465039Z output_x = self.ff(output_x) 2025-12-04T09:49:21.1465289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.1465371Z output = self.activation_function(output) 2025-12-04T09:49:21.1465575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.1465649Z return self.act(input) 2025-12-04T09:49:21.1465652Z 2025-12-04T09:49:21.1465726Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1465830Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1466014Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1466073Z res = mod(**inputs) 2025-12-04T09:49:21.1466339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1466417Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1466677Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1466740Z outputs = layer_module( 2025-12-04T09:49:21.1466980Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1467052Z outputs = self.rel_attn( 2025-12-04T09:49:21.1467290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.1467380Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.1467383Z 2025-12-04T09:49:21.1467485Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1467710Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1467781Z res = mod(**inputs) 2025-12-04T09:49:21.1468024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1468103Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1468350Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1468412Z outputs = layer_module( 2025-12-04T09:49:21.1468651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1468722Z outputs = self.rel_attn( 2025-12-04T09:49:21.1468966Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.1469068Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.1469073Z 2025-12-04T09:49:21.1469168Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1469357Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1469428Z res = mod(**inputs) 2025-12-04T09:49:21.1469670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1469753Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1469994Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1470055Z outputs = layer_module( 2025-12-04T09:49:21.1470301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1470363Z outputs = self.rel_attn( 2025-12-04T09:49:21.1470604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1470679Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1470937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.1471069Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.1471073Z 2025-12-04T09:49:21.1471167Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1471352Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1471420Z res = mod(**inputs) 2025-12-04T09:49:21.1471659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1471741Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1471982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1472061Z outputs = layer_module( 2025-12-04T09:49:21.1472319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1472383Z outputs = self.rel_attn( 2025-12-04T09:49:21.1472616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.1472746Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.1472749Z 2025-12-04T09:49:21.1472842Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1473111Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1473179Z res = mod(**inputs) 2025-12-04T09:49:21.1473425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1473541Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1473782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1473854Z outputs = layer_module( 2025-12-04T09:49:21.1474094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1474160Z outputs = self.rel_attn( 2025-12-04T09:49:21.1474407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1474474Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1474729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.1474859Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.1474865Z 2025-12-04T09:49:21.1474962Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1475153Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1475216Z res = mod(**inputs) 2025-12-04T09:49:21.1475458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1475545Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1475785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1475854Z outputs = layer_module( 2025-12-04T09:49:21.1476093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1476156Z outputs = self.rel_attn( 2025-12-04T09:49:21.1476406Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.1476497Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.1476501Z 2025-12-04T09:49:21.1476597Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1476793Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1476854Z res = mod(**inputs) 2025-12-04T09:49:21.1477100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1477174Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1477412Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1477482Z outputs = layer_module( 2025-12-04T09:49:21.1477715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1477794Z outputs = self.rel_attn( 2025-12-04T09:49:21.1478042Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1478123Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1478387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.1478511Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.1478514Z 2025-12-04T09:49:21.1478609Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1478791Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1478860Z res = mod(**inputs) 2025-12-04T09:49:21.1479101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1479208Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1479454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1479518Z outputs = layer_module( 2025-12-04T09:49:21.1479760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1479823Z outputs = self.rel_attn( 2025-12-04T09:49:21.1480059Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1480148Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1480412Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1480522Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1480527Z 2025-12-04T09:49:21.1480623Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1480805Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1480873Z res = mod(**inputs) 2025-12-04T09:49:21.1481109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1481185Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1481430Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1481494Z outputs = layer_module( 2025-12-04T09:49:21.1481736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1481799Z outputs = self.rel_attn( 2025-12-04T09:49:21.1482043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1482133Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1482395Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1482498Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1482508Z 2025-12-04T09:49:21.1482582Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1482675Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1482865Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1482924Z res = mod(**inputs) 2025-12-04T09:49:21.1483167Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1483249Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1483503Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1483574Z outputs = layer_module( 2025-12-04T09:49:21.1483857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.1484058Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.1484319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.1484394Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.1484643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.1484717Z output_x = self.ff(output_x) 2025-12-04T09:49:21.1484955Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.1485075Z output = self.activation_function(output) 2025-12-04T09:49:21.1485280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.1485345Z return self.act(input) 2025-12-04T09:49:21.1485348Z 2025-12-04T09:49:21.1485429Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1485523Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1485715Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1485774Z res = mod(**inputs) 2025-12-04T09:49:21.1486019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1486100Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1486343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1486409Z outputs = layer_module( 2025-12-04T09:49:21.1486658Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1486721Z outputs = self.rel_attn( 2025-12-04T09:49:21.1486971Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.1487060Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.1487063Z 2025-12-04T09:49:21.1487157Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1487353Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1487413Z res = mod(**inputs) 2025-12-04T09:49:21.1487660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1487746Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1487993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1488065Z outputs = layer_module( 2025-12-04T09:49:21.1488306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1488370Z outputs = self.rel_attn( 2025-12-04T09:49:21.1488618Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.1488710Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.1488713Z 2025-12-04T09:49:21.1488813Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1488999Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1489060Z res = mod(**inputs) 2025-12-04T09:49:21.1489329Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1489406Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1489668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1489739Z outputs = layer_module( 2025-12-04T09:49:21.1489985Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1490055Z outputs = self.rel_attn( 2025-12-04T09:49:21.1490291Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1490358Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1490621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.1490804Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.1490808Z 2025-12-04T09:49:21.1490910Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1491099Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1491158Z res = mod(**inputs) 2025-12-04T09:49:21.1491408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1491484Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1491724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1491792Z outputs = layer_module( 2025-12-04T09:49:21.1492030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1492101Z outputs = self.rel_attn( 2025-12-04T09:49:21.1492339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.1492465Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.1492468Z 2025-12-04T09:49:21.1492574Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1492758Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1492824Z res = mod(**inputs) 2025-12-04T09:49:21.1493063Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1493139Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1493388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1493454Z outputs = layer_module( 2025-12-04T09:49:21.1493692Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1493763Z outputs = self.rel_attn( 2025-12-04T09:49:21.1494002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1494079Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1494335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.1494458Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.1494461Z 2025-12-04T09:49:21.1494564Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1494748Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1494817Z res = mod(**inputs) 2025-12-04T09:49:21.1495075Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1495155Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1495429Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1495492Z outputs = layer_module( 2025-12-04T09:49:21.1495733Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1495803Z outputs = self.rel_attn( 2025-12-04T09:49:21.1496037Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.1496134Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.1496137Z 2025-12-04T09:49:21.1496259Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1496444Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1496511Z res = mod(**inputs) 2025-12-04T09:49:21.1496752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1496828Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1497079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1497140Z outputs = layer_module( 2025-12-04T09:49:21.1497388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1497451Z outputs = self.rel_attn( 2025-12-04T09:49:21.1497695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1497771Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1498030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.1498154Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.1498157Z 2025-12-04T09:49:21.1498251Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1498434Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1498502Z res = mod(**inputs) 2025-12-04T09:49:21.1498745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1498820Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1499068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1499132Z outputs = layer_module( 2025-12-04T09:49:21.1499377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1499441Z outputs = self.rel_attn( 2025-12-04T09:49:21.1499682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1499770Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1500032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1500141Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1500145Z 2025-12-04T09:49:21.1500237Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1500419Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1500488Z res = mod(**inputs) 2025-12-04T09:49:21.1500744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1500820Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1501082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1501147Z outputs = layer_module( 2025-12-04T09:49:21.1501396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1501457Z outputs = self.rel_attn( 2025-12-04T09:49:21.1501695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1501784Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1502054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1502193Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1502197Z 2025-12-04T09:49:21.1502272Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1502367Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1502556Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1502616Z res = mod(**inputs) 2025-12-04T09:49:21.1502859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1502943Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1503185Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1503253Z outputs = layer_module( 2025-12-04T09:49:21.1503491Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.1503693Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.1503952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.1504025Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.1504273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.1504341Z output_x = self.ff(output_x) 2025-12-04T09:49:21.1504582Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.1504672Z output = self.activation_function(output) 2025-12-04T09:49:21.1504874Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.1504942Z return self.act(input) 2025-12-04T09:49:21.1504947Z 2025-12-04T09:49:21.1505029Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1505125Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1505317Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1505378Z res = mod(**inputs) 2025-12-04T09:49:21.1505618Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1505889Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1506138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1506202Z outputs = layer_module( 2025-12-04T09:49:21.1506452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1506521Z outputs = self.rel_attn( 2025-12-04T09:49:21.1506807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.1506922Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.1506926Z 2025-12-04T09:49:21.1507022Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1507214Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1507272Z res = mod(**inputs) 2025-12-04T09:49:21.1507524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1507599Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1507839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1507956Z outputs = layer_module( 2025-12-04T09:49:21.1508195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1508259Z outputs = self.rel_attn( 2025-12-04T09:49:21.1508511Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.1508604Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.1508607Z 2025-12-04T09:49:21.1508711Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1508894Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1508953Z res = mod(**inputs) 2025-12-04T09:49:21.1509201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1509277Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1509526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1509589Z outputs = layer_module( 2025-12-04T09:49:21.1509830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1509901Z outputs = self.rel_attn( 2025-12-04T09:49:21.1510139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1510206Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1510471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.1510592Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.1510597Z 2025-12-04T09:49:21.1510698Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1510888Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1510948Z res = mod(**inputs) 2025-12-04T09:49:21.1511196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1511273Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1511513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1511584Z outputs = layer_module( 2025-12-04T09:49:21.1511829Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1511898Z outputs = self.rel_attn( 2025-12-04T09:49:21.1512134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.1512260Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.1512263Z 2025-12-04T09:49:21.1512391Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1512591Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1512659Z res = mod(**inputs) 2025-12-04T09:49:21.1512901Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1512976Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1513275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1513343Z outputs = layer_module( 2025-12-04T09:49:21.1513587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1513674Z outputs = self.rel_attn( 2025-12-04T09:49:21.1513930Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1514004Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1514259Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.1514381Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.1514384Z 2025-12-04T09:49:21.1514489Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1514676Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1514747Z res = mod(**inputs) 2025-12-04T09:49:21.1514991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1515067Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1515328Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1515391Z outputs = layer_module( 2025-12-04T09:49:21.1515638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1515711Z outputs = self.rel_attn( 2025-12-04T09:49:21.1515952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.1516052Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.1516055Z 2025-12-04T09:49:21.1516149Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1516334Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1516404Z res = mod(**inputs) 2025-12-04T09:49:21.1516650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1516737Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1516981Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1517044Z outputs = layer_module( 2025-12-04T09:49:21.1517289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1517352Z outputs = self.rel_attn( 2025-12-04T09:49:21.1517593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1517666Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1517927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.1518053Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.1518056Z 2025-12-04T09:49:21.1518165Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1518350Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1518430Z res = mod(**inputs) 2025-12-04T09:49:21.1518674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1518749Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1518995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1519057Z outputs = layer_module( 2025-12-04T09:49:21.1519306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1519367Z outputs = self.rel_attn( 2025-12-04T09:49:21.1519634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1519724Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1519986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1520095Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1520098Z 2025-12-04T09:49:21.1520192Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1520376Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1520443Z res = mod(**inputs) 2025-12-04T09:49:21.1520683Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1520758Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1521013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1521077Z outputs = layer_module( 2025-12-04T09:49:21.1521322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1521386Z outputs = self.rel_attn( 2025-12-04T09:49:21.1521625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1521711Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1521969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1522079Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1522082Z 2025-12-04T09:49:21.1522156Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1522255Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1522448Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1522508Z res = mod(**inputs) 2025-12-04T09:49:21.1522753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1522837Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1523074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1523141Z outputs = layer_module( 2025-12-04T09:49:21.1523379Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.1523577Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.1523851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.1523927Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.1524190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.1524260Z output_x = self.ff(output_x) 2025-12-04T09:49:21.1524496Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.1524580Z output = self.activation_function(output) 2025-12-04T09:49:21.1524779Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.1524842Z return self.act(input) 2025-12-04T09:49:21.1524845Z 2025-12-04T09:49:21.1524925Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1525018Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1525232Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1525292Z res = mod(**inputs) 2025-12-04T09:49:21.1525533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1525617Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1525857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1525919Z outputs = layer_module( 2025-12-04T09:49:21.1526162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1526224Z outputs = self.rel_attn( 2025-12-04T09:49:21.1526469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.1526561Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.1526565Z 2025-12-04T09:49:21.1526658Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1526849Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1526908Z res = mod(**inputs) 2025-12-04T09:49:21.1527158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1527236Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1527479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1527548Z outputs = layer_module( 2025-12-04T09:49:21.1527789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1527852Z outputs = self.rel_attn( 2025-12-04T09:49:21.1528104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.1528195Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.1528199Z 2025-12-04T09:49:21.1528299Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1528483Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1528540Z res = mod(**inputs) 2025-12-04T09:49:21.1528790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1528865Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1529104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1529173Z outputs = layer_module( 2025-12-04T09:49:21.1529411Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1529497Z outputs = self.rel_attn( 2025-12-04T09:49:21.1529737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1529819Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1530085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.1530209Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.1530212Z 2025-12-04T09:49:21.1530313Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1530498Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1530557Z res = mod(**inputs) 2025-12-04T09:49:21.1530805Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1531226Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1531466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1531539Z outputs = layer_module( 2025-12-04T09:49:21.1531776Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1531846Z outputs = self.rel_attn( 2025-12-04T09:49:21.1532081Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.1532204Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.1532207Z 2025-12-04T09:49:21.1532309Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1532492Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1532560Z res = mod(**inputs) 2025-12-04T09:49:21.1532802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1532912Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1533193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1533256Z outputs = layer_module( 2025-12-04T09:49:21.1533497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1533567Z outputs = self.rel_attn( 2025-12-04T09:49:21.1533799Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1533873Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1534218Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.1534346Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.1534349Z 2025-12-04T09:49:21.1534454Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1534638Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1534703Z res = mod(**inputs) 2025-12-04T09:49:21.1534943Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1535018Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1535264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1535326Z outputs = layer_module( 2025-12-04T09:49:21.1535567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1535662Z outputs = self.rel_attn( 2025-12-04T09:49:21.1535900Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.1536015Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.1536019Z 2025-12-04T09:49:21.1536113Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1536296Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1536363Z res = mod(**inputs) 2025-12-04T09:49:21.1536602Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1536683Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1536922Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1537016Z outputs = layer_module( 2025-12-04T09:49:21.1537266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1537330Z outputs = self.rel_attn( 2025-12-04T09:49:21.1537568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1537644Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1537900Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.1538018Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.1538022Z 2025-12-04T09:49:21.1538115Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1538300Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1538368Z res = mod(**inputs) 2025-12-04T09:49:21.1538612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1538688Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1538939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1539000Z outputs = layer_module( 2025-12-04T09:49:21.1539246Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1539306Z outputs = self.rel_attn( 2025-12-04T09:49:21.1539544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1539630Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1539891Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1540002Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1540005Z 2025-12-04T09:49:21.1540098Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1540287Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1540352Z res = mod(**inputs) 2025-12-04T09:49:21.1540595Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1540669Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1540916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1540977Z outputs = layer_module( 2025-12-04T09:49:21.1541224Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1541302Z outputs = self.rel_attn( 2025-12-04T09:49:21.1541554Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1541656Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1541914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1542019Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1542022Z 2025-12-04T09:49:21.1542096Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1542190Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1542379Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1542437Z res = mod(**inputs) 2025-12-04T09:49:21.1542693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1542794Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1543030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1543097Z outputs = layer_module( 2025-12-04T09:49:21.1543330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.1543521Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.1543773Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.1543844Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.1544083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.1544153Z output_x = self.ff(output_x) 2025-12-04T09:49:21.1544384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.1544470Z output = self.activation_function(output) 2025-12-04T09:49:21.1544670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.1544734Z return self.act(input) 2025-12-04T09:49:21.1544737Z 2025-12-04T09:49:21.1544817Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1544910Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1545097Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1545154Z res = mod(**inputs) 2025-12-04T09:49:21.1545386Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1545471Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1545705Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1545768Z outputs = layer_module( 2025-12-04T09:49:21.1546005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1546065Z outputs = self.rel_attn( 2025-12-04T09:49:21.1546301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.1546388Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.1546391Z 2025-12-04T09:49:21.1546482Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1546669Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1546730Z res = mod(**inputs) 2025-12-04T09:49:21.1546984Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1547059Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1547312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1547382Z outputs = layer_module( 2025-12-04T09:49:21.1547616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1547677Z outputs = self.rel_attn( 2025-12-04T09:49:21.1547915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.1548006Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.1548009Z 2025-12-04T09:49:21.1548140Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1548326Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1548385Z res = mod(**inputs) 2025-12-04T09:49:21.1548637Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1548713Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1548958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1549020Z outputs = layer_module( 2025-12-04T09:49:21.1549257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1549326Z outputs = self.rel_attn( 2025-12-04T09:49:21.1549563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1549631Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1549899Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.1550022Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.1550025Z 2025-12-04T09:49:21.1550126Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1550309Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1550367Z res = mod(**inputs) 2025-12-04T09:49:21.1550614Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1550689Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1550934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1551000Z outputs = layer_module( 2025-12-04T09:49:21.1551240Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1551308Z outputs = self.rel_attn( 2025-12-04T09:49:21.1551544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.1551665Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.1551669Z 2025-12-04T09:49:21.1551769Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1551951Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1552015Z res = mod(**inputs) 2025-12-04T09:49:21.1552255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1552333Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1552594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1552655Z outputs = layer_module( 2025-12-04T09:49:21.1552916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1553058Z outputs = self.rel_attn( 2025-12-04T09:49:21.1553321Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1553394Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1553656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.1553779Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.1553782Z 2025-12-04T09:49:21.1553936Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1554121Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1554187Z res = mod(**inputs) 2025-12-04T09:49:21.1554430Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1554505Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1554751Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1554813Z outputs = layer_module( 2025-12-04T09:49:21.1555051Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1555121Z outputs = self.rel_attn( 2025-12-04T09:49:21.1555356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.1555455Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.1555460Z 2025-12-04T09:49:21.1555554Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1555747Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1555815Z res = mod(**inputs) 2025-12-04T09:49:21.1556064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1556148Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1556395Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1556457Z outputs = layer_module( 2025-12-04T09:49:21.1556708Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1556773Z outputs = self.rel_attn( 2025-12-04T09:49:21.1557020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1557092Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1557354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.1557475Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.1557478Z 2025-12-04T09:49:21.1557573Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1557759Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1557827Z res = mod(**inputs) 2025-12-04T09:49:21.1558078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1558162Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1558431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1558495Z outputs = layer_module( 2025-12-04T09:49:21.1558764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1558829Z outputs = self.rel_attn( 2025-12-04T09:49:21.1559078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1559167Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1559434Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1559545Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1559548Z 2025-12-04T09:49:21.1559658Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1559864Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1559931Z res = mod(**inputs) 2025-12-04T09:49:21.1560180Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1560262Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1560510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1560573Z outputs = layer_module( 2025-12-04T09:49:21.1560828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1560892Z outputs = self.rel_attn( 2025-12-04T09:49:21.1561139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1561230Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1561499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1561611Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1561614Z 2025-12-04T09:49:21.1561691Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1561787Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1561982Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1562043Z res = mod(**inputs) 2025-12-04T09:49:21.1562304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1562387Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1562638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1562710Z outputs = layer_module( 2025-12-04T09:49:21.1562954Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.1563157Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.1563420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.1563494Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.1563748Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.1563818Z output_x = self.ff(output_x) 2025-12-04T09:49:21.1564063Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.1564155Z output = self.activation_function(output) 2025-12-04T09:49:21.1564380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.1564446Z return self.act(input) 2025-12-04T09:49:21.1564456Z 2025-12-04T09:49:21.1564544Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1564641Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1564837Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1564896Z res = mod(**inputs) 2025-12-04T09:49:21.1565143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1565229Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1565477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1565577Z outputs = layer_module( 2025-12-04T09:49:21.1565825Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1565887Z outputs = self.rel_attn( 2025-12-04T09:49:21.1566139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.1566231Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.1566234Z 2025-12-04T09:49:21.1566328Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1566526Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1566586Z res = mod(**inputs) 2025-12-04T09:49:21.1566843Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1566920Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1567167Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1567237Z outputs = layer_module( 2025-12-04T09:49:21.1567490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1567553Z outputs = self.rel_attn( 2025-12-04T09:49:21.1567800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.1567889Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.1567893Z 2025-12-04T09:49:21.1567991Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1568175Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1568234Z res = mod(**inputs) 2025-12-04T09:49:21.1568484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1568560Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1568804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1568866Z outputs = layer_module( 2025-12-04T09:49:21.1569108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1569178Z outputs = self.rel_attn( 2025-12-04T09:49:21.1569417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1569482Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1569743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.1569867Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.1569870Z 2025-12-04T09:49:21.1569982Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1570172Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1570246Z res = mod(**inputs) 2025-12-04T09:49:21.1570498Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1570575Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1570826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1570888Z outputs = layer_module( 2025-12-04T09:49:21.1571128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1571212Z outputs = self.rel_attn( 2025-12-04T09:49:21.1571467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.1571591Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.1571601Z 2025-12-04T09:49:21.1571698Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1571877Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1571942Z res = mod(**inputs) 2025-12-04T09:49:21.1572183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1572258Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1572503Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1572565Z outputs = layer_module( 2025-12-04T09:49:21.1572816Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1572879Z outputs = self.rel_attn( 2025-12-04T09:49:21.1573119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1573190Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1573445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.1573564Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.1573568Z 2025-12-04T09:49:21.1573669Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1573850Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1573914Z res = mod(**inputs) 2025-12-04T09:49:21.1574156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1574234Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1574479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1574541Z outputs = layer_module( 2025-12-04T09:49:21.1574791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1574854Z outputs = self.rel_attn( 2025-12-04T09:49:21.1575088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.1575184Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.1575188Z 2025-12-04T09:49:21.1575279Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1575461Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1575570Z res = mod(**inputs) 2025-12-04T09:49:21.1575813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1575917Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1576154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1576214Z outputs = layer_module( 2025-12-04T09:49:21.1576458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1576521Z outputs = self.rel_attn( 2025-12-04T09:49:21.1576758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1576830Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1577112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.1577232Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.1577235Z 2025-12-04T09:49:21.1577330Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1577512Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1577580Z res = mod(**inputs) 2025-12-04T09:49:21.1577817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1577900Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1578139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1578200Z outputs = layer_module( 2025-12-04T09:49:21.1578444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1578508Z outputs = self.rel_attn( 2025-12-04T09:49:21.1578743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1578830Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1579083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1579193Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1579197Z 2025-12-04T09:49:21.1579291Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1579473Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1579542Z res = mod(**inputs) 2025-12-04T09:49:21.1579778Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1579865Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1580103Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1580164Z outputs = layer_module( 2025-12-04T09:49:21.1580404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1580465Z outputs = self.rel_attn( 2025-12-04T09:49:21.1580700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1580787Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1581041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1581149Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1581152Z 2025-12-04T09:49:21.1581241Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1581334Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1581540Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1581599Z res = mod(**inputs) 2025-12-04T09:49:21.1581846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1581920Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1582157Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1582223Z outputs = layer_module( 2025-12-04T09:49:21.1582461Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.1582697Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.1582953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.1583027Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.1583273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.1583341Z output_x = self.ff(output_x) 2025-12-04T09:49:21.1583581Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.1583667Z output = self.activation_function(output) 2025-12-04T09:49:21.1583869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.1583938Z return self.act(input) 2025-12-04T09:49:21.1583944Z 2025-12-04T09:49:21.1584017Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1584113Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1584305Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1584366Z res = mod(**inputs) 2025-12-04T09:49:21.1584608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1584690Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1584928Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1584996Z outputs = layer_module( 2025-12-04T09:49:21.1585234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1585296Z outputs = self.rel_attn( 2025-12-04T09:49:21.1585544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:49:21.1585635Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:49:21.1585638Z 2025-12-04T09:49:21.1585732Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1585925Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1585985Z res = mod(**inputs) 2025-12-04T09:49:21.1586234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1586309Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1586548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1586617Z outputs = layer_module( 2025-12-04T09:49:21.1586853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1586939Z outputs = self.rel_attn( 2025-12-04T09:49:21.1587178Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:49:21.1587283Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:49:21.1587286Z 2025-12-04T09:49:21.1587390Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1587575Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1587633Z res = mod(**inputs) 2025-12-04T09:49:21.1587894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1587970Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1588221Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1588311Z outputs = layer_module( 2025-12-04T09:49:21.1588543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1588615Z outputs = self.rel_attn( 2025-12-04T09:49:21.1588846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1588912Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1589167Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:49:21.1589286Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:49:21.1589289Z 2025-12-04T09:49:21.1589388Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1589570Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1589632Z res = mod(**inputs) 2025-12-04T09:49:21.1589878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1589953Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1590194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1590256Z outputs = layer_module( 2025-12-04T09:49:21.1590486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1590555Z outputs = self.rel_attn( 2025-12-04T09:49:21.1590788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:49:21.1590909Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:49:21.1590921Z 2025-12-04T09:49:21.1591015Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1591198Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1591265Z res = mod(**inputs) 2025-12-04T09:49:21.1591501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1591576Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1591819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1591881Z outputs = layer_module( 2025-12-04T09:49:21.1592120Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1592183Z outputs = self.rel_attn( 2025-12-04T09:49:21.1592416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1592508Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1592765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:49:21.1592899Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:49:21.1592909Z 2025-12-04T09:49:21.1593071Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1593280Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1593351Z res = mod(**inputs) 2025-12-04T09:49:21.1593613Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1593695Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1593971Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1594067Z outputs = layer_module( 2025-12-04T09:49:21.1594326Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1594391Z outputs = self.rel_attn( 2025-12-04T09:49:21.1594642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:49:21.1594744Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:49:21.1594748Z 2025-12-04T09:49:21.1594845Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1595035Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1595105Z res = mod(**inputs) 2025-12-04T09:49:21.1595362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1595446Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1595695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1595756Z outputs = layer_module( 2025-12-04T09:49:21.1596007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1596069Z outputs = self.rel_attn( 2025-12-04T09:49:21.1596321Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:49:21.1596389Z attn_vec = self.rel_attn_core( 2025-12-04T09:49:21.1596654Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:49:21.1596776Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:49:21.1596781Z 2025-12-04T09:49:21.1596877Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1597066Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1597136Z res = mod(**inputs) 2025-12-04T09:49:21.1597388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1597472Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1597723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1597785Z outputs = layer_module( 2025-12-04T09:49:21.1598040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1598103Z outputs = self.rel_attn( 2025-12-04T09:49:21.1598351Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1598466Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1598734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1598861Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1598864Z 2025-12-04T09:49:21.1598961Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1599150Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1599217Z res = mod(**inputs) 2025-12-04T09:49:21.1599466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1599550Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1599798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1599890Z outputs = layer_module( 2025-12-04T09:49:21.1600144Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:49:21.1600208Z outputs = self.rel_attn( 2025-12-04T09:49:21.1600458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:49:21.1600548Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:49:21.1600813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:49:21.1600925Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:49:21.1600928Z 2025-12-04T09:49:21.1601003Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1601098Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1601297Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1601360Z res = mod(**inputs) 2025-12-04T09:49:21.1601624Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:49:21.1601702Z transformer_outputs = self.transformer( 2025-12-04T09:49:21.1601955Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:49:21.1602025Z outputs = layer_module( 2025-12-04T09:49:21.1602273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:49:21.1602474Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:49:21.1602741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:21.1602816Z return forward_fn(*input_tensors) 2025-12-04T09:49:21.1603075Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:49:21.1603145Z output_x = self.ff(output_x) 2025-12-04T09:49:21.1603389Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:49:21.1603480Z output = self.activation_function(output) 2025-12-04T09:49:21.1603696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:21.1603768Z return self.act(input) 2025-12-04T09:49:21.1603771Z 2025-12-04T09:49:21.1603844Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1603916Z cudagraph partition due to non gpu ops 2025-12-04T09:49:21.1604018Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:21.1604208Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:21.1604285Z res = mod(**inputs) 2025-12-04T09:49:21.1604544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1639, in forward 2025-12-04T09:49:21.1604685Z loss = loss_fct(logits.view(-1, logits.size(-1)), labels.view(-1)) 2025-12-04T09:49:21.1604688Z 2025-12-04T09:49:33.3207115Z Compilation time (from dynamo_timed): 34.534347843 2025-12-04T09:49:33.3278024Z pass 2025-12-04T09:49:33.3280809Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:49:33.3285577Z TIMING: _recursive_pre_grad_passes:0.05634 _recursive_joint_graph_passes:1.26854 _recursive_post_grad_passes:0.14058 async_compile.wait:0.61607 code_gen:11.40652 inductor_compile:16.13534 backend_compile:28.78272 gc:0.00052 entire_frame_compile:34.53435 total_wall_time:34.53435 2025-12-04T09:49:33.3286950Z STATS: call_* op count: 818 | FakeTensorMode.__torch_dispatch__:68213 | FakeTensor.__torch_dispatch__:13735 | ProxyTorchDispatchMode.__torch_dispatch__:6838 2025-12-04T09:49:33.3287494Z Dynamo produced 1 graphs covering 818 ops with 0 graph breaks (0 unique) 2025-12-04T09:49:36.4673911Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:49:36.4674745Z import pynvml # type: ignore[import] 2025-12-04T09:49:39.5578697Z 2025-12-04T09:49:40.5904516Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:49:40.5909040Z loading model: 0it [00:01, ?it/s] 2025-12-04T09:49:40.5919385Z cpu eval YituTechConvBert 2025-12-04T09:49:41.4227712Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:49:41.6534532Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:49:41.8898272Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:49:56.5208455Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5213484Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5215995Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5216337Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5221423Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5223393Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5223637Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5223981Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5229112Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5229622Z res = mod(**inputs) 2025-12-04T09:49:56.5230124Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5231127Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5231651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5232111Z hidden_states = self.encoder( 2025-12-04T09:49:56.5234733Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5236800Z layer_outputs = layer_module( 2025-12-04T09:49:56.5237356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5242690Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5243358Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5243931Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5244684Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5245236Z self_outputs = self.self( 2025-12-04T09:49:56.5245680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5246175Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5246661Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:49:56.5247063Z x = self.depthwise(hidden_states) 2025-12-04T09:49:56.5247194Z 2025-12-04T09:49:56.5247304Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5247674Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5248103Z res = mod(**inputs) 2025-12-04T09:49:56.5248477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5248907Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5249313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5249725Z hidden_states = self.encoder( 2025-12-04T09:49:56.5250105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5250496Z layer_outputs = layer_module( 2025-12-04T09:49:56.5250846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5251204Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5251604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5252027Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5252437Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5252828Z self_outputs = self.self( 2025-12-04T09:49:56.5253251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5253718Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5254187Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:49:56.5254565Z x = self.pointwise(x) 2025-12-04T09:49:56.5254678Z 2025-12-04T09:49:56.5254758Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5255662Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5256130Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5256472Z res = mod(**inputs) 2025-12-04T09:49:56.5256883Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5257314Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5257722Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5258126Z hidden_states = self.encoder( 2025-12-04T09:49:56.5258525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5258918Z layer_outputs = layer_module( 2025-12-04T09:49:56.5259404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5259888Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5260300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5260731Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5261141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5261535Z self_outputs = self.self( 2025-12-04T09:49:56.5261917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:49:56.5262541Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:49:56.5262759Z 2025-12-04T09:49:56.5262841Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5263049Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5263327Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5263725Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5264051Z res = mod(**inputs) 2025-12-04T09:49:56.5264431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5264837Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5265254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5265648Z hidden_states = self.encoder( 2025-12-04T09:49:56.5266038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5266425Z layer_outputs = layer_module( 2025-12-04T09:49:56.5266770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5267134Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5267529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5267936Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5268391Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5268785Z self_outputs = self.self( 2025-12-04T09:49:56.5269158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:49:56.5269595Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:49:56.5269767Z 2025-12-04T09:49:56.5269853Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5270084Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5270432Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5270745Z res = mod(**inputs) 2025-12-04T09:49:56.5271114Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5271519Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5271926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5272321Z hidden_states = self.encoder( 2025-12-04T09:49:56.5272713Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5273208Z layer_outputs = layer_module( 2025-12-04T09:49:56.5273547Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5273901Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5274352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5274745Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5275150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5275534Z self_outputs = self.self( 2025-12-04T09:49:56.5275898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:49:56.5276319Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:49:56.5276475Z 2025-12-04T09:49:56.5276561Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5276762Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5276977Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5277358Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5277664Z res = mod(**inputs) 2025-12-04T09:49:56.5278023Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5278420Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5278811Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5279194Z hidden_states = self.encoder( 2025-12-04T09:49:56.5279562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5279946Z layer_outputs = layer_module( 2025-12-04T09:49:56.5280280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5280622Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5281009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:49:56.5281404Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:56.5281793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:56.5282164Z return forward_fn(*input_tensors) 2025-12-04T09:49:56.5282573Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:49:56.5283032Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:56.5283456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:49:56.5283868Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:56.5284240Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:56.5284568Z return self.act(input) 2025-12-04T09:49:56.5284672Z 2025-12-04T09:49:56.5284743Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5284943Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5285133Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5285324Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5285507Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5285695Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5285887Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5286070Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5286288Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5286630Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5286932Z res = mod(**inputs) 2025-12-04T09:49:56.5287307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5287699Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5288104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5288487Z hidden_states = self.encoder( 2025-12-04T09:49:56.5288857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5289231Z layer_outputs = layer_module( 2025-12-04T09:49:56.5289551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5289894Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5290272Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5290713Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5291090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5291468Z self_outputs = self.self( 2025-12-04T09:49:56.5291834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5292296Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5292755Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:49:56.5293145Z x = self.depthwise(hidden_states) 2025-12-04T09:49:56.5293265Z 2025-12-04T09:49:56.5293374Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5293709Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5294016Z res = mod(**inputs) 2025-12-04T09:49:56.5294378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5294768Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5295144Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5295524Z hidden_states = self.encoder( 2025-12-04T09:49:56.5295901Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5296277Z layer_outputs = layer_module( 2025-12-04T09:49:56.5296598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5296946Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5297334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5297720Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5298111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5298484Z self_outputs = self.self( 2025-12-04T09:49:56.5298848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5299300Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5299758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:49:56.5300134Z x = self.pointwise(x) 2025-12-04T09:49:56.5300237Z 2025-12-04T09:49:56.5300317Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5300548Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5300890Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5301211Z res = mod(**inputs) 2025-12-04T09:49:56.5301569Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5301959Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5302346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5302727Z hidden_states = self.encoder( 2025-12-04T09:49:56.5303094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5303473Z layer_outputs = layer_module( 2025-12-04T09:49:56.5303837Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5304178Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5304574Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5304972Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5305366Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5306095Z self_outputs = self.self( 2025-12-04T09:49:56.5306470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:49:56.5306907Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:49:56.5307075Z 2025-12-04T09:49:56.5307160Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5307359Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5307588Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5307938Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5308261Z res = mod(**inputs) 2025-12-04T09:49:56.5308622Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5309010Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5309394Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5309765Z hidden_states = self.encoder( 2025-12-04T09:49:56.5310140Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5310523Z layer_outputs = layer_module( 2025-12-04T09:49:56.5310850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5311198Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5311586Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5311974Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5312361Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5312739Z self_outputs = self.self( 2025-12-04T09:49:56.5313156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:49:56.5313594Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:49:56.5313763Z 2025-12-04T09:49:56.5313836Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5314065Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5314479Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5314779Z res = mod(**inputs) 2025-12-04T09:49:56.5315165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5315563Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5315951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5316325Z hidden_states = self.encoder( 2025-12-04T09:49:56.5316702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5317078Z layer_outputs = layer_module( 2025-12-04T09:49:56.5317402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5317798Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5318186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5318577Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5318958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5319339Z self_outputs = self.self( 2025-12-04T09:49:56.5319705Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:49:56.5320119Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:49:56.5320276Z 2025-12-04T09:49:56.5320351Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5320549Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5320776Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5321111Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5321418Z res = mod(**inputs) 2025-12-04T09:49:56.5321777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5322172Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5322549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5322928Z hidden_states = self.encoder( 2025-12-04T09:49:56.5323298Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5323669Z layer_outputs = layer_module( 2025-12-04T09:49:56.5323997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5324348Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5324728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:49:56.5325116Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:56.5325503Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:56.5325879Z return forward_fn(*input_tensors) 2025-12-04T09:49:56.5326284Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:49:56.5326731Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:56.5327151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:49:56.5327570Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:56.5327936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:56.5328265Z return self.act(input) 2025-12-04T09:49:56.5328390Z 2025-12-04T09:49:56.5328465Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5328664Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5328850Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5329043Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5329235Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5329421Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5329615Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5329806Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5330016Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5330358Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5330694Z res = mod(**inputs) 2025-12-04T09:49:56.5331054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5331441Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5331827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5332208Z hidden_states = self.encoder( 2025-12-04T09:49:56.5332573Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5332949Z layer_outputs = layer_module( 2025-12-04T09:49:56.5333273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5333614Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5333993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5334382Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5334767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5335143Z self_outputs = self.self( 2025-12-04T09:49:56.5335501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5335966Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5336434Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:49:56.5336815Z x = self.depthwise(hidden_states) 2025-12-04T09:49:56.5336936Z 2025-12-04T09:49:56.5337037Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5337381Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5337684Z res = mod(**inputs) 2025-12-04T09:49:56.5338034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5338428Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5338820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5339199Z hidden_states = self.encoder( 2025-12-04T09:49:56.5339562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5339940Z layer_outputs = layer_module( 2025-12-04T09:49:56.5340272Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5340628Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5341015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5341434Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5341824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5342194Z self_outputs = self.self( 2025-12-04T09:49:56.5342563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5343023Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5343480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:49:56.5343886Z x = self.pointwise(x) 2025-12-04T09:49:56.5343997Z 2025-12-04T09:49:56.5344073Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5344296Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5344630Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5344936Z res = mod(**inputs) 2025-12-04T09:49:56.5345293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5345682Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5346060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5346439Z hidden_states = self.encoder( 2025-12-04T09:49:56.5346808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5347192Z layer_outputs = layer_module( 2025-12-04T09:49:56.5347515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5347858Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5348243Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5348622Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5349006Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5349380Z self_outputs = self.self( 2025-12-04T09:49:56.5349745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:49:56.5350157Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:49:56.5350329Z 2025-12-04T09:49:56.5350402Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5350601Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5350812Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5351151Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5351455Z res = mod(**inputs) 2025-12-04T09:49:56.5351811Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5352193Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5352582Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5352959Z hidden_states = self.encoder( 2025-12-04T09:49:56.5353424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5353824Z layer_outputs = layer_module( 2025-12-04T09:49:56.5354193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5354541Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5354935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5355330Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5355720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5356103Z self_outputs = self.self( 2025-12-04T09:49:56.5356463Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:49:56.5356890Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:49:56.5357091Z 2025-12-04T09:49:56.5357174Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5357390Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5357733Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5358042Z res = mod(**inputs) 2025-12-04T09:49:56.5358399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5358781Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5359168Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5359551Z hidden_states = self.encoder( 2025-12-04T09:49:56.5359917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5360302Z layer_outputs = layer_module( 2025-12-04T09:49:56.5360632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5360973Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5361347Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5361742Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5362132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5362510Z self_outputs = self.self( 2025-12-04T09:49:56.5362873Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:49:56.5363292Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:49:56.5363443Z 2025-12-04T09:49:56.5363527Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5363720Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5363944Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5364287Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5364594Z res = mod(**inputs) 2025-12-04T09:49:56.5364945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5365335Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5365720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5366092Z hidden_states = self.encoder( 2025-12-04T09:49:56.5366464Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5366845Z layer_outputs = layer_module( 2025-12-04T09:49:56.5367194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5367528Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5367918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:49:56.5368310Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:56.5368693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:56.5369063Z return forward_fn(*input_tensors) 2025-12-04T09:49:56.5369471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:49:56.5369922Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:56.5370361Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:49:56.5370801Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:56.5371165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:56.5371492Z return self.act(input) 2025-12-04T09:49:56.5371597Z 2025-12-04T09:49:56.5371671Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5371869Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5372065Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5372249Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5372442Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5372634Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5372827Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5373009Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5373239Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5373582Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5373881Z res = mod(**inputs) 2025-12-04T09:49:56.5374245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5374640Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5375027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5375402Z hidden_states = self.encoder( 2025-12-04T09:49:56.5375773Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5376153Z layer_outputs = layer_module( 2025-12-04T09:49:56.5376479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5376828Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5377215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5377606Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5377986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5378365Z self_outputs = self.self( 2025-12-04T09:49:56.5378734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5379193Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5379650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:49:56.5380037Z x = self.depthwise(hidden_states) 2025-12-04T09:49:56.5380157Z 2025-12-04T09:49:56.5380274Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5380608Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5380927Z res = mod(**inputs) 2025-12-04T09:49:56.5381285Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5381677Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5382055Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5382435Z hidden_states = self.encoder( 2025-12-04T09:49:56.5382806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5383180Z layer_outputs = layer_module( 2025-12-04T09:49:56.5383541Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5383885Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5384273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5384651Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5385040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5385419Z self_outputs = self.self( 2025-12-04T09:49:56.5385788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5386240Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5386700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:49:56.5387085Z x = self.pointwise(x) 2025-12-04T09:49:56.5387187Z 2025-12-04T09:49:56.5387270Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5387490Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5387833Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5388135Z res = mod(**inputs) 2025-12-04T09:49:56.5388486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5388876Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5389263Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5389642Z hidden_states = self.encoder( 2025-12-04T09:49:56.5390013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5390398Z layer_outputs = layer_module( 2025-12-04T09:49:56.5390729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5391070Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5391453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5391843Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5392233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5392604Z self_outputs = self.self( 2025-12-04T09:49:56.5392970Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:49:56.5393474Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:49:56.5393639Z 2025-12-04T09:49:56.5393756Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5393952Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5394194Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5394533Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5394827Z res = mod(**inputs) 2025-12-04T09:49:56.5395181Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5395573Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5395947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5396327Z hidden_states = self.encoder( 2025-12-04T09:49:56.5396713Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5397131Z layer_outputs = layer_module( 2025-12-04T09:49:56.5397449Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5397788Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5398165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5398551Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5398926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5399301Z self_outputs = self.self( 2025-12-04T09:49:56.5399665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:49:56.5400090Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:49:56.5400266Z 2025-12-04T09:49:56.5400339Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5400560Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5400902Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5401198Z res = mod(**inputs) 2025-12-04T09:49:56.5401555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5401946Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5402333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5402704Z hidden_states = self.encoder( 2025-12-04T09:49:56.5403078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5403466Z layer_outputs = layer_module( 2025-12-04T09:49:56.5403791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5404136Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5404525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5404910Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5405469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5406041Z self_outputs = self.self( 2025-12-04T09:49:56.5406418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:49:56.5406840Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:49:56.5407002Z 2025-12-04T09:49:56.5407120Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5407321Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5407541Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5407894Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5408210Z res = mod(**inputs) 2025-12-04T09:49:56.5408570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5408960Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5409341Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5409725Z hidden_states = self.encoder( 2025-12-04T09:49:56.5410100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5410520Z layer_outputs = layer_module( 2025-12-04T09:49:56.5410851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5411197Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5411583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:49:56.5411966Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:56.5412351Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:56.5412725Z return forward_fn(*input_tensors) 2025-12-04T09:49:56.5413132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:49:56.5413581Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:56.5414009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:49:56.5414429Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:56.5414785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:56.5415110Z return self.act(input) 2025-12-04T09:49:56.5415223Z 2025-12-04T09:49:56.5415297Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5415489Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5415673Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5415861Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5416055Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5416238Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5416428Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5416624Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5416836Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5417182Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5417486Z res = mod(**inputs) 2025-12-04T09:49:56.5417844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5418226Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5418618Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5419000Z hidden_states = self.encoder( 2025-12-04T09:49:56.5419367Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5419745Z layer_outputs = layer_module( 2025-12-04T09:49:56.5420078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5420438Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5420832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5421228Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5421615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5421995Z self_outputs = self.self( 2025-12-04T09:49:56.5422355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5422822Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5423282Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:49:56.5423700Z x = self.depthwise(hidden_states) 2025-12-04T09:49:56.5423830Z 2025-12-04T09:49:56.5423928Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5424267Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5424571Z res = mod(**inputs) 2025-12-04T09:49:56.5424922Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5425311Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5425697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5426078Z hidden_states = self.encoder( 2025-12-04T09:49:56.5426440Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5426823Z layer_outputs = layer_module( 2025-12-04T09:49:56.5427152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5427491Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5427876Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5428267Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5428661Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5429030Z self_outputs = self.self( 2025-12-04T09:49:56.5429399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5429856Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5430315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:49:56.5430684Z x = self.pointwise(x) 2025-12-04T09:49:56.5430790Z 2025-12-04T09:49:56.5430864Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5431088Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5431419Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5431720Z res = mod(**inputs) 2025-12-04T09:49:56.5432076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5432460Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5432836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5433289Z hidden_states = self.encoder( 2025-12-04T09:49:56.5433679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5434055Z layer_outputs = layer_module( 2025-12-04T09:49:56.5434400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5434751Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5435137Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5435528Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5435913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5436296Z self_outputs = self.self( 2025-12-04T09:49:56.5436660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:49:56.5437135Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:49:56.5437304Z 2025-12-04T09:49:56.5437378Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5437577Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5437791Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5438129Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5438436Z res = mod(**inputs) 2025-12-04T09:49:56.5438799Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5439182Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5439566Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5439949Z hidden_states = self.encoder( 2025-12-04T09:49:56.5440315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5440697Z layer_outputs = layer_module( 2025-12-04T09:49:56.5441027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5441371Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5441747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5442141Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5442527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5442901Z self_outputs = self.self( 2025-12-04T09:49:56.5443259Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:49:56.5443693Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:49:56.5443864Z 2025-12-04T09:49:56.5443945Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5444162Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5444502Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5444804Z res = mod(**inputs) 2025-12-04T09:49:56.5445160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5445543Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5445929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5446308Z hidden_states = self.encoder( 2025-12-04T09:49:56.5446687Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5447070Z layer_outputs = layer_module( 2025-12-04T09:49:56.5447421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5447766Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5448140Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5448527Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5448907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5449282Z self_outputs = self.self( 2025-12-04T09:49:56.5449639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:49:56.5450086Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:49:56.5450242Z 2025-12-04T09:49:56.5450325Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5450514Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5450736Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5451069Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5451375Z res = mod(**inputs) 2025-12-04T09:49:56.5451719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5452105Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5452488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5452861Z hidden_states = self.encoder( 2025-12-04T09:49:56.5453237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5453615Z layer_outputs = layer_module( 2025-12-04T09:49:56.5453943Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5454287Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5454671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:49:56.5455063Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:56.5455443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:56.5455816Z return forward_fn(*input_tensors) 2025-12-04T09:49:56.5456219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:49:56.5456673Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:56.5457086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:49:56.5457506Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:56.5457868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:56.5458195Z return self.act(input) 2025-12-04T09:49:56.5458299Z 2025-12-04T09:49:56.5458373Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5458568Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5458763Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5458944Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5459135Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5459326Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5459522Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5459730Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5459948Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5460302Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5460605Z res = mod(**inputs) 2025-12-04T09:49:56.5460964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5461360Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5461739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5462119Z hidden_states = self.encoder( 2025-12-04T09:49:56.5462489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5462913Z layer_outputs = layer_module( 2025-12-04T09:49:56.5463237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5463581Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5463966Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5464360Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5464740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5465120Z self_outputs = self.self( 2025-12-04T09:49:56.5465491Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5465949Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5466416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:49:56.5466805Z x = self.depthwise(hidden_states) 2025-12-04T09:49:56.5466926Z 2025-12-04T09:49:56.5467034Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5467369Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5467679Z res = mod(**inputs) 2025-12-04T09:49:56.5468041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5468432Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5468810Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5469192Z hidden_states = self.encoder( 2025-12-04T09:49:56.5469574Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5469949Z layer_outputs = layer_module( 2025-12-04T09:49:56.5470281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5470627Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5471011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5471399Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5471790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5472167Z self_outputs = self.self( 2025-12-04T09:49:56.5472533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5472996Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5473550Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:49:56.5473952Z x = self.pointwise(x) 2025-12-04T09:49:56.5474058Z 2025-12-04T09:49:56.5474134Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5474359Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5474697Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5475003Z res = mod(**inputs) 2025-12-04T09:49:56.5475357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5475749Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5476140Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5476580Z hidden_states = self.encoder( 2025-12-04T09:49:56.5476944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5477322Z layer_outputs = layer_module( 2025-12-04T09:49:56.5477651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5477987Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5478373Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5478760Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5479144Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5479510Z self_outputs = self.self( 2025-12-04T09:49:56.5479876Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:49:56.5480294Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:49:56.5480455Z 2025-12-04T09:49:56.5480531Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5480731Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5480950Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5481288Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5481583Z res = mod(**inputs) 2025-12-04T09:49:56.5481937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5482322Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5482701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5483094Z hidden_states = self.encoder( 2025-12-04T09:49:56.5483471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5483850Z layer_outputs = layer_module( 2025-12-04T09:49:56.5484171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5484517Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5484902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5485292Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5485672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5486054Z self_outputs = self.self( 2025-12-04T09:49:56.5486440Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:49:56.5486872Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:49:56.5487050Z 2025-12-04T09:49:56.5487139Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5487365Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5487705Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5488001Z res = mod(**inputs) 2025-12-04T09:49:56.5488365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5488757Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5489138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5489565Z hidden_states = self.encoder( 2025-12-04T09:49:56.5489940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5490321Z layer_outputs = layer_module( 2025-12-04T09:49:56.5490650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5490995Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5491379Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5491768Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5492146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5492526Z self_outputs = self.self( 2025-12-04T09:49:56.5492901Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:49:56.5493316Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:49:56.5493481Z 2025-12-04T09:49:56.5493556Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5493754Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5493977Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5494311Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5494615Z res = mod(**inputs) 2025-12-04T09:49:56.5494967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5495350Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5495735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5496120Z hidden_states = self.encoder( 2025-12-04T09:49:56.5496495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5496866Z layer_outputs = layer_module( 2025-12-04T09:49:56.5497192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5497541Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5497923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:49:56.5498310Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:56.5498695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:56.5499074Z return forward_fn(*input_tensors) 2025-12-04T09:49:56.5499503Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:49:56.5499968Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:56.5500408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:49:56.5500828Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:56.5501182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:56.5501516Z return self.act(input) 2025-12-04T09:49:56.5501620Z 2025-12-04T09:49:56.5501703Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5501903Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5502092Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5502285Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5502496Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5502697Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5502889Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5503081Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5503291Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5503638Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5503945Z res = mod(**inputs) 2025-12-04T09:49:56.5504301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5504683Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5505072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5505451Z hidden_states = self.encoder( 2025-12-04T09:49:56.5505936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5506330Z layer_outputs = layer_module( 2025-12-04T09:49:56.5506664Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5507013Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5507388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5507787Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5508175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5508546Z self_outputs = self.self( 2025-12-04T09:49:56.5508918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5509384Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5509857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:49:56.5510240Z x = self.depthwise(hidden_states) 2025-12-04T09:49:56.5510374Z 2025-12-04T09:49:56.5510474Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5510817Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5511120Z res = mod(**inputs) 2025-12-04T09:49:56.5511466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5511856Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5512242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5512617Z hidden_states = self.encoder( 2025-12-04T09:49:56.5513071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5513471Z layer_outputs = layer_module( 2025-12-04T09:49:56.5513830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5514176Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5514565Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5514960Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5515348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5515723Z self_outputs = self.self( 2025-12-04T09:49:56.5516096Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5516608Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5517064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:49:56.5517443Z x = self.pointwise(x) 2025-12-04T09:49:56.5517553Z 2025-12-04T09:49:56.5517628Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5517848Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5518179Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5518486Z res = mod(**inputs) 2025-12-04T09:49:56.5518844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5519237Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5519625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5520011Z hidden_states = self.encoder( 2025-12-04T09:49:56.5520386Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5520754Z layer_outputs = layer_module( 2025-12-04T09:49:56.5521085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5521427Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5521812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5522193Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5522581Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5522971Z self_outputs = self.self( 2025-12-04T09:49:56.5523333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:49:56.5523760Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:49:56.5523930Z 2025-12-04T09:49:56.5524005Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5524202Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5524415Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5524754Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5525059Z res = mod(**inputs) 2025-12-04T09:49:56.5525411Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5525802Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5526205Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5526592Z hidden_states = self.encoder( 2025-12-04T09:49:56.5526971Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5527355Z layer_outputs = layer_module( 2025-12-04T09:49:56.5527684Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5528025Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5528399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5528786Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5529169Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5529578Z self_outputs = self.self( 2025-12-04T09:49:56.5529952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:49:56.5530389Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:49:56.5530559Z 2025-12-04T09:49:56.5530638Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5530853Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5531194Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5531499Z res = mod(**inputs) 2025-12-04T09:49:56.5531855Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5532240Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5532627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5533014Z hidden_states = self.encoder( 2025-12-04T09:49:56.5533386Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5533767Z layer_outputs = layer_module( 2025-12-04T09:49:56.5534098Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5534439Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5534814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5535200Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5535587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5535963Z self_outputs = self.self( 2025-12-04T09:49:56.5536335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:49:56.5536753Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:49:56.5536908Z 2025-12-04T09:49:56.5536992Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5537186Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5537410Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5537750Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5538054Z res = mod(**inputs) 2025-12-04T09:49:56.5538405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5538795Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5539185Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5539578Z hidden_states = self.encoder( 2025-12-04T09:49:56.5539980Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5540367Z layer_outputs = layer_module( 2025-12-04T09:49:56.5540698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5541035Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5541422Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:49:56.5541814Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:56.5542191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:56.5542589Z return forward_fn(*input_tensors) 2025-12-04T09:49:56.5542992Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:49:56.5543439Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:56.5543851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:49:56.5544255Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:56.5544610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:56.5544931Z return self.act(input) 2025-12-04T09:49:56.5545033Z 2025-12-04T09:49:56.5545106Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5545301Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5545491Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5545679Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5545870Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5546064Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5546246Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5546435Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5546654Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5546991Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5547287Z res = mod(**inputs) 2025-12-04T09:49:56.5547643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5548029Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5548405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5548781Z hidden_states = self.encoder( 2025-12-04T09:49:56.5549149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5549521Z layer_outputs = layer_module( 2025-12-04T09:49:56.5549841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5550184Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5550561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5550944Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5551315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5551685Z self_outputs = self.self( 2025-12-04T09:49:56.5552045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5552507Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5552989Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:49:56.5553458Z x = self.depthwise(hidden_states) 2025-12-04T09:49:56.5553582Z 2025-12-04T09:49:56.5553693Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5554029Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5554342Z res = mod(**inputs) 2025-12-04T09:49:56.5554706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5555096Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5555476Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5555900Z hidden_states = self.encoder( 2025-12-04T09:49:56.5556275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5556643Z layer_outputs = layer_module( 2025-12-04T09:49:56.5556975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5557324Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5557707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5558087Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5558470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5558851Z self_outputs = self.self( 2025-12-04T09:49:56.5559209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5559671Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5560131Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:49:56.5560509Z x = self.pointwise(x) 2025-12-04T09:49:56.5560610Z 2025-12-04T09:49:56.5560682Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5560903Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5561248Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5561550Z res = mod(**inputs) 2025-12-04T09:49:56.5561897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5562283Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5562664Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5563033Z hidden_states = self.encoder( 2025-12-04T09:49:56.5563404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5563781Z layer_outputs = layer_module( 2025-12-04T09:49:56.5564106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5564443Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5564827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5565215Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5565611Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5565994Z self_outputs = self.self( 2025-12-04T09:49:56.5566376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:49:56.5566803Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:49:56.5566964Z 2025-12-04T09:49:56.5567037Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5567236Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5567460Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5567800Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5568100Z res = mod(**inputs) 2025-12-04T09:49:56.5568458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5568889Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5569267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5569654Z hidden_states = self.encoder( 2025-12-04T09:49:56.5570029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5570405Z layer_outputs = layer_module( 2025-12-04T09:49:56.5570725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5571071Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5571457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5571839Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5572232Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5572615Z self_outputs = self.self( 2025-12-04T09:49:56.5572984Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:49:56.5573407Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:49:56.5573582Z 2025-12-04T09:49:56.5573655Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5573883Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5574223Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5574522Z res = mod(**inputs) 2025-12-04T09:49:56.5574878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5575272Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5575654Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5576034Z hidden_states = self.encoder( 2025-12-04T09:49:56.5576408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5576782Z layer_outputs = layer_module( 2025-12-04T09:49:56.5577105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5577454Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5577836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5578223Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5578601Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5578994Z self_outputs = self.self( 2025-12-04T09:49:56.5579363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:49:56.5579785Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:49:56.5579950Z 2025-12-04T09:49:56.5580024Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5580224Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5580442Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5580771Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5581076Z res = mod(**inputs) 2025-12-04T09:49:56.5581434Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5581840Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5582237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5582610Z hidden_states = self.encoder( 2025-12-04T09:49:56.5582977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5583341Z layer_outputs = layer_module( 2025-12-04T09:49:56.5583665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5584005Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5584370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:49:56.5584751Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:56.5585127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:56.5585497Z return forward_fn(*input_tensors) 2025-12-04T09:49:56.5585889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:49:56.5586333Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:56.5586743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:49:56.5587153Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:56.5587501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:56.5587821Z return self.act(input) 2025-12-04T09:49:56.5587925Z 2025-12-04T09:49:56.5588005Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5588194Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5588393Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5588586Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5588774Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5588957Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5589148Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5589343Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5589553Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5589888Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5590190Z res = mod(**inputs) 2025-12-04T09:49:56.5590534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5590913Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5591289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5591663Z hidden_states = self.encoder( 2025-12-04T09:49:56.5592082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5592490Z layer_outputs = layer_module( 2025-12-04T09:49:56.5592819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5593231Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5593620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5594010Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5594400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5594775Z self_outputs = self.self( 2025-12-04T09:49:56.5595188Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5595658Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5596126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:49:56.5596504Z x = self.depthwise(hidden_states) 2025-12-04T09:49:56.5596634Z 2025-12-04T09:49:56.5596732Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5597070Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5597375Z res = mod(**inputs) 2025-12-04T09:49:56.5597729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5598119Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5598512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5598888Z hidden_states = self.encoder( 2025-12-04T09:49:56.5599154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5599221Z layer_outputs = layer_module( 2025-12-04T09:49:56.5599446Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5599522Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5599776Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5599860Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5600114Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5600181Z self_outputs = self.self( 2025-12-04T09:49:56.5600444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5600593Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5600853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:49:56.5600918Z x = self.pointwise(x) 2025-12-04T09:49:56.5600921Z 2025-12-04T09:49:56.5600996Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5601105Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5601292Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5601360Z res = mod(**inputs) 2025-12-04T09:49:56.5601616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5601710Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5601985Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5602053Z hidden_states = self.encoder( 2025-12-04T09:49:56.5602308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5602380Z layer_outputs = layer_module( 2025-12-04T09:49:56.5602594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5602674Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5602929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5603048Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5603311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5603376Z self_outputs = self.self( 2025-12-04T09:49:56.5603639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:49:56.5603752Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:49:56.5603755Z 2025-12-04T09:49:56.5603828Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5603907Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5604003Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5604192Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5604260Z res = mod(**inputs) 2025-12-04T09:49:56.5604512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5604597Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5604851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5604918Z hidden_states = self.encoder( 2025-12-04T09:49:56.5605184Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5605248Z layer_outputs = layer_module( 2025-12-04T09:49:56.5605460Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5605541Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5605946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5606042Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5606300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5606367Z self_outputs = self.self( 2025-12-04T09:49:56.5606631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:49:56.5606750Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:49:56.5606754Z 2025-12-04T09:49:56.5606835Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5606935Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5607123Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5607192Z res = mod(**inputs) 2025-12-04T09:49:56.5607450Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5607569Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5607835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5607925Z hidden_states = self.encoder( 2025-12-04T09:49:56.5608185Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5608249Z layer_outputs = layer_module( 2025-12-04T09:49:56.5608455Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5608535Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5608782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5608862Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5609163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5609228Z self_outputs = self.self( 2025-12-04T09:49:56.5609495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:49:56.5609600Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:49:56.5609603Z 2025-12-04T09:49:56.5609674Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5609751Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5609845Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5610035Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5610094Z res = mod(**inputs) 2025-12-04T09:49:56.5610348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5610433Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5610683Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5610751Z hidden_states = self.encoder( 2025-12-04T09:49:56.5611010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5611075Z layer_outputs = layer_module( 2025-12-04T09:49:56.5611293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5611364Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5611614Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:49:56.5611701Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:56.5611956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:56.5612036Z return forward_fn(*input_tensors) 2025-12-04T09:49:56.5612322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:49:56.5612434Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:56.5612693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:49:56.5612797Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:56.5613006Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:56.5613071Z return self.act(input) 2025-12-04T09:49:56.5613074Z 2025-12-04T09:49:56.5613148Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5613228Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5613318Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5613390Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5613469Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5613555Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5613629Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5613708Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5613804Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5614000Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5614059Z res = mod(**inputs) 2025-12-04T09:49:56.5614314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5614397Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5614691Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5614759Z hidden_states = self.encoder( 2025-12-04T09:49:56.5615030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5615096Z layer_outputs = layer_module( 2025-12-04T09:49:56.5615315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5615387Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5615635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5615717Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5615968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5616043Z self_outputs = self.self( 2025-12-04T09:49:56.5616295Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5616446Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5616703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:49:56.5616772Z x = self.depthwise(hidden_states) 2025-12-04T09:49:56.5616775Z 2025-12-04T09:49:56.5616871Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5617059Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5617119Z res = mod(**inputs) 2025-12-04T09:49:56.5617377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5617454Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5617705Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5617780Z hidden_states = self.encoder( 2025-12-04T09:49:56.5618033Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5618106Z layer_outputs = layer_module( 2025-12-04T09:49:56.5618315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5618385Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5618643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5618719Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5618986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5619060Z self_outputs = self.self( 2025-12-04T09:49:56.5619333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5619492Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5619744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:49:56.5619808Z x = self.pointwise(x) 2025-12-04T09:49:56.5619812Z 2025-12-04T09:49:56.5619891Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5619988Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5620180Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5620272Z res = mod(**inputs) 2025-12-04T09:49:56.5620531Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5620612Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5620875Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5620942Z hidden_states = self.encoder( 2025-12-04T09:49:56.5621203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5621268Z layer_outputs = layer_module( 2025-12-04T09:49:56.5621486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5621559Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5621812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5621898Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5622154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5622228Z self_outputs = self.self( 2025-12-04T09:49:56.5622484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:49:56.5622597Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:49:56.5622600Z 2025-12-04T09:49:56.5622681Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5622753Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5622847Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5623041Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5623103Z res = mod(**inputs) 2025-12-04T09:49:56.5623365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5623439Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5623696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5623769Z hidden_states = self.encoder( 2025-12-04T09:49:56.5624028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5624094Z layer_outputs = layer_module( 2025-12-04T09:49:56.5624312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5624387Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5624646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5624739Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5624994Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5625095Z self_outputs = self.self( 2025-12-04T09:49:56.5625351Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:49:56.5625477Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:49:56.5625480Z 2025-12-04T09:49:56.5625552Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5625647Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5625839Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5625898Z res = mod(**inputs) 2025-12-04T09:49:56.5626190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5626274Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5626528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5626601Z hidden_states = self.encoder( 2025-12-04T09:49:56.5626853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5626917Z layer_outputs = layer_module( 2025-12-04T09:49:56.5627140Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5627210Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5627470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5627547Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5627800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5627872Z self_outputs = self.self( 2025-12-04T09:49:56.5628124Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:49:56.5628225Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:49:56.5628228Z 2025-12-04T09:49:56.5628307Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5628377Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5628483Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5628667Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5628728Z res = mod(**inputs) 2025-12-04T09:49:56.5628989Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5629063Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5629314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5629389Z hidden_states = self.encoder( 2025-12-04T09:49:56.5629642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5629718Z layer_outputs = layer_module( 2025-12-04T09:49:56.5629929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5630000Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5630257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:49:56.5630338Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:56.5630609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:56.5630697Z return forward_fn(*input_tensors) 2025-12-04T09:49:56.5630988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:49:56.5631106Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:56.5631361Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:49:56.5631466Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:56.5631673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:56.5631754Z return self.act(input) 2025-12-04T09:49:56.5631774Z 2025-12-04T09:49:56.5631855Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5631929Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5632001Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5632081Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5632153Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5632223Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5632303Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5632373Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5632475Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5632659Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5632719Z res = mod(**inputs) 2025-12-04T09:49:56.5632981Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5633123Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5633389Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5633467Z hidden_states = self.encoder( 2025-12-04T09:49:56.5633726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5633800Z layer_outputs = layer_module( 2025-12-04T09:49:56.5634011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5634083Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5634344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5634421Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5634675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5634753Z self_outputs = self.self( 2025-12-04T09:49:56.5635007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5635163Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5635418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:49:56.5635489Z x = self.depthwise(hidden_states) 2025-12-04T09:49:56.5635493Z 2025-12-04T09:49:56.5635598Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5635779Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5635846Z res = mod(**inputs) 2025-12-04T09:49:56.5636103Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5636196Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5636471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5636538Z hidden_states = self.encoder( 2025-12-04T09:49:56.5636799Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5636863Z layer_outputs = layer_module( 2025-12-04T09:49:56.5637073Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5637150Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5637402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5637560Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5637826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5637893Z self_outputs = self.self( 2025-12-04T09:49:56.5638157Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5638304Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5638561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:49:56.5638637Z x = self.pointwise(x) 2025-12-04T09:49:56.5638640Z 2025-12-04T09:49:56.5638717Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5638823Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5639010Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5639078Z res = mod(**inputs) 2025-12-04T09:49:56.5639340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5639420Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5639679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5639755Z hidden_states = self.encoder( 2025-12-04T09:49:56.5640012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5640089Z layer_outputs = layer_module( 2025-12-04T09:49:56.5640304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5640381Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5640649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5640729Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5640987Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5641065Z self_outputs = self.self( 2025-12-04T09:49:56.5641324Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:49:56.5641444Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:49:56.5641448Z 2025-12-04T09:49:56.5641524Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5641598Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5641704Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5641894Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5641981Z res = mod(**inputs) 2025-12-04T09:49:56.5642243Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5642332Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5642605Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5642671Z hidden_states = self.encoder( 2025-12-04T09:49:56.5642927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5643001Z layer_outputs = layer_module( 2025-12-04T09:49:56.5643211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5643288Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5643575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5643650Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5643910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5643974Z self_outputs = self.self( 2025-12-04T09:49:56.5644225Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:49:56.5644349Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:49:56.5644352Z 2025-12-04T09:49:56.5644423Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5644524Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5644706Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5644769Z res = mod(**inputs) 2025-12-04T09:49:56.5645030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5645103Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5645362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5645428Z hidden_states = self.encoder( 2025-12-04T09:49:56.5645679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5645751Z layer_outputs = layer_module( 2025-12-04T09:49:56.5645958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5646029Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5646291Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5646369Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5646629Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5646694Z self_outputs = self.self( 2025-12-04T09:49:56.5646947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:49:56.5647058Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:49:56.5647062Z 2025-12-04T09:49:56.5647134Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5647213Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5647308Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5647490Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5647559Z res = mod(**inputs) 2025-12-04T09:49:56.5647826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5647902Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5648177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5648246Z hidden_states = self.encoder( 2025-12-04T09:49:56.5648509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5648576Z layer_outputs = layer_module( 2025-12-04T09:49:56.5648787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5648867Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5649139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:49:56.5649237Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:56.5649490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:56.5649562Z return forward_fn(*input_tensors) 2025-12-04T09:49:56.5649847Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:49:56.5649958Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:56.5650212Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:49:56.5650324Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:56.5650529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:56.5650603Z return self.act(input) 2025-12-04T09:49:56.5650607Z 2025-12-04T09:49:56.5650679Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5650750Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5650828Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5650899Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5650967Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5651045Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5651112Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5651188Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5651281Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5651463Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5651529Z res = mod(**inputs) 2025-12-04T09:49:56.5651780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5651857Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5652115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5652182Z hidden_states = self.encoder( 2025-12-04T09:49:56.5652444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5652509Z layer_outputs = layer_module( 2025-12-04T09:49:56.5652721Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5652801Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5653050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5653126Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5653399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5653466Z self_outputs = self.self( 2025-12-04T09:49:56.5654549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5654706Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5654958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:49:56.5655036Z x = self.depthwise(hidden_states) 2025-12-04T09:49:56.5655039Z 2025-12-04T09:49:56.5655136Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5655329Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5655423Z res = mod(**inputs) 2025-12-04T09:49:56.5655675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5655758Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5656009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5656076Z hidden_states = self.encoder( 2025-12-04T09:49:56.5656331Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5656397Z layer_outputs = layer_module( 2025-12-04T09:49:56.5656612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5656683Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5656933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5657019Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5657268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5657342Z self_outputs = self.self( 2025-12-04T09:49:56.5657590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:49:56.5657733Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:49:56.5657989Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:49:56.5658052Z x = self.pointwise(x) 2025-12-04T09:49:56.5658055Z 2025-12-04T09:49:56.5658126Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5658231Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5658415Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5658481Z res = mod(**inputs) 2025-12-04T09:49:56.5658731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5658806Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5659060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5659126Z hidden_states = self.encoder( 2025-12-04T09:49:56.5659382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5659446Z layer_outputs = layer_module( 2025-12-04T09:49:56.5659653Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5659733Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5659999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5660101Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5660369Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5660433Z self_outputs = self.self( 2025-12-04T09:49:56.5660692Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:49:56.5660804Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:49:56.5660808Z 2025-12-04T09:49:56.5660879Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5660960Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5661072Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5661274Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5661342Z res = mod(**inputs) 2025-12-04T09:49:56.5661595Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5661678Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5661930Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5661996Z hidden_states = self.encoder( 2025-12-04T09:49:56.5662257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5662323Z layer_outputs = layer_module( 2025-12-04T09:49:56.5662540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5662613Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5662867Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5662950Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5663201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5663264Z self_outputs = self.self( 2025-12-04T09:49:56.5663523Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:49:56.5663638Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:49:56.5663641Z 2025-12-04T09:49:56.5663720Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5663815Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5663999Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5664069Z res = mod(**inputs) 2025-12-04T09:49:56.5664324Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5664406Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5664659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5664725Z hidden_states = self.encoder( 2025-12-04T09:49:56.5664986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5665050Z layer_outputs = layer_module( 2025-12-04T09:49:56.5665259Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5665338Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5665602Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:49:56.5665683Z self_attention_outputs = self.attention( 2025-12-04T09:49:56.5665944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:49:56.5666010Z self_outputs = self.self( 2025-12-04T09:49:56.5666262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:49:56.5666363Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:49:56.5666367Z 2025-12-04T09:49:56.5666444Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5666515Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5666608Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:56.5666832Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:56.5666896Z res = mod(**inputs) 2025-12-04T09:49:56.5667154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:49:56.5667239Z generator_hidden_states = self.convbert( 2025-12-04T09:49:56.5667495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:49:56.5667569Z hidden_states = self.encoder( 2025-12-04T09:49:56.5667824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:49:56.5667889Z layer_outputs = layer_module( 2025-12-04T09:49:56.5668107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:56.5668181Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:56.5668441Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:49:56.5668527Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:56.5668777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:56.5668855Z return forward_fn(*input_tensors) 2025-12-04T09:49:56.5669141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:49:56.5669253Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:56.5669513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:49:56.5669614Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:56.5669830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:56.5669895Z return self.act(input) 2025-12-04T09:49:56.5669901Z 2025-12-04T09:49:56.5669974Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5670055Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5670128Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5670198Z cudagraph partition due to non gpu ops 2025-12-04T09:49:56.5670275Z cudagraph partition due to non gpu ops 2025-12-04T09:50:07.0038787Z Compilation time (from dynamo_timed): 24.493078024 2025-12-04T09:50:07.0091591Z pass 2025-12-04T09:50:07.0095495Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:50:07.0099202Z TIMING: _recursive_pre_grad_passes:0.30659 _recursive_joint_graph_passes:0.55905 _recursive_post_grad_passes:0.13101 async_compile.wait:0.71131 code_gen:10.84103 inductor_compile:13.1655 backend_compile:20.7738 gc:0.00014 entire_frame_compile:24.49308 total_wall_time:24.49308 2025-12-04T09:50:07.0100667Z STATS: call_* op count: 634 | FakeTensorMode.__torch_dispatch__:38075 | FakeTensor.__torch_dispatch__:5654 | ProxyTorchDispatchMode.__torch_dispatch__:5038 2025-12-04T09:50:07.0101224Z Dynamo produced 1 graphs covering 634 ops with 0 graph breaks (0 unique) 2025-12-04T09:50:08.9050538Z accuracy pass_rate=92.59% 2025-12-04T09:50:08.9054421Z calls_captured gmean=0.00x mean=579.519x 2025-12-04T09:50:08.9056254Z unique_graphs gmean=0.00x mean=1.111x 2025-12-04T09:50:08.9056611Z graph_breaks gmean=0.00x mean=0.222x 2025-12-04T09:50:08.9061880Z unique_graph_breaks gmean=0.00x mean=0.074x 2025-12-04T09:50:08.9066043Z autograd_captures gmean=0.00x mean=0.000x 2025-12-04T09:50:08.9068044Z autograd_compiles gmean=0.00x mean=0.000x 2025-12-04T09:50:08.9068374Z cudagraph_skips gmean=0.00x mean=1.111x 2025-12-04T09:50:08.9068587Z compilation_latency mean=21.238 seconds 2025-12-04T09:50:09.6740610Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/inference_huggingface.csv --expected benchmarks/dynamo/ci_expected_accuracy/cpu_inductor_freezing_huggingface_inference.csv 2025-12-04T09:50:09.9289754Z AlbertForMaskedLM PASS 2025-12-04T09:50:09.9293414Z AllenaiLongformerBase PASS 2025-12-04T09:50:09.9297770Z BartForCausalLM PASS 2025-12-04T09:50:09.9299941Z BertForMaskedLM PASS 2025-12-04T09:50:09.9300295Z BlenderbotForCausalLM XFAIL 2025-12-04T09:50:09.9305043Z DebertaV2ForMaskedLM XFAIL 2025-12-04T09:50:09.9307017Z DistilBertForMaskedLM PASS 2025-12-04T09:50:09.9312010Z DistillGPT2 PASS 2025-12-04T09:50:09.9313774Z ElectraForCausalLM PASS 2025-12-04T09:50:09.9314048Z GPT2ForSequenceClassification PASS 2025-12-04T09:50:09.9314430Z GoogleFnet PASS 2025-12-04T09:50:09.9320197Z LayoutLMForMaskedLM PASS 2025-12-04T09:50:09.9322212Z M2M100ForConditionalGeneration PASS 2025-12-04T09:50:09.9322560Z MBartForCausalLM PASS 2025-12-04T09:50:09.9330896Z MT5ForConditionalGeneration PASS 2025-12-04T09:50:09.9331283Z MegatronBertForCausalLM PASS 2025-12-04T09:50:09.9336662Z MobileBertForMaskedLM PASS 2025-12-04T09:50:09.9339252Z OPTForCausalLM PASS 2025-12-04T09:50:09.9339607Z PLBartForCausalLM PASS 2025-12-04T09:50:09.9344870Z PegasusForCausalLM PASS 2025-12-04T09:50:09.9349545Z RobertaForCausalLM PASS 2025-12-04T09:50:09.9349966Z T5ForConditionalGeneration PASS 2025-12-04T09:50:09.9350291Z T5Small PASS 2025-12-04T09:50:09.9350569Z TrOCRForCausalLM PASS 2025-12-04T09:50:09.9358501Z XGLMForCausalLM PASS 2025-12-04T09:50:09.9362992Z XLNetLMHeadModel PASS 2025-12-04T09:50:09.9365015Z YituTechConvBert PASS 2025-12-04T09:50:09.9859650Z + python benchmarks/dynamo/check_graph_breaks.py --actual /var/lib/jenkins/workspace/test/test-reports/inference_huggingface.csv --expected benchmarks/dynamo/ci_expected_accuracy/cpu_inductor_freezing_huggingface_inference.csv 2025-12-04T09:50:10.2617186Z AlbertForMaskedLM PASS 2025-12-04T09:50:10.2621166Z AllenaiLongformerBase PASS 2025-12-04T09:50:10.2625671Z BartForCausalLM PASS 2025-12-04T09:50:10.2629747Z BertForMaskedLM PASS 2025-12-04T09:50:10.2634595Z BlenderbotForCausalLM PASS 2025-12-04T09:50:10.2636409Z DebertaV2ForMaskedLM PASS 2025-12-04T09:50:10.2636665Z DistilBertForMaskedLM PASS 2025-12-04T09:50:10.2636970Z DistillGPT2 PASS 2025-12-04T09:50:10.2642128Z ElectraForCausalLM PASS 2025-12-04T09:50:10.2642581Z GPT2ForSequenceClassification PASS 2025-12-04T09:50:10.2642888Z GoogleFnet PASS 2025-12-04T09:50:10.2643467Z LayoutLMForMaskedLM PASS 2025-12-04T09:50:10.2653028Z M2M100ForConditionalGeneration PASS 2025-12-04T09:50:10.2655124Z MBartForCausalLM PASS 2025-12-04T09:50:10.2655688Z MT5ForConditionalGeneration PASS 2025-12-04T09:50:10.2662843Z MegatronBertForCausalLM PASS 2025-12-04T09:50:10.2664806Z MobileBertForMaskedLM PASS 2025-12-04T09:50:10.2669816Z OPTForCausalLM PASS 2025-12-04T09:50:10.2674436Z PLBartForCausalLM PASS 2025-12-04T09:50:10.2676369Z PegasusForCausalLM PASS 2025-12-04T09:50:10.2679803Z RobertaForCausalLM PASS 2025-12-04T09:50:10.2680172Z T5ForConditionalGeneration PASS 2025-12-04T09:50:10.2685054Z T5Small PASS 2025-12-04T09:50:10.2689564Z TrOCRForCausalLM PASS 2025-12-04T09:50:10.2691604Z XGLMForCausalLM PASS_BUT_FLAKY 2025-12-04T09:50:10.2691989Z XLNetLMHeadModel PASS 2025-12-04T09:50:10.2697317Z YituTechConvBert PASS 2025-12-04T09:50:10.3178755Z + sccache_epilogue 2025-12-04T09:50:10.3180345Z + echo '::group::Sccache Compilation Log' 2025-12-04T09:50:10.3180972Z ##[group]Sccache Compilation Log 2025-12-04T09:50:10.3184858Z + echo '=================== sccache compilation log ===================' 2025-12-04T09:50:10.3187221Z =================== sccache compilation log =================== 2025-12-04T09:50:10.3187682Z + python /var/lib/jenkins/workspace/.ci/pytorch/print_sccache_log.py /var/lib/jenkins/sccache_error.log 2025-12-04T09:50:10.3384034Z + echo '=========== If your build fails, please take a look at the log above for possible reasons ===========' 2025-12-04T09:50:10.3385837Z =========== If your build fails, please take a look at the log above for possible reasons =========== 2025-12-04T09:50:10.3386333Z + sccache --show-stats 2025-12-04T09:50:10.3416629Z Compile requests 281 2025-12-04T09:50:10.3418435Z Compile requests executed 0 2025-12-04T09:50:10.3418682Z Cache hits 0 2025-12-04T09:50:10.3418877Z Cache misses 0 2025-12-04T09:50:10.3419075Z Cache hits rate - 2025-12-04T09:50:10.3419257Z Cache timeouts 0 2025-12-04T09:50:10.3419443Z Cache read errors 0 2025-12-04T09:50:10.3419629Z Forced recaches 0 2025-12-04T09:50:10.3419809Z Cache write errors 0 2025-12-04T09:50:10.3419991Z Cache errors 0 2025-12-04T09:50:10.3420173Z Compilations 0 2025-12-04T09:50:10.3420357Z Compilation failures 0 2025-12-04T09:50:10.3420561Z Non-cacheable compilations 0 2025-12-04T09:50:10.3420755Z Non-cacheable calls 25 2025-12-04T09:50:10.3420944Z Non-compilation calls 256 2025-12-04T09:50:10.3421146Z Unsupported compiler calls 0 2025-12-04T09:50:10.3421349Z Average cache write 0.000 s 2025-12-04T09:50:10.3421556Z Average compiler 0.000 s 2025-12-04T09:50:10.3422172Z Average cache read hit 0.000 s 2025-12-04T09:50:10.3422387Z Failed distributed compilations 0 2025-12-04T09:50:10.3422519Z 2025-12-04T09:50:10.3422596Z Non-cacheable reasons: 2025-12-04T09:50:10.3422773Z -E 25 2025-12-04T09:50:10.3422899Z 2025-12-04T09:50:10.3423050Z Cache location s3, name: ossci-compiler-cache-circleci-v2, prefix: / 2025-12-04T09:50:10.3423329Z Version (client) 0.10.0 2025-12-04T09:50:10.3423525Z + sccache --stop-server 2025-12-04T09:50:10.3437331Z Stopping sccache server... 2025-12-04T09:50:10.3446826Z Compile requests 281 2025-12-04T09:50:10.3450290Z Compile requests executed 0 2025-12-04T09:50:10.3452317Z Cache hits 0 2025-12-04T09:50:10.3457678Z Cache misses 0 2025-12-04T09:50:10.3461941Z Cache hits rate - 2025-12-04T09:50:10.3462196Z Cache timeouts 0 2025-12-04T09:50:10.3462407Z Cache read errors 0 2025-12-04T09:50:10.3462645Z Forced recaches 0 2025-12-04T09:50:10.3462834Z Cache write errors 0 2025-12-04T09:50:10.3463010Z Cache errors 0 2025-12-04T09:50:10.3463196Z Compilations 0 2025-12-04T09:50:10.3463382Z Compilation failures 0 2025-12-04T09:50:10.3463569Z Non-cacheable compilations 0 2025-12-04T09:50:10.3463764Z Non-cacheable calls 25 2025-12-04T09:50:10.3463958Z Non-compilation calls 256 2025-12-04T09:50:10.3464144Z Unsupported compiler calls 0 2025-12-04T09:50:10.3464352Z Average cache write 0.000 s 2025-12-04T09:50:10.3464654Z Average compiler 0.000 s 2025-12-04T09:50:10.3464851Z Average cache read hit 0.000 s 2025-12-04T09:50:10.3465058Z Failed distributed compilations 0 2025-12-04T09:50:10.3465196Z 2025-12-04T09:50:10.3465268Z Non-cacheable reasons: 2025-12-04T09:50:10.3465447Z -E 25 2025-12-04T09:50:10.3465580Z 2025-12-04T09:50:10.3465734Z Cache location s3, name: ossci-compiler-cache-circleci-v2, prefix: / 2025-12-04T09:50:10.3466008Z Version (client) 0.10.0 2025-12-04T09:50:10.3466235Z + echo ::endgroup:: 2025-12-04T09:50:10.3466753Z ##[endgroup] 2025-12-04T09:50:10.3466914Z + cleanup_workspace 2025-12-04T09:50:10.3467218Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2025-12-04T09:50:10.3467675Z sudo may print the following warning message that can be ignored. The chown command will still run. 2025-12-04T09:50:10.3468051Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2025-12-04T09:50:10.3468336Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2025-12-04T09:50:10.3468684Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2025-12-04T09:50:10.3469044Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2025-12-04T09:50:10.3469334Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2025-12-04T09:50:10.7718744Z ##[group]Run pytorch/test-infra/.github/actions/upload-benchmark-results@main 2025-12-04T09:50:10.7719033Z with: 2025-12-04T09:50:10.7719205Z benchmark-results-dir: test/test-reports 2025-12-04T09:50:10.7719398Z dry-run: false 2025-12-04T09:50:10.7719555Z schema-version: v3 2025-12-04T09:50:10.7719887Z github-token: *** 2025-12-04T09:50:10.7720036Z env: 2025-12-04T09:50:10.7720173Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:10.7720399Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:10.7720785Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:10.7721213Z ##[endgroup] 2025-12-04T09:50:10.7732767Z ##[group]Run set -eux 2025-12-04T09:50:10.7732959Z set -eux 2025-12-04T09:50:10.7733110Z  2025-12-04T09:50:10.7733248Z if [[ -n "" ]]; then 2025-12-04T09:50:10.7733424Z  source "" 2025-12-04T09:50:10.7733578Z fi 2025-12-04T09:50:10.7733829Z python3 -mpip install boto3==1.35.33 psutil==7.0.0 pynvml==12.0.0 2025-12-04T09:50:10.7734070Z  2025-12-04T09:50:10.7734210Z DEVICE_NAME="" 2025-12-04T09:50:10.7734387Z DEVICE_TYPE="" 2025-12-04T09:50:10.7734544Z  2025-12-04T09:50:10.7734688Z if command -v nvidia-smi; then 2025-12-04T09:50:10.7734957Z  # NB: I'm using PyTorch here to get the device name, however, it needs to 2025-12-04T09:50:10.7735284Z  # install the correct version of PyTorch manually for now. Any PyTorch 2025-12-04T09:50:10.7735581Z  # version is fine, I just use 2.7.1 to satify PYPIDEP linter 2025-12-04T09:50:10.7735841Z  python3 -mpip install torch==2.7.1 2025-12-04T09:50:10.7736047Z elif command -v rocminfo; then 2025-12-04T09:50:10.7736302Z  # NB: Installing torch on ROCm runner with pip here causes CI to fail 2025-12-04T09:50:10.7736615Z  # with a memoryview is too large error only on MI300 runners. Is pip 2025-12-04T09:50:10.7736930Z  # version on ROCm runner there too old? As a workaround, let's use the 2025-12-04T09:50:10.7737211Z  # GPU device name coming from rocminfo instead 2025-12-04T09:50:10.7737418Z  DEVICE_NAME=rocm 2025-12-04T09:50:10.7737701Z  DEVICE_TYPE=$(rocminfo | grep "Marketing Name" | tail -n1 | awk -F':' '{print $2}' | xargs) 2025-12-04T09:50:10.7737982Z fi 2025-12-04T09:50:10.7738119Z  2025-12-04T09:50:10.7738281Z echo "DEVICE_NAME=$DEVICE_NAME" >> $GITHUB_ENV 2025-12-04T09:50:10.7738524Z echo "DEVICE_TYPE=$DEVICE_TYPE" >> $GITHUB_ENV 2025-12-04T09:50:10.7745656Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:50:10.7745886Z env: 2025-12-04T09:50:10.7746036Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:10.7746215Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:10.7746485Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:10.7746786Z ##[endgroup] 2025-12-04T09:50:10.7778734Z + [[ -n '' ]] 2025-12-04T09:50:10.7780763Z + python3 -mpip install boto3==1.35.33 psutil==7.0.0 pynvml==12.0.0 2025-12-04T09:50:10.9432867Z Defaulting to user installation because normal site-packages is not writeable 2025-12-04T09:50:11.6826979Z Collecting boto3==1.35.33 2025-12-04T09:50:11.6976613Z Downloading boto3-1.35.33-py3-none-any.whl (139 kB) 2025-12-04T09:50:11.9194966Z Collecting psutil==7.0.0 2025-12-04T09:50:11.9233579Z Downloading psutil-7.0.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (277 kB) 2025-12-04T09:50:11.9496394Z Collecting pynvml==12.0.0 2025-12-04T09:50:11.9530329Z Downloading pynvml-12.0.0-py3-none-any.whl (26 kB) 2025-12-04T09:50:11.9891246Z Collecting s3transfer<0.11.0,>=0.10.0 2025-12-04T09:50:11.9926939Z Downloading s3transfer-0.10.4-py3-none-any.whl (83 kB) 2025-12-04T09:50:12.7545457Z Collecting botocore<1.36.0,>=1.35.33 2025-12-04T09:50:12.7588829Z Downloading botocore-1.35.99-py3-none-any.whl (13.3 MB) 2025-12-04T09:50:12.8391010Z Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /usr/lib/python3.9/site-packages (from boto3==1.35.33) (0.10.0) 2025-12-04T09:50:12.8699823Z Collecting nvidia-ml-py<13.0.0a0,>=12.0.0 2025-12-04T09:50:12.8737396Z Downloading nvidia_ml_py-12.575.51-py3-none-any.whl (47 kB) 2025-12-04T09:50:12.8806630Z Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /usr/lib/python3.9/site-packages (from botocore<1.36.0,>=1.35.33->boto3==1.35.33) (2.8.1) 2025-12-04T09:50:12.8813624Z Requirement already satisfied: urllib3<1.27,>=1.25.4 in /usr/lib/python3.9/site-packages (from botocore<1.36.0,>=1.35.33->boto3==1.35.33) (1.25.10) 2025-12-04T09:50:12.9896198Z Requirement already satisfied: six>=1.5 in /usr/lib/python3.9/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.36.0,>=1.35.33->boto3==1.35.33) (1.15.0) 2025-12-04T09:50:13.0871436Z Installing collected packages: botocore, s3transfer, nvidia-ml-py, pynvml, psutil, boto3 2025-12-04T09:50:13.4114827Z Attempting uninstall: nvidia-ml-py 2025-12-04T09:50:13.4118882Z Found existing installation: nvidia-ml-py 11.525.84 2025-12-04T09:50:13.4124919Z Uninstalling nvidia-ml-py-11.525.84: 2025-12-04T09:50:13.4246293Z Successfully uninstalled nvidia-ml-py-11.525.84 2025-12-04T09:50:13.4697921Z Attempting uninstall: psutil 2025-12-04T09:50:13.4702321Z Found existing installation: psutil 5.9.8 2025-12-04T09:50:13.4743817Z Uninstalling psutil-5.9.8: 2025-12-04T09:50:13.4748049Z Successfully uninstalled psutil-5.9.8 2025-12-04T09:50:13.6002815Z Successfully installed boto3-1.35.33 botocore-1.35.99 nvidia-ml-py-12.575.51 psutil-7.0.0 pynvml-12.0.0 s3transfer-0.10.4 2025-12-04T09:50:13.7120860Z + DEVICE_NAME= 2025-12-04T09:50:13.7124744Z + DEVICE_TYPE= 2025-12-04T09:50:13.7128295Z + command -v nvidia-smi 2025-12-04T09:50:13.7132419Z + command -v rocminfo 2025-12-04T09:50:13.7136440Z + echo DEVICE_NAME= 2025-12-04T09:50:13.7138451Z + echo DEVICE_TYPE= 2025-12-04T09:50:13.7152694Z ##[group]Run set -eux 2025-12-04T09:50:13.7152880Z set -eux 2025-12-04T09:50:13.7153162Z  2025-12-04T09:50:13.7170207Z if [[ -z "${GITHUB_TOKEN}" ]]; then 2025-12-04T09:50:13.7170546Z  echo "Missing github-token input" 2025-12-04T09:50:13.7170748Z  exit 1 2025-12-04T09:50:13.7170906Z fi 2025-12-04T09:50:13.7175508Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:50:13.7175738Z env: 2025-12-04T09:50:13.7175896Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:13.7176072Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:13.7176534Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:13.7176833Z DEVICE_NAME: 2025-12-04T09:50:13.7176989Z DEVICE_TYPE: 2025-12-04T09:50:13.7177363Z GITHUB_TOKEN: *** 2025-12-04T09:50:13.7177517Z ##[endgroup] 2025-12-04T09:50:13.7203444Z + [[ -z *** ]] 2025-12-04T09:50:13.7236477Z ##[group]Run pytorch/test-infra/.github/actions/get-workflow-job-id@main 2025-12-04T09:50:13.7236740Z with: 2025-12-04T09:50:13.7236995Z github-token: *** 2025-12-04T09:50:13.7237149Z env: 2025-12-04T09:50:13.7237284Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:13.7237455Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:13.7237730Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:13.7238015Z DEVICE_NAME: 2025-12-04T09:50:13.7238166Z DEVICE_TYPE: 2025-12-04T09:50:13.7238315Z ##[endgroup] 2025-12-04T09:50:13.7247541Z ##[group]Run set -eux 2025-12-04T09:50:13.7247717Z set -eux 2025-12-04T09:50:13.7247870Z  2025-12-04T09:50:13.7248154Z python3 "${GITHUB_ACTION_PATH}/../../scripts/get_workflow_job_id.py" "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2025-12-04T09:50:13.7251879Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:50:13.7252104Z env: 2025-12-04T09:50:13.7252255Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:13.7252420Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:13.7252807Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:13.7253101Z DEVICE_NAME: 2025-12-04T09:50:13.7253254Z DEVICE_TYPE: 2025-12-04T09:50:13.7253509Z GITHUB_TOKEN: *** 2025-12-04T09:50:13.7253663Z ##[endgroup] 2025-12-04T09:50:13.7272082Z + python3 /home/ec2-user/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/get-workflow-job-id/../../scripts/get_workflow_job_id.py 19923066595 i-04ab61807b125ed1d 2025-12-04T09:50:15.7581409Z setting job-id=57118563323 2025-12-04T09:50:15.7582090Z setting job-name=periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_huggingface, 1, 1, linux.8xlarge.amx) 2025-12-04T09:50:15.7684065Z ##[group]Run set -eux 2025-12-04T09:50:15.7684260Z set -eux 2025-12-04T09:50:15.7684398Z  2025-12-04T09:50:15.7684544Z if [[ -n "" ]]; then 2025-12-04T09:50:15.7684723Z  source "" 2025-12-04T09:50:15.7684866Z fi 2025-12-04T09:50:15.7685019Z  2025-12-04T09:50:15.7685260Z python3 "${GITHUB_ACTION_PATH}/../../scripts/benchmarks/gather_metadata.py" \ 2025-12-04T09:50:15.7685559Z  --schema-version "${SCHEMA_VERSION}" \ 2025-12-04T09:50:15.7685764Z  --repo "${REPO}" \ 2025-12-04T09:50:15.7685952Z  --head-branch "${HEAD_BRANCH}" \ 2025-12-04T09:50:15.7686152Z  --head-sha "${HEAD_SHA}" \ 2025-12-04T09:50:15.7686352Z  --workflow-id "${WORKFLOW_RUN_ID}" \ 2025-12-04T09:50:15.7686566Z  --run-attempt "${RUN_ATTEMPT}" \ 2025-12-04T09:50:15.7686762Z  --job-id "${JOB_ID}" \ 2025-12-04T09:50:15.7686959Z  --job-name "${JOB_NAME}" 2025-12-04T09:50:15.7692003Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:50:15.7692234Z env: 2025-12-04T09:50:15.7692388Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:15.7692558Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:15.7692848Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:15.7693141Z DEVICE_NAME: 2025-12-04T09:50:15.7693286Z DEVICE_TYPE: 2025-12-04T09:50:15.7693438Z SCHEMA_VERSION: v3 2025-12-04T09:50:15.7693604Z REPO: pytorch/pytorch 2025-12-04T09:50:15.7693766Z HEAD_BRANCH: refs/heads/main 2025-12-04T09:50:15.7693973Z HEAD_SHA: ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:50:15.7694189Z WORKFLOW_RUN_ID: 19923066595 2025-12-04T09:50:15.7694355Z RUN_ATTEMPT: 1 2025-12-04T09:50:15.7694498Z JOB_ID: 57118563323 2025-12-04T09:50:15.7694837Z JOB_NAME: periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_huggingface, 1, 1, linux.8xlarge.amx) 2025-12-04T09:50:15.7695298Z ##[endgroup] 2025-12-04T09:50:15.7720139Z + [[ -n '' ]] 2025-12-04T09:50:15.7725430Z + python3 /home/ec2-user/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/upload-benchmark-results/../../scripts/benchmarks/gather_metadata.py --schema-version v3 --repo pytorch/pytorch --head-branch refs/heads/main --head-sha ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 --workflow-id 19923066595 --run-attempt 1 --job-id 57118563323 --job-name 'periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_huggingface, 1, 1, linux.8xlarge.amx)' 2025-12-04T09:50:15.7962645Z ##[group]Run set -eux 2025-12-04T09:50:15.7962827Z set -eux 2025-12-04T09:50:15.7962976Z  2025-12-04T09:50:15.7963118Z if [[ -n "" ]]; then 2025-12-04T09:50:15.7963297Z  source "" 2025-12-04T09:50:15.7963451Z fi 2025-12-04T09:50:15.7963585Z  2025-12-04T09:50:15.7963828Z python3 "${GITHUB_ACTION_PATH}/../../scripts/benchmarks/gather_runners_info.py" 2025-12-04T09:50:15.7967923Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:50:15.7968146Z env: 2025-12-04T09:50:15.7968288Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:15.7968454Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:15.7968727Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:15.7969123Z DEVICE_NAME: 2025-12-04T09:50:15.7969278Z DEVICE_TYPE: 2025-12-04T09:50:15.7969425Z ##[endgroup] 2025-12-04T09:50:15.7988778Z + [[ -n '' ]] 2025-12-04T09:50:15.7989447Z + python3 /home/ec2-user/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/upload-benchmark-results/../../scripts/benchmarks/gather_runners_info.py 2025-12-04T09:50:15.8294855Z INFO:root:Fail to import torch to get the device name 2025-12-04T09:50:15.8386111Z ##[group]Run set -eux 2025-12-04T09:50:15.8386288Z set -eux 2025-12-04T09:50:15.8386434Z  2025-12-04T09:50:15.8386614Z # TODO (huydhn): Implement this part 2025-12-04T09:50:15.8386843Z echo "dependencies={}" >> "${GITHUB_OUTPUT}" 2025-12-04T09:50:15.8390797Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:50:15.8391031Z env: 2025-12-04T09:50:15.8391174Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:15.8391344Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:15.8391629Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:15.8391927Z DEVICE_NAME: 2025-12-04T09:50:15.8392069Z DEVICE_TYPE: 2025-12-04T09:50:15.8392215Z ##[endgroup] 2025-12-04T09:50:15.8412473Z + echo 'dependencies={}' 2025-12-04T09:50:15.8429659Z ##[group]Run set -eux 2025-12-04T09:50:15.8429847Z set -eux 2025-12-04T09:50:15.8429997Z  2025-12-04T09:50:15.8430147Z if [[ -n "" ]]; then 2025-12-04T09:50:15.8430312Z  source "" 2025-12-04T09:50:15.8430466Z fi 2025-12-04T09:50:15.8430606Z  2025-12-04T09:50:15.8430783Z if [[ ! -d "${BENCHMARK_RESULTS_DIR}" ]]; then 2025-12-04T09:50:15.8431053Z  echo "${BENCHMARK_RESULTS_DIR} does not exist, skipping" 2025-12-04T09:50:15.8431343Z  # We don't want the job to fail if the directory doesn't exist 2025-12-04T09:50:15.8431576Z  exit 0 2025-12-04T09:50:15.8431710Z fi 2025-12-04T09:50:15.8431852Z  2025-12-04T09:50:15.8432005Z if [[ "${DRY_RUN}" == "true" ]]; then 2025-12-04T09:50:15.8432280Z  python3 "${GITHUB_ACTION_PATH}/../../scripts/upload_benchmark_results.py" \ 2025-12-04T09:50:15.8432605Z  --benchmark-results-dir "${BENCHMARK_RESULTS_DIR}" \ 2025-12-04T09:50:15.8432857Z  --metadata "${BENCHMARK_METADATA}" \ 2025-12-04T09:50:15.8433199Z  --runners "${RUNNER_INFO}" \ 2025-12-04T09:50:15.8433412Z  --dependencies "${DEPENDENCIES}" \ 2025-12-04T09:50:15.8433613Z  --dry-run 2025-12-04T09:50:15.8433872Z else 2025-12-04T09:50:15.8434094Z  python3 "${GITHUB_ACTION_PATH}/../../scripts/upload_benchmark_results.py" \ 2025-12-04T09:50:15.8434411Z  --benchmark-results-dir "${BENCHMARK_RESULTS_DIR}" \ 2025-12-04T09:50:15.8434661Z  --metadata "${BENCHMARK_METADATA}" \ 2025-12-04T09:50:15.8434875Z  --runners "${RUNNER_INFO}" \ 2025-12-04T09:50:15.8435080Z  --dependencies "${DEPENDENCIES}" 2025-12-04T09:50:15.8435273Z fi 2025-12-04T09:50:15.8438759Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:50:15.8438978Z env: 2025-12-04T09:50:15.8439124Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:15.8439295Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:15.8439561Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:15.8439855Z DEVICE_NAME: 2025-12-04T09:50:15.8440006Z DEVICE_TYPE: 2025-12-04T09:50:15.8440166Z BENCHMARK_RESULTS_DIR: test/test-reports 2025-12-04T09:50:15.8440365Z DRY_RUN: false 2025-12-04T09:50:15.8441260Z BENCHMARK_METADATA: {"timestamp": 1764841815, "schema_version": "v3", "name": "periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_huggingface, 1, 1, linux.8xlarge.amx)", "repo": "pytorch/pytorch", "head_branch": "refs/heads/main", "head_sha": "ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32", "workflow_id": 19923066595, "run_attempt": 1, "job_id": 57118563323} 2025-12-04T09:50:15.8442263Z RUNNER_INFO: [{"cpu_info": "x86_64", "cpu_count": 32, "avail_mem_in_gb": 123, "extra_info": {"hostname": "ip-10-0-71-55.ec2.internal"}, "name": "", "type": ""}] 2025-12-04T09:50:15.8442622Z DEPENDENCIES: {} 2025-12-04T09:50:15.8442769Z ##[endgroup] 2025-12-04T09:50:15.8464245Z + [[ -n '' ]] 2025-12-04T09:50:15.8465987Z + [[ ! -d test/test-reports ]] 2025-12-04T09:50:15.8466317Z + [[ false == \t\r\u\e ]] 2025-12-04T09:50:15.8470856Z + python3 /home/ec2-user/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/upload-benchmark-results/../../scripts/upload_benchmark_results.py --benchmark-results-dir test/test-reports --metadata '{"timestamp": 1764841815, "schema_version": "v3", "name": "periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_huggingface, 1, 1, linux.8xlarge.amx)", "repo": "pytorch/pytorch", "head_branch": "refs/heads/main", "head_sha": "ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32", "workflow_id": 19923066595, "run_attempt": 1, "job_id": 57118563323}' --runners '[{"cpu_info": "x86_64", "cpu_count": 32, "avail_mem_in_gb": 123, "extra_info": {"hostname": "ip-10-0-71-55.ec2.internal"}, "name": "", "type": ""}]' --dependencies '{}' 2025-12-04T09:50:15.9501557Z INFO:root:Upload test/test-reports/inference_huggingface.json to s3://ossci-benchmarks/v3/pytorch/pytorch/19923066595/57118563323/inference_huggingface.json 2025-12-04T09:50:15.9769400Z INFO:botocore.credentials:Found credentials from IAM Role: gh-ci-github-action-runners-runner-role 2025-12-04T09:50:16.2007834Z ##[group]Run cat test/**/*_toprint.log || true 2025-12-04T09:50:16.2008110Z cat test/**/*_toprint.log || true 2025-12-04T09:50:16.2012576Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:50:16.2012800Z env: 2025-12-04T09:50:16.2012954Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:16.2013127Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:16.2013396Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:16.2013698Z DEVICE_NAME: 2025-12-04T09:50:16.2013850Z DEVICE_TYPE: 2025-12-04T09:50:16.2014000Z ##[endgroup] 2025-12-04T09:50:16.2086676Z cat: 'test/**/*_toprint.log': No such file or directory 2025-12-04T09:50:16.2108181Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2025-12-04T09:50:16.2108418Z kill "$MONITOR_SCRIPT_PID" 2025-12-04T09:50:16.2112089Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:50:16.2112316Z env: 2025-12-04T09:50:16.2112461Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:16.2112634Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:16.2113009Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:16.2113391Z DEVICE_NAME: 2025-12-04T09:50:16.2113545Z DEVICE_TYPE: 2025-12-04T09:50:16.2113697Z MONITOR_SCRIPT_PID: 48478 2025-12-04T09:50:16.2113860Z ##[endgroup] 2025-12-04T09:50:16.2133879Z /home/ec2-user/actions-runner/_work/_temp/7a80b7dd-8129-4912-9417-36d186dccd31.sh: line 1: kill: (48478) - No such process 2025-12-04T09:50:16.2143828Z ##[error]Process completed with exit code 1. 2025-12-04T09:50:16.2224264Z Prepare all required actions 2025-12-04T09:50:16.2224559Z Getting action download info 2025-12-04T09:50:16.4195709Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2025-12-04T09:50:16.7022400Z Download action repository 'actions/upload-artifact@v4' (SHA:ea165f8d65b6e75b540449e92b4886f43607fa02) 2025-12-04T09:50:17.2265729Z ##[group]Run ./.github/actions/upload-test-artifacts 2025-12-04T09:50:17.2265961Z with: 2025-12-04T09:50:17.2266236Z file-suffix: test-cpu_inductor_freezing_huggingface-1-1-linux.8xlarge.amx_57118563323 2025-12-04T09:50:17.2266542Z s3-bucket: gha-artifacts 2025-12-04T09:50:17.2266706Z env: 2025-12-04T09:50:17.2266854Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:17.2267026Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:17.2267293Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:17.2267580Z DEVICE_NAME: 2025-12-04T09:50:17.2267762Z DEVICE_TYPE: 2025-12-04T09:50:17.2267899Z ##[endgroup] 2025-12-04T09:50:17.2311258Z ##[group]Run # Remove any previous test jsons if they exist 2025-12-04T09:50:17.2311548Z # Remove any previous test jsons if they exist 2025-12-04T09:50:17.2311783Z rm -f test-jsons-*.zip 2025-12-04T09:50:17.2312049Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test/test-reports -i '*.json' 2025-12-04T09:50:17.2316632Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:50:17.2316878Z env: 2025-12-04T09:50:17.2317030Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:17.2317205Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:17.2317479Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:17.2317771Z DEVICE_NAME: 2025-12-04T09:50:17.2317918Z DEVICE_TYPE: 2025-12-04T09:50:17.2318166Z FILE_SUFFIX: test-cpu_inductor_freezing_huggingface-1-1-linux.8xlarge.amx_57118563323 2025-12-04T09:50:17.2318468Z ##[endgroup] 2025-12-04T09:50:17.2474056Z adding: test/test-reports/inference_huggingface.json (deflated 99%) 2025-12-04T09:50:17.2493848Z ##[group]Run # Remove any previous test reports if they exist 2025-12-04T09:50:17.2494136Z # Remove any previous test reports if they exist 2025-12-04T09:50:17.2494379Z rm -f test-reports-*.zip 2025-12-04T09:50:17.2494668Z zip -r "test-reports-${FILE_SUFFIX}.zip" test/test-reports -i '*.xml' -i '*.csv' 2025-12-04T09:50:17.2498767Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:50:17.2499008Z env: 2025-12-04T09:50:17.2499156Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:17.2499325Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:17.2499591Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:17.2499879Z DEVICE_NAME: 2025-12-04T09:50:17.2500028Z DEVICE_TYPE: 2025-12-04T09:50:17.2500276Z FILE_SUFFIX: test-cpu_inductor_freezing_huggingface-1-1-linux.8xlarge.amx_57118563323 2025-12-04T09:50:17.2500567Z ##[endgroup] 2025-12-04T09:50:17.2549987Z adding: test/test-reports/inference_huggingface.csv (deflated 62%) 2025-12-04T09:50:17.2550542Z adding: test/test-reports/inference_huggingface_graph_breaks.csv (deflated 85%) 2025-12-04T09:50:17.2551056Z adding: test/test-reports/inference_huggingface_graph_break_deduped.csv (deflated 65%) 2025-12-04T09:50:17.2571283Z ##[group]Run # Remove any previous usage logs if they exist 2025-12-04T09:50:17.2571570Z # Remove any previous usage logs if they exist 2025-12-04T09:50:17.2571902Z rm -f logs-*.zip 2025-12-04T09:50:17.2572120Z zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' || true 2025-12-04T09:50:17.2572423Z zip -r "logs-${FILE_SUFFIX}.zip" test/test-reports -i '*.log' || true 2025-12-04T09:50:17.2576075Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:50:17.2576303Z env: 2025-12-04T09:50:17.2576447Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:17.2576611Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:17.2576992Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:17.2577288Z DEVICE_NAME: 2025-12-04T09:50:17.2577429Z DEVICE_TYPE: 2025-12-04T09:50:17.2577695Z FILE_SUFFIX: test-cpu_inductor_freezing_huggingface-1-1-linux.8xlarge.amx_57118563323 2025-12-04T09:50:17.2577988Z ##[endgroup] 2025-12-04T09:50:17.2617413Z adding: usage_log.txt (deflated 58%) 2025-12-04T09:50:17.2631200Z 2025-12-04T09:50:17.2631699Z zip error: Nothing to do! (logs-test-cpu_inductor_freezing_huggingface-1-1-linux.8xlarge.amx_57118563323.zip) 2025-12-04T09:50:17.2646075Z ##[group]Run # Remove any previous debugging artifacts if they exist 2025-12-04T09:50:17.2646406Z # Remove any previous debugging artifacts if they exist 2025-12-04T09:50:17.2646651Z rm -f debug-*.zip 2025-12-04T09:50:17.2646837Z if [ -d 'test/debug' ]; then 2025-12-04T09:50:17.2647065Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2025-12-04T09:50:17.2647274Z fi 2025-12-04T09:50:17.2650685Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:50:17.2650913Z env: 2025-12-04T09:50:17.2651052Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:17.2651222Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:17.2651497Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:17.2651786Z DEVICE_NAME: 2025-12-04T09:50:17.2651939Z DEVICE_TYPE: 2025-12-04T09:50:17.2652199Z FILE_SUFFIX: test-cpu_inductor_freezing_huggingface-1-1-linux.8xlarge.amx_57118563323 2025-12-04T09:50:17.2652482Z ##[endgroup] 2025-12-04T09:50:17.2713453Z ##[group]Run seemethere/upload-artifact-s3@v5 2025-12-04T09:50:17.2713658Z with: 2025-12-04T09:50:17.2713808Z s3-bucket: gha-artifacts 2025-12-04T09:50:17.2714014Z s3-prefix: pytorch/pytorch/19923066595/1/artifact 2025-12-04T09:50:17.2714222Z retention-days: 14 2025-12-04T09:50:17.2714386Z if-no-files-found: warn 2025-12-04T09:50:17.2714562Z path: test-jsons-*.zip 2025-12-04T09:50:17.2714722Z name: artifact 2025-12-04T09:50:17.2714871Z region: us-east-1 2025-12-04T09:50:17.2715019Z env: 2025-12-04T09:50:17.2715152Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:17.2715319Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:17.2715593Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:17.2715875Z DEVICE_NAME: 2025-12-04T09:50:17.2716016Z DEVICE_TYPE: 2025-12-04T09:50:17.2716159Z ##[endgroup] 2025-12-04T09:50:17.5217073Z NOTE: s3-prefix specified, ignoring name parameter 2025-12-04T09:50:17.5220990Z With the provided path, there will be 1 file uploaded 2025-12-04T09:50:17.5221298Z Uploading to s3 prefix: pytorch/pytorch/19923066595/1/artifact 2025-12-04T09:50:17.5247913Z Starting upload of test-jsons-test-cpu_inductor_freezing_huggingface-1-1-linux.8xlarge.amx_57118563323.zip 2025-12-04T09:50:17.6318332Z Finished upload of test-jsons-test-cpu_inductor_freezing_huggingface-1-1-linux.8xlarge.amx_57118563323.zip 2025-12-04T09:50:17.6469866Z ##[group]Run seemethere/upload-artifact-s3@v5 2025-12-04T09:50:17.6470074Z with: 2025-12-04T09:50:17.6470234Z s3-bucket: gha-artifacts 2025-12-04T09:50:17.6470440Z s3-prefix: pytorch/pytorch/19923066595/1/artifact 2025-12-04T09:50:17.6470649Z retention-days: 14 2025-12-04T09:50:17.6470814Z if-no-files-found: error 2025-12-04T09:50:17.6470994Z path: test-reports-*.zip 2025-12-04T09:50:17.6471150Z name: artifact 2025-12-04T09:50:17.6471301Z region: us-east-1 2025-12-04T09:50:17.6471520Z env: 2025-12-04T09:50:17.6471705Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:17.6471875Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:17.6472155Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:17.6472449Z DEVICE_NAME: 2025-12-04T09:50:17.6472593Z DEVICE_TYPE: 2025-12-04T09:50:17.6472742Z ##[endgroup] 2025-12-04T09:50:17.8973983Z NOTE: s3-prefix specified, ignoring name parameter 2025-12-04T09:50:17.8974873Z With the provided path, there will be 1 file uploaded 2025-12-04T09:50:17.8975765Z Uploading to s3 prefix: pytorch/pytorch/19923066595/1/artifact 2025-12-04T09:50:17.9004188Z Starting upload of test-reports-test-cpu_inductor_freezing_huggingface-1-1-linux.8xlarge.amx_57118563323.zip 2025-12-04T09:50:18.0360478Z Finished upload of test-reports-test-cpu_inductor_freezing_huggingface-1-1-linux.8xlarge.amx_57118563323.zip 2025-12-04T09:50:18.0506955Z ##[group]Run seemethere/upload-artifact-s3@v5 2025-12-04T09:50:18.0507165Z with: 2025-12-04T09:50:18.0507342Z s3-bucket: gha-artifacts 2025-12-04T09:50:18.0507546Z s3-prefix: pytorch/pytorch/19923066595/1/artifact 2025-12-04T09:50:18.0507749Z retention-days: 14 2025-12-04T09:50:18.0507912Z if-no-files-found: ignore 2025-12-04T09:50:18.0508082Z path: logs-*.zip 2025-12-04T09:50:18.0508223Z name: artifact 2025-12-04T09:50:18.0508372Z region: us-east-1 2025-12-04T09:50:18.0508519Z env: 2025-12-04T09:50:18.0508653Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:18.0508823Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:18.0509109Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:18.0509426Z DEVICE_NAME: 2025-12-04T09:50:18.0509575Z DEVICE_TYPE: 2025-12-04T09:50:18.0509711Z ##[endgroup] 2025-12-04T09:50:18.2894270Z NOTE: s3-prefix specified, ignoring name parameter 2025-12-04T09:50:18.2896083Z With the provided path, there will be 1 file uploaded 2025-12-04T09:50:18.2896491Z Uploading to s3 prefix: pytorch/pytorch/19923066595/1/artifact 2025-12-04T09:50:18.2925057Z Starting upload of logs-test-cpu_inductor_freezing_huggingface-1-1-linux.8xlarge.amx_57118563323.zip 2025-12-04T09:50:18.3847146Z Finished upload of logs-test-cpu_inductor_freezing_huggingface-1-1-linux.8xlarge.amx_57118563323.zip 2025-12-04T09:50:18.4002391Z ##[group]Run seemethere/upload-artifact-s3@v5 2025-12-04T09:50:18.4002603Z with: 2025-12-04T09:50:18.4002764Z s3-bucket: gha-artifacts 2025-12-04T09:50:18.4002976Z s3-prefix: pytorch/pytorch/19923066595/1/artifact 2025-12-04T09:50:18.4003204Z retention-days: 14 2025-12-04T09:50:18.4003378Z if-no-files-found: ignore 2025-12-04T09:50:18.4003557Z path: debug-*.zip 2025-12-04T09:50:18.4003707Z name: artifact 2025-12-04T09:50:18.4003862Z region: us-east-1 2025-12-04T09:50:18.4004013Z env: 2025-12-04T09:50:18.4004154Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:18.4004329Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:18.4004609Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:18.4004914Z DEVICE_NAME: 2025-12-04T09:50:18.4005057Z DEVICE_TYPE: 2025-12-04T09:50:18.4005205Z ##[endgroup] 2025-12-04T09:50:18.6293823Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2025-12-04T09:50:18.6460877Z ##[group]Run # shellcheck disable=SC2156 2025-12-04T09:50:18.6461127Z # shellcheck disable=SC2156 2025-12-04T09:50:18.6461476Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2025-12-04T09:50:18.6466598Z shell: /usr/bin/bash -e {0} 2025-12-04T09:50:18.6466771Z env: 2025-12-04T09:50:18.6466919Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:18.6467082Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:18.6467356Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:18.6467649Z DEVICE_NAME: 2025-12-04T09:50:18.6467792Z DEVICE_TYPE: 2025-12-04T09:50:18.6467939Z ##[endgroup] 2025-12-04T09:50:18.8202030Z Prepare all required actions 2025-12-04T09:50:18.8202445Z Getting action download info 2025-12-04T09:50:19.0131811Z Download action repository 'actions/setup-python@v6' (SHA:83679a892e2d95755f2dac6acb0bfd1e9ac5d548) 2025-12-04T09:50:19.3968527Z ##[group]Run ./.github/actions/upload-utilization-stats 2025-12-04T09:50:19.3968758Z with: 2025-12-04T09:50:19.3968913Z job_id: 57118563323 2025-12-04T09:50:19.3969251Z job_name: periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_huggingface, 1, 1, linux.8xlarge.amx) 2025-12-04T09:50:19.3969623Z workflow_name: inductor-periodic 2025-12-04T09:50:19.3969807Z workflow_run_id: 19923066595 2025-12-04T09:50:19.3969979Z workflow_attempt: 1 2025-12-04T09:50:19.3970127Z env: 2025-12-04T09:50:19.3970261Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:19.3970431Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:19.3970702Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:19.3971032Z DEVICE_NAME: 2025-12-04T09:50:19.3971180Z DEVICE_TYPE: 2025-12-04T09:50:19.3971332Z ##[endgroup] 2025-12-04T09:50:19.3995326Z ##[group]Run actions/setup-python@v6 2025-12-04T09:50:19.3995526Z with: 2025-12-04T09:50:19.3995701Z python-version: 3.10 2025-12-04T09:50:19.3995856Z check-latest: false 2025-12-04T09:50:19.3996232Z token: *** 2025-12-04T09:50:19.3996403Z update-environment: true 2025-12-04T09:50:19.3996576Z allow-prereleases: false 2025-12-04T09:50:19.3996750Z freethreaded: false 2025-12-04T09:50:19.3996902Z env: 2025-12-04T09:50:19.3997045Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:19.3997206Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:19.3997482Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:19.3997772Z DEVICE_NAME: 2025-12-04T09:50:19.3997913Z DEVICE_TYPE: 2025-12-04T09:50:19.3998058Z ##[endgroup] 2025-12-04T09:50:19.5092653Z ##[group]Installed versions 2025-12-04T09:50:19.5096153Z Version 3.10 was not found in the local cache 2025-12-04T09:50:19.5239598Z (node:68993) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead. 2025-12-04T09:50:19.5243612Z (Use `node --trace-deprecation ...` to show where the warning was created) 2025-12-04T09:50:19.8307712Z ##[error]The version '3.10' with architecture 'x64' was not found for this operating system. The list of all available versions can be found here: https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json 2025-12-04T09:50:19.8442576Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2025-12-04T09:50:19.8442840Z with: 2025-12-04T09:50:19.8442985Z env: 2025-12-04T09:50:19.8443123Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:19.8443301Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:19.8443586Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:19.8443871Z DEVICE_NAME: 2025-12-04T09:50:19.8444020Z DEVICE_TYPE: 2025-12-04T09:50:19.8444159Z ##[endgroup] 2025-12-04T09:50:19.8493518Z ##[group]Run set -eou pipefail 2025-12-04T09:50:19.8493728Z set -eou pipefail 2025-12-04T09:50:19.8493899Z  2025-12-04T09:50:19.8494129Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2025-12-04T09:50:19.8494402Z for _ in $(seq 1440); do 2025-12-04T09:50:19.8494605Z  # Break if no ssh session exists anymore 2025-12-04T09:50:19.8494814Z  if [ "$(who)" = "" ]; then 2025-12-04T09:50:19.8495029Z  break 2025-12-04T09:50:19.8495178Z  fi 2025-12-04T09:50:19.8495323Z  echo "." 2025-12-04T09:50:19.8495475Z  sleep 5 2025-12-04T09:50:19.8495614Z done 2025-12-04T09:50:19.8500319Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:50:19.8500547Z env: 2025-12-04T09:50:19.8500686Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:19.8500855Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:19.8501125Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:19.8501551Z DEVICE_NAME: 2025-12-04T09:50:19.8501704Z DEVICE_TYPE: 2025-12-04T09:50:19.8501851Z ##[endgroup] 2025-12-04T09:50:19.8526000Z Holding runner for 2 hours until all ssh sessions have logged out 2025-12-04T09:50:19.8595301Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2025-12-04T09:50:19.8595625Z # ignore expansion of "docker ps -q" since it could be empty 2025-12-04T09:50:19.8595892Z # shellcheck disable=SC2046 2025-12-04T09:50:19.8596100Z docker stop $(docker ps -q) || true 2025-12-04T09:50:19.8596305Z # Prune all of the docker images 2025-12-04T09:50:19.8596498Z docker system prune -af 2025-12-04T09:50:19.8600331Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:50:19.8600559Z env: 2025-12-04T09:50:19.8600700Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:50:19.8600870Z HAS_NVIDIA_GPU: false 2025-12-04T09:50:19.8601150Z DOCKER_CONTAINER_ID: 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:19.8601438Z DEVICE_NAME: 2025-12-04T09:50:19.8601585Z DEVICE_TYPE: 2025-12-04T09:50:19.8601728Z ##[endgroup] 2025-12-04T09:50:30.4280683Z 267402d4cbe5 2025-12-04T09:50:30.7313860Z Deleted Containers: 2025-12-04T09:50:30.7317993Z 267402d4cbe5327a12de0bd8079bc34384a7cf95e6efe1e960bec3addeb89edb 2025-12-04T09:50:30.7319355Z 2025-12-04T09:50:37.6571362Z Deleted Images: 2025-12-04T09:50:37.6577134Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:50:37.6578159Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image@sha256:b178ee928adfcab963ebbc6ea05ea1a0f0c605bb095e9076f010bf92d150869d 2025-12-04T09:50:37.6578734Z deleted: sha256:40bd34edc3497e05e2150804a15313235b6426bcc4b8c19c03c3daf5524be913 2025-12-04T09:50:37.6579092Z deleted: sha256:947006dce32e090f6b67d556f519398b957121651577f10819c7b179a5b6b938 2025-12-04T09:50:37.6579472Z deleted: sha256:dca16630fc5508e008012b760ac178fac5af74cb679706a3d9898100501d6f8e 2025-12-04T09:50:37.6579834Z deleted: sha256:a35e2e4cfaaa05e1f193fbe67c4151ded52737cb30c75429050cb7cd7777da23 2025-12-04T09:50:37.6580195Z deleted: sha256:d91e1a0a76e64abd7933fc58b8a553d76ac6cd82269c1bf701ade4e2c5ab58ff 2025-12-04T09:50:37.6580561Z deleted: sha256:f96fff40164ffa27ae535a4e5da2bbf2b18ec293366adc73519c830606f07baa 2025-12-04T09:50:37.6581317Z deleted: sha256:649089a3156b98c2cbf16b0af2b1ebdaa8644f9bf90534d9a1f7657b78b9f90e 2025-12-04T09:50:37.6581690Z deleted: sha256:aebabb7ea7535ad2aca44b23eb308b16d5de44f5b412574f94620d923436f42e 2025-12-04T09:50:37.6582045Z deleted: sha256:bcb08dc52ae55732fc33ab8c011347049a9279cc5f45a50d960c95d3a7cd5a3d 2025-12-04T09:50:37.6582406Z deleted: sha256:aa538cda5b22ea6e81fb69326637459a1356cce39b618657a075ce25d9124876 2025-12-04T09:50:37.6582773Z deleted: sha256:e28671ddaf02cbac7c1dcfdb2b08a40d3b7e2dab69c9a9e6c48ede3d8d550d66 2025-12-04T09:50:37.6583139Z deleted: sha256:ba797a4f759946d2fa7414073a5590cc12337aaa43197c35326ce456a3f84adf 2025-12-04T09:50:37.6583481Z deleted: sha256:25b28b66e2a53c3498a56aff9f543bf3a4641534c9e69741c673e9d018c652d8 2025-12-04T09:50:37.6583832Z deleted: sha256:860240198a7e7dcec697a58bcf979cfde64334033f2e97d4c9609277e051558c 2025-12-04T09:50:37.6584183Z deleted: sha256:39de54a9c331b8da952419605b26c35862b4cbb8c8a0967cc536a02b96eda874 2025-12-04T09:50:37.6584528Z deleted: sha256:6f78957316e0887707bc6534ea2aa43d1cd016130ec6a0d3148370b879696e1a 2025-12-04T09:50:37.6584885Z deleted: sha256:8656f7aa172ef2a37ec1524ac6f3ecd2b3fd9675a78a777b9a902d0f26047ea9 2025-12-04T09:50:37.6585247Z deleted: sha256:255344ff6c764e507ea2f21fbbdc6e0a1aaab0e5e0835935494c9df7d620e687 2025-12-04T09:50:37.6585605Z deleted: sha256:b6a1addb4370e1500d2fd65255b80922ad308c95a846a3460be730442d75fbe4 2025-12-04T09:50:37.6585971Z deleted: sha256:8dafa1eff3c9aada8b6725626ba0dd7de62719fb5ff307be3005cf2c6e72e2df 2025-12-04T09:50:37.6586409Z deleted: sha256:9e3fde92e8ff4f05cadac548766ef5ef9e8beb3c89cb48b4e2a17843ff60d649 2025-12-04T09:50:37.6586816Z deleted: sha256:7169a6f70d1f624733e52107b4913effe35e1e63c99388164b0afd2345658178 2025-12-04T09:50:37.6587155Z deleted: sha256:4c0d4c67766b690c1f1834310cd3898a900cd9742b7157de7e8f64d651600e35 2025-12-04T09:50:37.6587506Z deleted: sha256:7343a9c5c68976b0328278e691ddbcf26f619f989ca2a51933ac8bf637d1d69c 2025-12-04T09:50:37.6587869Z deleted: sha256:b790f29be6db3408a1a53fabdc69e76bb38cef221474a4f3aef1e233bb85d6d3 2025-12-04T09:50:37.6588222Z deleted: sha256:ed0831c232a6a410214e113695ab9288063623b523a6032a05b443fa3e9a68ae 2025-12-04T09:50:37.6588552Z deleted: sha256:554ed49d988a9f7a1381d193479542175c40d702a22d246ecef7958a8662a2eb 2025-12-04T09:50:37.6588903Z deleted: sha256:47b04d260bfb2b1f68a2f12aa70926ae2a2a53be6f4d8811d6fbe17fd7f13e17 2025-12-04T09:50:37.6589259Z deleted: sha256:6eb59e87109db04da60df057ff6b232045b6df4e2729ce7534a7a77fbe663761 2025-12-04T09:50:37.6589615Z deleted: sha256:0d9763e3d6e83eee92752c3a5dcfce74703dfa0aba42042761d7600edd95baa5 2025-12-04T09:50:37.6589964Z deleted: sha256:5811882cf3f14a50f47dc91a87d1be09daf3c20494aaf7e7343554066e9f90b7 2025-12-04T09:50:37.6590337Z deleted: sha256:4a6f4f545e9062949802a6c597609ef772633eae7d6925d41823d2b1a8991d4d 2025-12-04T09:50:37.6590674Z deleted: sha256:69a716cb9dc95d7d58d1416f4d127e49885c5d96882dc593686a6eb97200582a 2025-12-04T09:50:37.6591023Z deleted: sha256:0542610effe51f46308d4570438af559e4f6a263f38714ea04c78e4661d1909a 2025-12-04T09:50:37.6591371Z deleted: sha256:86dcdb716a38a00a400b483f42e4ac58714a70e89f5671b50ca5c45d8f577217 2025-12-04T09:50:37.6591735Z deleted: sha256:a9aaf6de8156c1b1a8ece816adbc7d2ed6edabb95e36732737ca293b26619b40 2025-12-04T09:50:37.6592091Z deleted: sha256:57ce7fcbaaa86795f8f7e6290c0aaa8622e13c9c82191d34952c0a4d2e203326 2025-12-04T09:50:37.6592449Z deleted: sha256:a90408b5f4c06a3ba881facc34fa55beb4ae4e51823e7f8d933e611f68465f4a 2025-12-04T09:50:37.6592811Z deleted: sha256:1ac1aab20bc98902f03ee9c657ff435af431f40a81ac9cebf1cd91d9041bc6b1 2025-12-04T09:50:37.6593307Z deleted: sha256:89e6c1c76b21fbe98c153a32627fa1b2337a8caa744b09b28253b67cb84012cb 2025-12-04T09:50:37.6593660Z deleted: sha256:3765d58e17aa81539a8e1eb3cb45914b48f8fb53f7ac12c1bf06c9390e274a93 2025-12-04T09:50:37.6594012Z deleted: sha256:6056a51594e8137d3425100cc233c8ab0f09dded2dc491ba083e8d7c0e1f1143 2025-12-04T09:50:37.6594365Z deleted: sha256:43a55512a68fc9eb9c622a73682dbf9f3c83f1b499287d9e12151f559edabe85 2025-12-04T09:50:37.6594795Z deleted: sha256:a5ad8e88a7e08768a61feeb801fb5a4c5041587feb2a2c77c35b285e869c25c9 2025-12-04T09:50:37.6595150Z deleted: sha256:55fd389078a5792f567a3592821e38fcfa0b0e114fa43248c4528c4d85cf6459 2025-12-04T09:50:37.6595497Z deleted: sha256:144567b58c97f1b5a75d54c3a1514a26ad5ed2dc7988e166316e324a76e76aa6 2025-12-04T09:50:37.6595840Z deleted: sha256:049fbb3123a92952eba68198938b7b29203fcb3d74dfc74e67f9929999576f19 2025-12-04T09:50:37.6596180Z deleted: sha256:22607d21e148d19a45f3f7afe927a15041b9a5663430f5580cc993c4bf9dd7dc 2025-12-04T09:50:37.6596528Z deleted: sha256:63a29311926064245f6d4a4661513586c8179b1cb460ceb91184c0d1114a742d 2025-12-04T09:50:37.6596878Z deleted: sha256:0072101bc0e6bbe2457f4a2d0de953ae643fc2aa6adc69bb59da9cbf6006c8a9 2025-12-04T09:50:37.6597231Z deleted: sha256:35c8f84104fcf39bc1be4682f46dc8b04cb1c5e4c26f66aac039cb42a3f08128 2025-12-04T09:50:37.6597588Z deleted: sha256:c6da65c085b8e72b2e240a3a7d79c4aa166c247737f5916587e51d40acff1ab8 2025-12-04T09:50:37.6597942Z deleted: sha256:b8b5ced5e8550748e26855aad0bb30864c4c435c64694097e63a9a686ef90ba0 2025-12-04T09:50:37.6598294Z deleted: sha256:9905f824759cfae96d29353305bc8a6130a7316db29dfc3efeb8ba9c7cbb5172 2025-12-04T09:50:37.6598721Z deleted: sha256:2b93a73a0fc57cdcfab2a04a30c9b313d84e5e447d076575212f198508242a00 2025-12-04T09:50:37.6599149Z deleted: sha256:baa73842b47530e5d1421010d895ed9059a567516a7e85b10943a6b5fe6fbe41 2025-12-04T09:50:37.6599636Z deleted: sha256:db79627a360d2322d58fb57c98268676011788961d4682ba034f17c9f5fdd855 2025-12-04T09:50:37.6600141Z deleted: sha256:8439f69d5eab9d42b15df43b938f56c690b18cd5e3884f4b80ddc9edf9e43822 2025-12-04T09:50:37.6600553Z deleted: sha256:e95ccdb10d43bbb6f2d6416bca8e5ad7d76a39985431aa5e40ff192ebeb52883 2025-12-04T09:50:37.6600907Z deleted: sha256:d878081c591fd25e8e494f4565e67f341cb3c459b21596ad8e8a707ff9ca63e6 2025-12-04T09:50:37.6601259Z deleted: sha256:2d594947cf73e58a20e36d52acf22f3d31b7a43291d7017c1ac71ea38c4aa9d1 2025-12-04T09:50:37.6601605Z deleted: sha256:933d315b5f20ac941653d01f2b896cc7710564c9b9c4bf9fd8e850115180c6f9 2025-12-04T09:50:37.6601960Z deleted: sha256:0edebbabed274b9e777a5933aaa21ed58569a412c97423c365eff4645e773981 2025-12-04T09:50:37.6602320Z deleted: sha256:744e1cebe181659b0a93ac857d4c16af4c2ce29f3227368eacd7ef18b69e06a7 2025-12-04T09:50:37.6602668Z deleted: sha256:71393f74ab60b183866b9f5995f85aaa65613968c611c8dc15d043befbac662b 2025-12-04T09:50:37.6603007Z deleted: sha256:741a07a29d03bdecf1fb4dfc8f1654a136064f14e5ded89a1497f476ff24623d 2025-12-04T09:50:37.6603356Z deleted: sha256:83df0b5b82e0926d6046012c3b5098dae9aa37177323a33d6eda09e392844b86 2025-12-04T09:50:37.6603716Z deleted: sha256:65b3d6509a37164c67b36965f4ca77ccb1a8e62bdd8f058e2d0886c82ba0940f 2025-12-04T09:50:37.6604073Z deleted: sha256:a38d6cb5dacc51c3e614f4ed42c05cdc53fc489f6e3dafc3b7d0eb39d3250908 2025-12-04T09:50:37.6604434Z deleted: sha256:c3de4f46ac5273a8218c6c5325247b0bd17f12037cfb49f5101a2b5085746704 2025-12-04T09:50:37.6604785Z deleted: sha256:7f3448d255710b28f79767ded93dcf11e0c1a941c7579d8752f4f0bc589aa0b5 2025-12-04T09:50:37.6605133Z deleted: sha256:fd33fe89098b4a8897870a9a316cd91868ab19f42f800d5b512e679356791e62 2025-12-04T09:50:37.6605483Z deleted: sha256:df2c12f62f5ceb2670bf7cb8aa53f268d11dbbfbf25f4b4d567673702cc3fc30 2025-12-04T09:50:37.6606101Z deleted: sha256:a3d7900a6180e4612708429f9c0edba3e999bb43fd67fd3406ae088ea7016a47 2025-12-04T09:50:37.6606463Z deleted: sha256:f592639a1c1bf8da30a1c21866ee22e76b33ae029ee236c7fd20b1ecc4716775 2025-12-04T09:50:37.6606812Z deleted: sha256:e11e4a9756b2bd4a6e62cde476dc36c6528d71a74e8e7570b271784bc494ab40 2025-12-04T09:50:37.6607178Z deleted: sha256:4c03ab513fc4cf698dd61fc37312e57abf7c4e44d4d4ddab126f38804fddec75 2025-12-04T09:50:37.6607542Z deleted: sha256:b4d3d8b5232e0bac87fa78b9c77997cbe8798faffce2aac259b820ab387ce4ad 2025-12-04T09:50:37.6607906Z deleted: sha256:dfda7f6e86c7217a3eb1fe089352248b5d84ca4f62734abce70beb4a6fc90779 2025-12-04T09:50:37.6608257Z deleted: sha256:00f41b846c66bc26a6381fe3663c611a2f539ae3b7e46e007436afd3335a7517 2025-12-04T09:50:37.6608726Z deleted: sha256:0f35c2638249a4525b29668586b079b4ee01f73bbe33732c9da7cfa7b6afb480 2025-12-04T09:50:37.6609080Z deleted: sha256:7441da072ac3269ddb67441b265fdf82904871335b5c19bb381d3622a95b7745 2025-12-04T09:50:37.6609429Z deleted: sha256:ba70851a3aedb994e8d7f8657f8f108bd90a19272843f389e6a8af48a61db720 2025-12-04T09:50:37.6609776Z deleted: sha256:1984ce6f5cabbd8c86386a8d9b5365923b92bc7ccb9aace9b4fe9570213bbbe1 2025-12-04T09:50:37.6610128Z deleted: sha256:73974f74b436f39a2fdb6461b1e3f7c3e41c73325776fa71d16b942a5b4a365b 2025-12-04T09:50:37.6610334Z 2025-12-04T09:50:37.6610433Z Total reclaimed space: 53.28GB 2025-12-04T09:50:37.6670041Z Post job cleanup. 2025-12-04T09:50:37.6698250Z Post job cleanup. 2025-12-04T09:50:37.7760217Z (node:69097) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead. 2025-12-04T09:50:37.7761745Z (Use `node --trace-deprecation ...` to show where the warning was created) 2025-12-04T09:50:37.7875303Z Post job cleanup. 2025-12-04T09:50:37.7902756Z Post job cleanup. 2025-12-04T09:50:37.8692803Z [command]/usr/bin/git version 2025-12-04T09:50:37.8729316Z git version 2.50.1 2025-12-04T09:50:37.8762106Z Copying '/home/ec2-user/.gitconfig' to '/home/ec2-user/actions-runner/_work/_temp/0fc51e48-5b41-42c6-9ddd-f81d75fc7b98/.gitconfig' 2025-12-04T09:50:37.8776792Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/0fc51e48-5b41-42c6-9ddd-f81d75fc7b98' before making global git config changes 2025-12-04T09:50:37.8777661Z Adding repository directory to the temporary git global config as a safe directory 2025-12-04T09:50:37.8782672Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2025-12-04T09:50:37.8832698Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2025-12-04T09:50:37.8864897Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2025-12-04T09:50:37.9164527Z Entering 'android/libs/fbjni' 2025-12-04T09:50:37.9220942Z Entering 'third_party/FP16' 2025-12-04T09:50:37.9271975Z Entering 'third_party/FXdiv' 2025-12-04T09:50:37.9325941Z Entering 'third_party/NNPACK' 2025-12-04T09:50:37.9379630Z Entering 'third_party/NVTX' 2025-12-04T09:50:37.9433724Z Entering 'third_party/VulkanMemoryAllocator' 2025-12-04T09:50:37.9489760Z Entering 'third_party/XNNPACK' 2025-12-04T09:50:37.9553243Z Entering 'third_party/aiter' 2025-12-04T09:50:37.9609700Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-12-04T09:50:37.9670135Z Entering 'third_party/benchmark' 2025-12-04T09:50:37.9723963Z Entering 'third_party/composable_kernel' 2025-12-04T09:50:37.9783200Z Entering 'third_party/cpp-httplib' 2025-12-04T09:50:37.9834994Z Entering 'third_party/cpuinfo' 2025-12-04T09:50:37.9890319Z Entering 'third_party/cudnn_frontend' 2025-12-04T09:50:37.9945414Z Entering 'third_party/cutlass' 2025-12-04T09:50:38.0003554Z Entering 'third_party/fbgemm' 2025-12-04T09:50:38.0059025Z Entering 'third_party/fbgemm/external/asmjit' 2025-12-04T09:50:38.0112703Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-12-04T09:50:38.0169946Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-12-04T09:50:38.0222191Z Entering 'third_party/fbgemm/external/cutlass' 2025-12-04T09:50:38.0283505Z Entering 'third_party/fbgemm/external/googletest' 2025-12-04T09:50:38.0335714Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-12-04T09:50:38.0388720Z Entering 'third_party/fbgemm/external/json' 2025-12-04T09:50:38.0446984Z Entering 'third_party/flash-attention' 2025-12-04T09:50:38.0497902Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-12-04T09:50:38.0561302Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-12-04T09:50:38.0619846Z Entering 'third_party/flatbuffers' 2025-12-04T09:50:38.0673183Z Entering 'third_party/fmt' 2025-12-04T09:50:38.0733284Z Entering 'third_party/gemmlowp/gemmlowp' 2025-12-04T09:50:38.0785204Z Entering 'third_party/gloo' 2025-12-04T09:50:38.0836077Z Entering 'third_party/googletest' 2025-12-04T09:50:38.0891264Z Entering 'third_party/ideep' 2025-12-04T09:50:38.0942800Z Entering 'third_party/ideep/mkl-dnn' 2025-12-04T09:50:38.0999300Z Entering 'third_party/ittapi' 2025-12-04T09:50:38.1052913Z Entering 'third_party/kineto' 2025-12-04T09:50:38.1102945Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-12-04T09:50:38.1153948Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-12-04T09:50:38.1206159Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-12-04T09:50:38.1260176Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-12-04T09:50:38.1310844Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-12-04T09:50:38.1361581Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-12-04T09:50:38.1421550Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-12-04T09:50:38.1472698Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-12-04T09:50:38.1524185Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-12-04T09:50:38.1578391Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-12-04T09:50:38.1629826Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp' 2025-12-04T09:50:38.1681649Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:50:38.1735343Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:50:38.1795181Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-12-04T09:50:38.1848810Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-12-04T09:50:38.1903194Z Entering 'third_party/kleidiai' 2025-12-04T09:50:38.1956981Z Entering 'third_party/mimalloc' 2025-12-04T09:50:38.2013315Z Entering 'third_party/nlohmann' 2025-12-04T09:50:38.2065067Z Entering 'third_party/onnx' 2025-12-04T09:50:38.2131367Z Entering 'third_party/onnx/third_party/pybind11' 2025-12-04T09:50:38.2189429Z Entering 'third_party/opentelemetry-cpp' 2025-12-04T09:50:38.2245255Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-12-04T09:50:38.2299135Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-12-04T09:50:38.2352560Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-12-04T09:50:38.2399888Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-12-04T09:50:38.2455253Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-12-04T09:50:38.2506467Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-12-04T09:50:38.2558942Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-12-04T09:50:38.2606917Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:50:38.2661778Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:50:38.2719195Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-12-04T09:50:38.2788751Z Entering 'third_party/pocketfft' 2025-12-04T09:50:38.2843208Z Entering 'third_party/protobuf' 2025-12-04T09:50:38.2899048Z Entering 'third_party/protobuf/third_party/benchmark' 2025-12-04T09:50:38.2952318Z Entering 'third_party/protobuf/third_party/googletest' 2025-12-04T09:50:38.3004359Z Entering 'third_party/psimd' 2025-12-04T09:50:38.3059396Z Entering 'third_party/pthreadpool' 2025-12-04T09:50:38.3113491Z Entering 'third_party/pybind11' 2025-12-04T09:50:38.3162628Z Entering 'third_party/python-peachpy' 2025-12-04T09:50:38.3212965Z Entering 'third_party/sleef' 2025-12-04T09:50:38.3266668Z Entering 'third_party/tensorpipe' 2025-12-04T09:50:38.3313339Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-12-04T09:50:38.3366842Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-12-04T09:50:38.3420308Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-12-04T09:50:38.3471836Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-12-04T09:50:38.3522668Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-12-04T09:50:38.3603094Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2025-12-04T09:50:38.3622998Z http.https://github.com/.extraheader 2025-12-04T09:50:38.3630094Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2025-12-04T09:50:38.3664703Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2025-12-04T09:50:38.3955312Z Entering 'android/libs/fbjni' 2025-12-04T09:50:38.3987782Z http.https://github.com/.extraheader 2025-12-04T09:50:38.4020498Z Entering 'third_party/FP16' 2025-12-04T09:50:38.4053322Z http.https://github.com/.extraheader 2025-12-04T09:50:38.4088302Z Entering 'third_party/FXdiv' 2025-12-04T09:50:38.4122881Z http.https://github.com/.extraheader 2025-12-04T09:50:38.4159670Z Entering 'third_party/NNPACK' 2025-12-04T09:50:38.4190034Z http.https://github.com/.extraheader 2025-12-04T09:50:38.4224491Z Entering 'third_party/NVTX' 2025-12-04T09:50:38.4256806Z http.https://github.com/.extraheader 2025-12-04T09:50:38.4292808Z Entering 'third_party/VulkanMemoryAllocator' 2025-12-04T09:50:38.4325212Z http.https://github.com/.extraheader 2025-12-04T09:50:38.4357196Z Entering 'third_party/XNNPACK' 2025-12-04T09:50:38.4390518Z http.https://github.com/.extraheader 2025-12-04T09:50:38.4436390Z Entering 'third_party/aiter' 2025-12-04T09:50:38.4469472Z http.https://github.com/.extraheader 2025-12-04T09:50:38.4503467Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-12-04T09:50:38.4536174Z http.https://github.com/.extraheader 2025-12-04T09:50:38.4581476Z Entering 'third_party/benchmark' 2025-12-04T09:50:38.4615436Z http.https://github.com/.extraheader 2025-12-04T09:50:38.4650400Z Entering 'third_party/composable_kernel' 2025-12-04T09:50:38.4682814Z http.https://github.com/.extraheader 2025-12-04T09:50:38.4724335Z Entering 'third_party/cpp-httplib' 2025-12-04T09:50:38.4755060Z http.https://github.com/.extraheader 2025-12-04T09:50:38.4792471Z Entering 'third_party/cpuinfo' 2025-12-04T09:50:38.4824816Z http.https://github.com/.extraheader 2025-12-04T09:50:38.4859282Z Entering 'third_party/cudnn_frontend' 2025-12-04T09:50:38.4894741Z http.https://github.com/.extraheader 2025-12-04T09:50:38.4926578Z Entering 'third_party/cutlass' 2025-12-04T09:50:38.4959585Z http.https://github.com/.extraheader 2025-12-04T09:50:38.4998368Z Entering 'third_party/fbgemm' 2025-12-04T09:50:38.5033419Z http.https://github.com/.extraheader 2025-12-04T09:50:38.5067006Z Entering 'third_party/fbgemm/external/asmjit' 2025-12-04T09:50:38.5097973Z http.https://github.com/.extraheader 2025-12-04T09:50:38.5135432Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-12-04T09:50:38.5166907Z http.https://github.com/.extraheader 2025-12-04T09:50:38.5203373Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-12-04T09:50:38.5238084Z http.https://github.com/.extraheader 2025-12-04T09:50:38.5273283Z Entering 'third_party/fbgemm/external/cutlass' 2025-12-04T09:50:38.5307375Z http.https://github.com/.extraheader 2025-12-04T09:50:38.5353371Z Entering 'third_party/fbgemm/external/googletest' 2025-12-04T09:50:38.5385928Z http.https://github.com/.extraheader 2025-12-04T09:50:38.5420210Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-12-04T09:50:38.5453560Z http.https://github.com/.extraheader 2025-12-04T09:50:38.5487838Z Entering 'third_party/fbgemm/external/json' 2025-12-04T09:50:38.5520806Z http.https://github.com/.extraheader 2025-12-04T09:50:38.5560930Z Entering 'third_party/flash-attention' 2025-12-04T09:50:38.5592846Z http.https://github.com/.extraheader 2025-12-04T09:50:38.5626311Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-12-04T09:50:38.5659470Z http.https://github.com/.extraheader 2025-12-04T09:50:38.5698440Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-12-04T09:50:38.5733996Z http.https://github.com/.extraheader 2025-12-04T09:50:38.5780597Z Entering 'third_party/flatbuffers' 2025-12-04T09:50:38.5815530Z http.https://github.com/.extraheader 2025-12-04T09:50:38.5853585Z Entering 'third_party/fmt' 2025-12-04T09:50:38.5886378Z http.https://github.com/.extraheader 2025-12-04T09:50:38.5923270Z Entering 'third_party/gemmlowp/gemmlowp' 2025-12-04T09:50:38.5957218Z http.https://github.com/.extraheader 2025-12-04T09:50:38.5990377Z Entering 'third_party/gloo' 2025-12-04T09:50:38.6024301Z http.https://github.com/.extraheader 2025-12-04T09:50:38.6058986Z Entering 'third_party/googletest' 2025-12-04T09:50:38.6092102Z http.https://github.com/.extraheader 2025-12-04T09:50:38.6131659Z Entering 'third_party/ideep' 2025-12-04T09:50:38.6162141Z http.https://github.com/.extraheader 2025-12-04T09:50:38.6195153Z Entering 'third_party/ideep/mkl-dnn' 2025-12-04T09:50:38.6234390Z http.https://github.com/.extraheader 2025-12-04T09:50:38.6278784Z Entering 'third_party/ittapi' 2025-12-04T09:50:38.6313905Z http.https://github.com/.extraheader 2025-12-04T09:50:38.6348705Z Entering 'third_party/kineto' 2025-12-04T09:50:38.6381565Z http.https://github.com/.extraheader 2025-12-04T09:50:38.6415116Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-12-04T09:50:38.6447308Z http.https://github.com/.extraheader 2025-12-04T09:50:38.6482735Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-12-04T09:50:38.6516589Z http.https://github.com/.extraheader 2025-12-04T09:50:38.6554733Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-12-04T09:50:38.6588467Z http.https://github.com/.extraheader 2025-12-04T09:50:38.6623494Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-12-04T09:50:38.6655378Z http.https://github.com/.extraheader 2025-12-04T09:50:38.6691760Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-12-04T09:50:38.6726582Z http.https://github.com/.extraheader 2025-12-04T09:50:38.6761259Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-12-04T09:50:38.6793617Z http.https://github.com/.extraheader 2025-12-04T09:50:38.6836361Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-12-04T09:50:38.6868158Z http.https://github.com/.extraheader 2025-12-04T09:50:38.6903464Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-12-04T09:50:38.6936637Z http.https://github.com/.extraheader 2025-12-04T09:50:38.6972266Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-12-04T09:50:38.7003433Z http.https://github.com/.extraheader 2025-12-04T09:50:38.7040607Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-12-04T09:50:38.7071891Z http.https://github.com/.extraheader 2025-12-04T09:50:38.7106800Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp' 2025-12-04T09:50:38.7141966Z http.https://github.com/.extraheader 2025-12-04T09:50:38.7173211Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:50:38.7204820Z http.https://github.com/.extraheader 2025-12-04T09:50:38.7244425Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:50:38.7278270Z http.https://github.com/.extraheader 2025-12-04T09:50:38.7318090Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-12-04T09:50:38.7350749Z http.https://github.com/.extraheader 2025-12-04T09:50:38.7385048Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-12-04T09:50:38.7419572Z http.https://github.com/.extraheader 2025-12-04T09:50:38.7458249Z Entering 'third_party/kleidiai' 2025-12-04T09:50:38.7491829Z http.https://github.com/.extraheader 2025-12-04T09:50:38.7529461Z Entering 'third_party/mimalloc' 2025-12-04T09:50:38.7562704Z http.https://github.com/.extraheader 2025-12-04T09:50:38.7597125Z Entering 'third_party/nlohmann' 2025-12-04T09:50:38.7632006Z http.https://github.com/.extraheader 2025-12-04T09:50:38.7667679Z Entering 'third_party/onnx' 2025-12-04T09:50:38.7702709Z http.https://github.com/.extraheader 2025-12-04T09:50:38.7750358Z Entering 'third_party/onnx/third_party/pybind11' 2025-12-04T09:50:38.7785436Z http.https://github.com/.extraheader 2025-12-04T09:50:38.7822730Z Entering 'third_party/opentelemetry-cpp' 2025-12-04T09:50:38.7856589Z http.https://github.com/.extraheader 2025-12-04T09:50:38.7891360Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-12-04T09:50:38.7922152Z http.https://github.com/.extraheader 2025-12-04T09:50:38.7957734Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-12-04T09:50:38.7987498Z http.https://github.com/.extraheader 2025-12-04T09:50:38.8022576Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-12-04T09:50:38.8055979Z http.https://github.com/.extraheader 2025-12-04T09:50:38.8091984Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-12-04T09:50:38.8124984Z http.https://github.com/.extraheader 2025-12-04T09:50:38.8163262Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-12-04T09:50:38.8194271Z http.https://github.com/.extraheader 2025-12-04T09:50:38.8226447Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-12-04T09:50:38.8260100Z http.https://github.com/.extraheader 2025-12-04T09:50:38.8293877Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-12-04T09:50:38.8325664Z http.https://github.com/.extraheader 2025-12-04T09:50:38.8357998Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:50:38.8390520Z http.https://github.com/.extraheader 2025-12-04T09:50:38.8426943Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:50:38.8458171Z http.https://github.com/.extraheader 2025-12-04T09:50:38.8494901Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-12-04T09:50:38.8533022Z http.https://github.com/.extraheader 2025-12-04T09:50:38.8582639Z Entering 'third_party/pocketfft' 2025-12-04T09:50:38.8615368Z http.https://github.com/.extraheader 2025-12-04T09:50:38.8650205Z Entering 'third_party/protobuf' 2025-12-04T09:50:38.8684021Z http.https://github.com/.extraheader 2025-12-04T09:50:38.8720717Z Entering 'third_party/protobuf/third_party/benchmark' 2025-12-04T09:50:38.8752934Z http.https://github.com/.extraheader 2025-12-04T09:50:38.8788009Z Entering 'third_party/protobuf/third_party/googletest' 2025-12-04T09:50:38.8821462Z http.https://github.com/.extraheader 2025-12-04T09:50:38.8859949Z Entering 'third_party/psimd' 2025-12-04T09:50:38.8893361Z http.https://github.com/.extraheader 2025-12-04T09:50:38.8930277Z Entering 'third_party/pthreadpool' 2025-12-04T09:50:38.8962078Z http.https://github.com/.extraheader 2025-12-04T09:50:38.8994523Z Entering 'third_party/pybind11' 2025-12-04T09:50:38.9028546Z http.https://github.com/.extraheader 2025-12-04T09:50:38.9067855Z Entering 'third_party/python-peachpy' 2025-12-04T09:50:38.9102861Z http.https://github.com/.extraheader 2025-12-04T09:50:38.9138226Z Entering 'third_party/sleef' 2025-12-04T09:50:38.9169546Z http.https://github.com/.extraheader 2025-12-04T09:50:38.9204014Z Entering 'third_party/tensorpipe' 2025-12-04T09:50:38.9241328Z http.https://github.com/.extraheader 2025-12-04T09:50:38.9272972Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-12-04T09:50:38.9303970Z http.https://github.com/.extraheader 2025-12-04T09:50:38.9341168Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-12-04T09:50:38.9373332Z http.https://github.com/.extraheader 2025-12-04T09:50:38.9406931Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-12-04T09:50:38.9439857Z http.https://github.com/.extraheader 2025-12-04T09:50:38.9473954Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-12-04T09:50:38.9506990Z http.https://github.com/.extraheader 2025-12-04T09:50:38.9539128Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-12-04T09:50:38.9571350Z http.https://github.com/.extraheader 2025-12-04T09:50:38.9632459Z [command]/usr/bin/git config --local --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:38.9679401Z [command]/usr/bin/git submodule foreach --recursive git config --local --show-origin --name-only --get-regexp remote.origin.url 2025-12-04T09:50:38.9985411Z Entering 'android/libs/fbjni' 2025-12-04T09:50:39.0010186Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2025-12-04T09:50:39.0023696Z Entering 'third_party/FP16' 2025-12-04T09:50:39.0045389Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2025-12-04T09:50:39.0061368Z Entering 'third_party/FXdiv' 2025-12-04T09:50:39.0083313Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2025-12-04T09:50:39.0099009Z Entering 'third_party/NNPACK' 2025-12-04T09:50:39.0122945Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2025-12-04T09:50:39.0139116Z Entering 'third_party/NVTX' 2025-12-04T09:50:39.0161718Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NVTX/config remote.origin.url 2025-12-04T09:50:39.0177007Z Entering 'third_party/VulkanMemoryAllocator' 2025-12-04T09:50:39.0197294Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2025-12-04T09:50:39.0218717Z Entering 'third_party/XNNPACK' 2025-12-04T09:50:39.0239055Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2025-12-04T09:50:39.0266733Z Entering 'third_party/aiter' 2025-12-04T09:50:39.0287179Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/aiter/config remote.origin.url 2025-12-04T09:50:39.0301336Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-12-04T09:50:39.0328181Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/aiter/modules/3rdparty/composable_kernel/config remote.origin.url 2025-12-04T09:50:39.0351708Z Entering 'third_party/benchmark' 2025-12-04T09:50:39.0373239Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2025-12-04T09:50:39.0390068Z Entering 'third_party/composable_kernel' 2025-12-04T09:50:39.0413542Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/composable_kernel/config remote.origin.url 2025-12-04T09:50:39.0437645Z Entering 'third_party/cpp-httplib' 2025-12-04T09:50:39.0459813Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2025-12-04T09:50:39.0477394Z Entering 'third_party/cpuinfo' 2025-12-04T09:50:39.0499986Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2025-12-04T09:50:39.0513386Z Entering 'third_party/cudnn_frontend' 2025-12-04T09:50:39.0537976Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2025-12-04T09:50:39.0554111Z Entering 'third_party/cutlass' 2025-12-04T09:50:39.0575605Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2025-12-04T09:50:39.0596301Z Entering 'third_party/fbgemm' 2025-12-04T09:50:39.0620482Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2025-12-04T09:50:39.0638325Z Entering 'third_party/fbgemm/external/asmjit' 2025-12-04T09:50:39.0659553Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/asmjit/config remote.origin.url 2025-12-04T09:50:39.0679126Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-12-04T09:50:39.0698278Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/composable_kernel/config remote.origin.url 2025-12-04T09:50:39.0724130Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-12-04T09:50:39.0744387Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/cpuinfo/config remote.origin.url 2025-12-04T09:50:39.0764979Z Entering 'third_party/fbgemm/external/cutlass' 2025-12-04T09:50:39.0784607Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/cutlass/config remote.origin.url 2025-12-04T09:50:39.0804600Z Entering 'third_party/fbgemm/external/googletest' 2025-12-04T09:50:39.0828667Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/googletest/config remote.origin.url 2025-12-04T09:50:39.0846976Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-12-04T09:50:39.0866589Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/hipify_torch/config remote.origin.url 2025-12-04T09:50:39.0881519Z Entering 'third_party/fbgemm/external/json' 2025-12-04T09:50:39.0901771Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/json/config remote.origin.url 2025-12-04T09:50:39.0923175Z Entering 'third_party/flash-attention' 2025-12-04T09:50:39.0943552Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/config remote.origin.url 2025-12-04T09:50:39.0958842Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-12-04T09:50:39.0980865Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/modules/csrc/composable_kernel/config remote.origin.url 2025-12-04T09:50:39.0998822Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-12-04T09:50:39.1023640Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/modules/csrc/cutlass/config remote.origin.url 2025-12-04T09:50:39.1046924Z Entering 'third_party/flatbuffers' 2025-12-04T09:50:39.1069141Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2025-12-04T09:50:39.1085310Z Entering 'third_party/fmt' 2025-12-04T09:50:39.1105252Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2025-12-04T09:50:39.1124997Z Entering 'third_party/gemmlowp/gemmlowp' 2025-12-04T09:50:39.1145580Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2025-12-04T09:50:39.1163358Z Entering 'third_party/gloo' 2025-12-04T09:50:39.1185450Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2025-12-04T09:50:39.1201056Z Entering 'third_party/googletest' 2025-12-04T09:50:39.1222939Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2025-12-04T09:50:39.1239258Z Entering 'third_party/ideep' 2025-12-04T09:50:39.1259876Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2025-12-04T09:50:39.1275488Z Entering 'third_party/ideep/mkl-dnn' 2025-12-04T09:50:39.1294648Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2025-12-04T09:50:39.1322767Z Entering 'third_party/ittapi' 2025-12-04T09:50:39.1343621Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2025-12-04T09:50:39.1363119Z Entering 'third_party/kineto' 2025-12-04T09:50:39.1383192Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2025-12-04T09:50:39.1398824Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-12-04T09:50:39.1422240Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2025-12-04T09:50:39.1439637Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-12-04T09:50:39.1459762Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2025-12-04T09:50:39.1479466Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-12-04T09:50:39.1500060Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2025-12-04T09:50:39.1516352Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-12-04T09:50:39.1537579Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2025-12-04T09:50:39.1553096Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-12-04T09:50:39.1575186Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2025-12-04T09:50:39.1590309Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-12-04T09:50:39.1613869Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2025-12-04T09:50:39.1635338Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-12-04T09:50:39.1655657Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2025-12-04T09:50:39.1672962Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-12-04T09:50:39.1693712Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2025-12-04T09:50:39.1712198Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-12-04T09:50:39.1732334Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2025-12-04T09:50:39.1750500Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-12-04T09:50:39.1772276Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2025-12-04T09:50:39.1790336Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp' 2025-12-04T09:50:39.1812706Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/prometheus-cpp/config remote.origin.url 2025-12-04T09:50:39.1829124Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:50:39.1851435Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2025-12-04T09:50:39.1871014Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:50:39.1893098Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2025-12-04T09:50:39.1916176Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-12-04T09:50:39.1935490Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2025-12-04T09:50:39.1952015Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-12-04T09:50:39.1974797Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2025-12-04T09:50:39.1993175Z Entering 'third_party/kleidiai' 2025-12-04T09:50:39.2017720Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kleidiai/config remote.origin.url 2025-12-04T09:50:39.2036182Z Entering 'third_party/mimalloc' 2025-12-04T09:50:39.2057439Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2025-12-04T09:50:39.2073497Z Entering 'third_party/nlohmann' 2025-12-04T09:50:39.2095923Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2025-12-04T09:50:39.2112007Z Entering 'third_party/onnx' 2025-12-04T09:50:39.2135691Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2025-12-04T09:50:39.2164522Z Entering 'third_party/onnx/third_party/pybind11' 2025-12-04T09:50:39.2185985Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2025-12-04T09:50:39.2203391Z Entering 'third_party/opentelemetry-cpp' 2025-12-04T09:50:39.2227449Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2025-12-04T09:50:39.2243679Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-12-04T09:50:39.2265136Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2025-12-04T09:50:39.2283932Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-12-04T09:50:39.2303635Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2025-12-04T09:50:39.2322554Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-12-04T09:50:39.2344582Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2025-12-04T09:50:39.2358192Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-12-04T09:50:39.2379933Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2025-12-04T09:50:39.2396287Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-12-04T09:50:39.2417963Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2025-12-04T09:50:39.2436491Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-12-04T09:50:39.2456500Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2025-12-04T09:50:39.2472588Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-12-04T09:50:39.2494710Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2025-12-04T09:50:39.2507378Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:50:39.2530600Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2025-12-04T09:50:39.2548867Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:50:39.2570123Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2025-12-04T09:50:39.2589926Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-12-04T09:50:39.2611295Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2025-12-04T09:50:39.2648423Z Entering 'third_party/pocketfft' 2025-12-04T09:50:39.2669023Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2025-12-04T09:50:39.2685770Z Entering 'third_party/protobuf' 2025-12-04T09:50:39.2706376Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2025-12-04T09:50:39.2725606Z Entering 'third_party/protobuf/third_party/benchmark' 2025-12-04T09:50:39.2747274Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2025-12-04T09:50:39.2762842Z Entering 'third_party/protobuf/third_party/googletest' 2025-12-04T09:50:39.2785651Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2025-12-04T09:50:39.2804261Z Entering 'third_party/psimd' 2025-12-04T09:50:39.2825438Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2025-12-04T09:50:39.2843719Z Entering 'third_party/pthreadpool' 2025-12-04T09:50:39.2865674Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2025-12-04T09:50:39.2883216Z Entering 'third_party/pybind11' 2025-12-04T09:50:39.2903394Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2025-12-04T09:50:39.2923316Z Entering 'third_party/python-peachpy' 2025-12-04T09:50:39.2943378Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2025-12-04T09:50:39.2958824Z Entering 'third_party/sleef' 2025-12-04T09:50:39.2980585Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2025-12-04T09:50:39.2996590Z Entering 'third_party/tensorpipe' 2025-12-04T09:50:39.3019491Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2025-12-04T09:50:39.3034111Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-12-04T09:50:39.3057016Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2025-12-04T09:50:39.3074157Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-12-04T09:50:39.3094137Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2025-12-04T09:50:39.3109862Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-12-04T09:50:39.3135021Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2025-12-04T09:50:39.3151125Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-12-04T09:50:39.3171157Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2025-12-04T09:50:39.3185999Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-12-04T09:50:39.3207724Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2025-12-04T09:50:39.3254264Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3289645Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3323718Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3351071Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3380559Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NVTX/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3403119Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3431577Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3459214Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/aiter/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3485720Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/aiter/modules/3rdparty/composable_kernel/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3508375Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3538623Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/composable_kernel/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3571458Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3601031Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3627700Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3656484Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3686210Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3712104Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/asmjit/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3737178Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/composable_kernel/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3765881Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/cpuinfo/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3790786Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/cutlass/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3823872Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/googletest/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3857764Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/hipify_torch/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3884373Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/json/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3916174Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3940539Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/modules/csrc/composable_kernel/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3971230Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/modules/csrc/cutlass/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.3991202Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4023030Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4052861Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4073646Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4098168Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4132514Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4150663Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4177404Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4206494Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4239926Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4269886Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4294619Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4327719Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4353047Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4385787Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4425003Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4458808Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4484155Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4516101Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4549669Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/prometheus-cpp/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4580100Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/prometheus-cpp/modules/civetweb/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4604849Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/prometheus-cpp/modules/googletest/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4632494Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4664821Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4688209Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kleidiai/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4719418Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4743530Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4768153Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4791961Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4827371Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4857419Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4882615Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4907126Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4936472Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4970965Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.4993471Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.5028406Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.5058739Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.5088960Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.5120181Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.5149750Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.5177848Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.5199704Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.5228451Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.5259802Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.5288888Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.5309990Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.5344688Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.5373137Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.5398169Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.5426425Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.5459352Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.5487374Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.5514186Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.5543445Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:50:39.5653036Z A job completed hook has been configured by the self-hosted runner administrator 2025-12-04T09:50:39.5664658Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2025-12-04T09:50:39.5667738Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:50:39.5667991Z ##[endgroup] 2025-12-04T09:50:39.5751401Z [!ALERT!] Swap in detected! [!ALERT!] 2025-12-04T09:50:47.7441128Z [!ALERT!] Swap out detected [!ALERT!] 2025-12-04T09:51:01.7963996Z Cleaning up orphan processes